USDA Funded McGovern-Dole International Food for Education Liberia Empowerment through Attendance, Reading, and Nutrition (LEARN) 2018 – 2022 Second Cohort Baseline Project Evaluation Report November 2018 Authors: Elnaz Safarha, M.S. Daniel Zaas, M.I.D.P Kajal Gulati, Ph.D. Jessica Li, M.I.D.P Submitted by: Project Director: Elnaz Safarha Submitted to: Chief of Party: Dina Rakotomalala DISCLAIMER: This evaluation was conducted by an independent third party firm. The views expressed in this report do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. USDA Funded McGovern-Dole International Food for Education Liberia Empowerment through Attendance, Reading, and Nutrition (LEARN) 2018 – 2022 Second Cohort Baseline Project Evaluation Report Program: McGovern-Dole International Food for Education Agreement Number: FFE-669-2017/032-00 Funding Year: Fiscal Year 2017 Project Duration: 2018-2022 Implemented by: Save the Children Evaluation Authored by: IMPAQ International LLC Elnaz Safarha Daniel Zaas Kajal Gulati Jessica Li Accessibility Note: An accessible version of this report can be made available upon request. Please contact FAS.Monitoring.Evaluation@usda.gov IMPAQ International, LLC Page i LEARN Second Cohort Baseline Report ACKNOWLEDGMENTS The authors would like to thank Save the Children (SC) and the U.S. Department of Agriculture (USDA) for their financial support. We also would like to thank the Ministry of Education of Liberia and its County Education offices in Grand Bassa, Rivercess, and River Gee counties for supporting and facilitating the rollout of LEARN data collection. We have very much enjoyed our collaboration with SC and SC’s Liberia office, and would like to thank Clay Westrope, Dina Rakotomalala, Timothy Grimes, Morris Gbessagee, and Job Milapo. We also thank James Samai Mussa for his great support on the fieldwork and school assessment. We thank the Center for Action Research and Training (CART) team for its professionalism and flexibility with data collection; we also thank all the enumerators for their excellent work in the field. We acknowledge Kou Johnson, the managing director of CART, for overseeing the data collection, collaborating with the Ministry of Education, and managing the fieldwork team. The findings, interpretations, and conclusions expressed in this document are entirely those of the authors. They do not necessarily represent the views of IMPAQ International, LLC; SC; or USDA. IMPAQ International, LLC Page ii LEARN Second Cohort Baseline Report TABLE OF CONTENTS Acknowledgments .......................................................................................................................................... i Table of Contents........................................................................................................................................... ii Table of Exhibits............................................................................................................................................. iv Acronym List… ...............................................................................................................................................vi Executive Summary....................................................................................................................................... vii Overview .................................................................................................................................................. vii Key Baseline Findings................................................................................................................................ vii Best Practice Recommendations.............................................................................................................. viii Section 1. Introduction................................................................................................................................... 1 Section 2. Evaluation Approach...................................................................................................................... 3 2.1 Research Questions.............................................................................................................................. 3 2.2 Project Evaluation Methodology .......................................................................................................... 3 2.3 Modifications to the Sampling Design .................................................................................................. 4 2.4 Data Sources........................................................................................................................................ 4 2.4.1 Student Survey and Literacy Boost Reading Assessment........................................................................ 4 2.4.2 School Assessment .................................................................................................................................. 4 2.5 Data Analysis........................................................................................................................................ 5 Section 3. Fieldwork....................................................................................................................................... 6 3.1 Human Subjects Protection .................................................................................................................. 6 3.2 Field Data Collection ............................................................................................................................ 6 3.2.1 Preparation.............................................................................................................................................. 6 3.2.2 Data Collection ........................................................................................................................................ 6 3.2.3 Logistical Challenges................................................................................................................................ 7 Section 4. Project Evaluation Baseline Values................................................................................................. 8 4.1 Project Evaluation Sample.................................................................................................................... 9 4.1.1 Composition and Characteristics of School Sample ................................................................................ 9 4.1.2 Composition and Characteristics of Student Sample .............................................................................. 9 4.2 Student Reading Outcomes.................................................................................................................11 4.2.1 Availability of Reading Materials...........................................................................................................11 4.2.2 Home Literacy Environment..................................................................................................................13 4.2.3 Student Attitudes Toward Schooling.....................................................................................................14 4.2.4 Presence of Teachers in School .............................................................................................................14 4.2.5 Reading Outcomes ................................................................................................................................15 IMPAQ International, LLC Page iii LEARN Second Cohort Baseline Report 4.3 Other Key Student Outcomes..............................................................................................................18 4.3.1 Hygiene and Handwashing Practices.....................................................................................................19 4.3.2 Nutrition Practices and Knowledge.......................................................................................................19 4.3.3 Sexual and Gender-Based Violence and Gender Norms.......................................................................21 4.3.4 Disability ................................................................................................................................................25 Section 5. Proposed Approach to Midline and Endline ..................................................................................26 5.1 Comparing Baselines...........................................................................................................................26 5.2 Midline and Endline Sampling Plan......................................................................................................26 5.3 Midline and Endline Analysis Plan .......................................................................................................27 Section 6. Conclusion ....................................................................................................................................28 6.1 Summary of Key Outcomes .................................................................................................................28 6.2 Limitations..........................................................................................................................................29 6.3 Best Practice Recommendations .........................................................................................................29 Appendix A: LEARN Results Framework ....................................................................................................... A1 Appendix B: McGovern-Dole Performance Indicators................................................................................... B1 Appendix C: Additional Tables and Complementary Outcomes .................................................................... C1 Appendix D: Other Subtests of Reading Assessment .................................................................................... D1 Appendix E: Regression Analyses .................................................................................................................. E1 Appendix F: Inter-Rater Reliability ................................................................................................................ F1 Appendix G. Evaluation Instruments............................................................................................................ G1 IMPAQ International, LLC Page iv LEARN Second Cohort Baseline Report TABLE OF EXHIBITS Exhibit 1. Project Evaluation Sample Sizes in Each County ..................................................................................... 3 Exhibit 2. Overview of Student Survey Key Topics.................................................................................................. 4 Exhibit 3. Baseline Levels for Key Project Indicators Required by PMP per County ............................................... 8 Exhibit 4. Numbers of Sampled Schools by County................................................................................................. 9 Exhibit 5. Locations of Sampled Schools ................................................................................................................. 9 Exhibit 6. Number of Students Surveyed by County ............................................................................................... 9 Exhibit 7. Gender and Grade Distribution of Students Surveyed.......................................................................... 10 Exhibit 8. Age and Grade Distribution of Students Surveyed................................................................................ 10 Exhibit 9. Household Possession of Durable Goods by County............................................................................. 11 Exhibit 10. Access to Non-Textbook Reading Materials in School ........................................................................ 12 Exhibit 11. Frequency with Which Students Borrowed Non-Textbook Reading Materials to Take Home........... 12 Exhibit 12. Availability of Reading Materials in the Home by County................................................................... 12 Exhibit 13. Household Literacy Activities in the Past Week .................................................................................. 13 Exhibit 14. Teacher Attendance by County ........................................................................................................... 14 Exhibit 15. Teacher Tardiness by County .............................................................................................................. 15 Exhibit 16. Second-Grade Students’ Literacy Skills by County .............................................................................. 16 Exhibit 17. Distribution of Letter Recognition Scores ........................................................................................... 17 Exhibit 18. Comprehension Subtests by County ................................................................................................... 18 Exhibit 19. Student Knowledge vs. Practice of Critical Handwashing Moments by County ................................. 19 Exhibit 20. Availability of Free School Meals by County ....................................................................................... 20 Exhibit 21. Students’ Eating Frequency................................................................................................................. 20 Exhibit 22. Students’ Perceptions of How Food Should Be Allocated to Boys and Girls....................................... 21 Exhibit 23. Student Identification of Rules to Guide Teacher Behavior................................................................ 22 Exhibit 24. Types of School Discipline by County .................................................................................................. 22 Exhibit 25. Student Perceptions of Gender Norms ............................................................................................... 24 Exhibit 26. Students who Disagreed with 4 of 5 Gender Norms by County.......................................................... 24 Exhibit 27. Proportion of Students Reporting No Visual, Oratory, or Physical Difficulties................................... 25 Exhibit 28. Midline and Endline Analysis............................................................................................................... 27 Exhibit 29. Program Activity Packages .................................................................................................................. A3 Exhibit 30. Main Language by County ................................................................................................................... C1 Exhibit 31. Proportion of Students who Read Non-Textbooks in the Last Week Outside School by County ....... C1 Exhibit 32. Type of Household Members Providing Literacy and Learning Support to Students......................... C1 Exhibit 33. Reasons Students Like Their School and Class .................................................................................... C2 IMPAQ International, LLC Page v LEARN Second Cohort Baseline Report Exhibit 34. Reasons Students Dislike Their School and Class................................................................................ C2 Exhibit 35. Students’ Knowledge vs. Practice of Critical Handwashing Practice by County (Breakdown)............ C3 Exhibit 36. Types of School Discipline by Grade.................................................................................................... C3 Exhibit 37. Perceptions of Gender Norms by County............................................................................................ C3 Exhibit 38. Comprehension and Reading Skills by Language ................................................................................ C4 Exhibit 39. Most Recognized Word by County ...................................................................................................... D1 Exhibit 40. Distribution of Most Used Words Identified by County...................................................................... D1 Exhibit 41. Invented Word Recognition ................................................................................................................ D2 Exhibit 42. Summary of Predictors of Literacy Skills at Baseline........................................................................... E1 Exhibit 43. Predictors of Literacy Skills at Baseline ............................................................................................... E2 Exhibit 44. Associations Between School Characteristics and Literacy Skills (Performance Sample)................... E3 Exhibit 45. IRR by Literacy Skill Subtests for Performance Sample ........................................................................ F1 IMPAQ International, LLC Page vi LEARN Second Cohort Baseline Report ACRONYM LIST CART Center for Action Research and Training DEO EGRA District Education Officer Early Grade Reading Assessment EMIS Educational Management Information System ICC Intra-cluster correlation IRB Institutional Review Board IRR Inter-rater Reliability LB Literacy Boost LBRA Literacy Boost Reading Assessment LD Liberian Dollars LEARN Liberia Empowerment through Attendance, Reading, and Nutrition MDE Minimum detectable effect MOE NGO Ministry of Education Non-governmental Organization PMP Performance Monitoring Plan PTA Parent-teacher association SC Save the Children SD Standard deviation SF School Feeding Base Package SGBV Sexual and gender-based violence SHN School health and nutrition TOR UNICEF Terms of reference United Nations International Children’s Emergency Fund UL-PIRE University of Liberia Pacific Institute for Research and Evaluation USDA U.S. Department of Agriculture WFP World Food Programme IMPAQ International, LLC Page vii LEARN Second Cohort Baseline Report EXECUTIVE SUMMARY Overview The U.S. Department of Agriculture (USDA), through the McGovern-Dole International Food for Education and Child Nutrition Program, has funded Save the Children (SC) to implement the Liberia Empowerment through Attendance and Reading (LEARN) project. SC in Liberia is implementing the LEARN program in partnership with Mercy Corps and with the Liberian Ministry of Education, Ministry of Agriculture, and Ministry of Health. This five-year project (1 October 2017 – 30 September 2022) aims to improve literacy outcomes for school age children and children’s attentiveness and attendance by decreasing short-term hunger (Strategic Objective 1) and improving health and dietary practices (Strategic Objective 2) with various activities such as school feeding, take-home rations, teacher training, provision of school supplies and reading materials, establishment of school gardens, and distribution of deworming medications, vitamins, and minerals. SC selected IMPAQ International (IMPAQ) to design and conduct impact and project evaluations of the LEARN program in four Liberian counties (Grand Bassa, Grand Gedeh, Rivercess, and River Gee) at baseline (2018), midline (2020), and endline (2022). For the project evaluation, due to modifications in SC’s implementation plan, the LEARN program added a second cohort of students across 85 schools in Grand Bassa, Rivercess, and River Gee (excluding Grand Gedeh, the site of the impact evaluation). This cohort of the LEARN program is planning to implement the same activities as the first cohort to achieve the strategic objectives mentioned above. IMPAQ visited 61 accessible and active schools out of the 85 additional schools to benchmark pre￾implementation values and to confirm indicator targets in literacy and in hygiene, health, and nutrition practices and knowledge. As with the first baseline evaluation, to answer the following research questions, IMPAQ conducted student surveys that included the Literacy Boost Reading Assessment (LBRA), as well as questions about water, sanitation, and hygiene (WASH), nutrition, gender norms, and sexual and gender-based violence (SGBV) knowledge and practices: 1. What are the baseline levels in letter identification and phonemic awareness among Grade 2 students? 2. What are the baseline levels in Grade 2 students’ reading and understanding of second-grade-level text? 3. What are the baseline levels in students’ knowledge and practices toward SGBV? 4. What are the baseline levels in Grade 6 students’ perceptions of gender norms? 5. What are the baseline levels in students’ handwashing knowledge and practices? 6. What are the baseline levels in students’ nutrition knowledge? IMPAQ also used the same school assessment , as was done in the first round of baseline, to assess the pre￾implementation characteristics, enrollment, and attendance in all 61 LEARN schools that were accessible for data collection. IMPAQ developed this checklist to help the program establish pre-implementation targets and indicators based on schools’ characteristics and to verify and update previously existing school data. Key Baseline Findings Below, IMPAQ highlights the second cohort baseline data findings most pertinent to the key research questions. Please refer to Section 4. Project Evaluation Baseline Values for details on the quantitative data. Project Evaluation Key Outcomes  Literacy. The evaluation team found that 81 percent of Grade 2 students could identify 21–26 letters, with an average of 22. However, students struggled with reading proficiency and comprehension, regardless of county or gender. Only 12 percent of surveyed second-graders were identified as readers, i.e., read at least five words correctly in 30 seconds, and 42 percent of them were able to answer at least 80 percent of the comprehension questions correctly (reading with comprehension). These results IMPAQ International, LLC Page viii LEARN Second Cohort Baseline Report confirmed the low reading proficiency of Grade 2 students at grade level at the end of the school year before the intervention was implemented.  Home environment. The majority of students (55 percent) said that someone in their household helped them study; 44 percent stated that someone read to them; and 40 percent that they saw someone reading.  Nutrition. Students lacked sufficient knowledge of a healthy diet; none of them could correctly identify the three types of foods considered as constituting a healthy diet, defined by the project as go, glow, and grow foods.  Handwashing. A high proportion of sampled students (88 percent) said that they had washed their hands during the day prior to the survey. The survey also gathered information on student knowledge and practice of handwashing at three critical moments: after using the toilet to defecate, after using the toilet to urinate, and before consuming food. Although nine percent of students said they should wash their hands at these moments, only six percent responded that they actually did.  Sexual and gender-based violence. To gauge students’ understanding of SGBV, as well as their willingness and ability to report such incidents, we examined the proportion of students who reported that they understood school rules and codes of appropriate conduct; said that they would report any cases of inappropriate teasing or touching; and listed any type of corporal or psychological teacher discipline. Our data showed that 78 percent of students responded that rules exist for how teachers should treat students at school. Furthermore, students in all grades stated that they would willingly report inappropriate teasing or touching.  Gender norms. We established a benchmark that considered students to be aware of gender norms if they disagreed with at least four of five statements related to stereotypical gender perceptions. Less than half of students (42 percent) disagreed with at least four statements, but this percentage was different for girls (50 percent) and boys (37 percent). A regional analysis of the data revealed county variations. A smaller proportion of students in River Gee disagreed with the gender statementsregarding stereotypes that were read to them, compared to Grand Bassa and River Gee. Best Practice Recommendations Overall, except for the reading outcomes, almost all of the quantitative results in the second baseline evaluation were similar to those in the first round. This suggests that recommendations from the first cohort are applicable to the second cohort. However, based on our experience in the field, and our analysis of the second cohort’s baseline data, and comparison to the first cohort’s data, we recommend the following to SC to enhance future evaluations, data collections, and program implementation.  Ensure enough heterogeneity across reading outcomes for midline and endline sample. Most of the indicators that IMPAQ measured in the second round of data collection showed similar results to the first round, except for the reading outcomes. IMPAQ will take into account variation in the reading outcomes in selecting the 85 schools (among the 130 visited in first and second rounds) for the project evaluation at midline and endline to ensure a more representative sample in measuring the reading outcomes over time.  Ensure the midline and endline data collection will be conducted during the dry season. High water on rivers and impassible roads caused by heavy rains in both rounds of baseline data collection severely limited the team’s ability to collect data efficiently and effectively. Even in the accessible schools, the rain limited students’ attendance, which impeded the team’s ability to meet the evaluation target sample. IMPAQ in coordination with SC should collect data for both midline and endline during the dry season. IMPAQ International, LLC Page ix LEARN Second Cohort Baseline Report  Consider including strategies for teaching decoding skills into the BLA training. Almost none of the second-grade students in both rounds of baseline evaluations were able to decode the made-up words, which could explain the low results on word recognitions. Decoding and encoding of words are the basic skills for reading with comprehension. To improve children’s reading outcomes at earlier grades, teachers should be trained on these techniques.  Educate parents as well as teachers on positive discipline. Although no qualitative assessment was performed for this report as agreed with SC, the data from the second cohort reaffirms our recommendation in the first report for a focus on educating teachers and parents on the benefits of positive discipline. In both baseline evaluation cohorts, a large majority of students, especially in second grade, reported teachers’ use of physical violence as a school discipline. We recommend the program to focus on improving teachers’ attitudes to create an atmosphere conducive to learning for students, in addition to strengthening teachers’ pedagogical practices. IMPAQ International, LLC Page 1 LEARN Second Cohort Baseline Report SECTION 1. INTRODUCTION The U.S. Department of Agriculture (USDA), through the McGovern-Dole International Food for Education and Child Nutrition Program, has funded Save the Children (SC) to implement the Liberia Empowerment through Attendance and Reading (LEARN) project. This five-year project (1 October 2017 – 30 September 2022) aims to improve literacy outcomes of school-age children and to enhance children’s attentiveness and attendance by decreasing short-term hunger and increasing the use of health and dietary practices. In this report, IMPAQ describes findings from a second baseline project evaluation conducted for the LEARN project after LEARN expanded to 85 additional schools in Grand Bassa, Rivercess, and River Gee in August 2018. In this section, we describe the rationale for this second baseline and how the findings from these two cohorts will feed into the midline and endline evaluations. Due to modifications in SC’s implementation plan, the LEARN program added a second cohort of students across 85 schools for the project evaluation in three counties: Grand Bassa, River Gee, and Rivercess. The LEARN program will implement the same activities in this cohort as in the first one to achieve the following strategic objectives:  Improve the literacy of school-age children by enhancing the quality of instruction and increasing students’ attentiveness and attendance; and,  Improve students’ health and dietary outcomes by enhancing their knowledge of health and hygiene best practices, upgrading sanitation facilities, and improving food safety and storage systems. Appendix A: LEARN Results Framework shows a snapshot of the results framework as well as the LEARN activities. To maximize comparability between the two cohorts, IMPAQ used the same methodology and tools as used in the first baseline study to address the following objectives (modified from the first baseline to exclude impact evaluation goals): 1. Benchmark pre-implementation values and confirm indicator targets in literacy and in hygiene, health, and nutrition practice and knowledge 2. Confirm project design assumptions 3. Identify potential threats to project implementation IMPAQ surveyed students in Grades 2 and 6 and administered the same reading assessment to measure the performance indicators similar to the first cohort and as requested by the TOR. Later, at midline and endline, IMPAQ will select a representative sample of 69 schools from the 130 schools (69 from the first round and 61 from the second round) visited for the project evaluation in both rounds of the baseline evaluation in Grand Bassa, River Gee, and Rivercess. To construct a sample of total 85 schools, we will also include the 16 schools in Grand Gedeh that were selected for the impact evaluation component. 1 We will use a probability proportional to size sampling method to have a more representative sample at midline/endline. Because IMPAQ has set benchmark values in a similar manner for both set of schools at baseline, we will be able to measure changes in the performance indicators at midline and endline using the combined set of 85 schools. Section 5 explains our strategy for sampling and data collection at midline and endline in greater detail for the performance evaluation component. Since the second round of the evaluation did not include any impact evaluation, the evaluation design (especially the 55 schools selected for the evaluation) will remain unchanged. To present our findings, we structured the report as follows: Section 2. Evaluation Approach briefly describes the evaluation, including research questions, the sampling design and its modification, data tools, and baseline data analysis; Section 3. Fieldwork describes the data collection fieldwork; Section 4. Project Evaluation Baseline Values describes the key characteristics of the project evaluation sample, as well as 1 16 out of 55 schools for impact evaluation in Grand Gedeh are also included in the project evaluation sample. We will revisit all of these 16 schools at midline and endline in addition to 69 selected schools at midline for the performance evaluation. IMPAQ International, LLC Page 2 LEARN Second Cohort Baseline Report performance indicator values for the second cohort; Section 5. Proposed Approach to Midline and Endline compares the results in first and second cohorts and proposes our strategy for midline and endline to account for the differences; and Section 6. Conclusion ends the report with lessons learned, study limitations, and recommendations. IMPAQ International, LLC Page 3 LEARN Second Cohort Baseline Report SECTION 2. EVALUATION APPROACH In this section, IMPAQ describes the quantitative approach for the second project baseline evaluation, including research questions, sampling design, data sources, and data analysis. To conduct this evaluation, we used the same evaluation approach as the first baseline to ensure the comparability of both baselines with midline and endline. Unlike the first cohort, however, the second cohort baseline evaluation did not include a qualitative assessment. 2.1 Research Questions To benchmark pre-implementation values and to confirm indicator targets in literacy and in practice, and students’ knowledge of hygiene, health, and nutrition, we answered the same quantitative evaluation research questions as the first cohort: 1. What are the baseline levels in Grade 2 students’ letter identification and phonemic awareness? 2. What are the baseline levels in Grade 2 students’ reading and understanding of second-grade-level text? 3. What are the baseline levels in students’ knowledge and practices toward SGBV? 4. What are the baseline levels in Grade 6 students’ perceptions of gender norms? 5. What are the baseline levels in students’ handwashing knowledge and practices? 6. What are the baseline levels in students’ nutrition knowledge? 2.2 Project Evaluation Methodology We designed the project evaluation to measure performance indicators in outcomes related to core LEARN activities at three points in time: baseline, midline, and endline. To accurately reflect changes in the program’s performance over time, we measure the same program indicators at all three data collection points. At the second project baseline evaluation, we measured the performance indicators consistently with the first round. IMPAQ followed the same sampling strategy from the first baseline to select students from among the 85 schools that SC identified for the team. IMPAQ intended to select 10 students (5 boys and 5 girls) randomly from second grade and 6 students (3 boys and 3 girls) from sixth grade. Exhibit 1 shows the representative sample of schools identified by SC in each county for the second cohort. Exhibit 1. Project Evaluation Sample Sizes in Each County County Evaluation Schools Sampled Second Graders (10 per school) Sampled Sixth Graders (6 per school) Grand Bassa 63 630 378 River Gee 7 70 42 Rivercess 15 150 90 Total 85 850 510 To select participants randomly, IMPAQ physically lined up the boys and girls separately in their classrooms. To identify the nth student for random selection, we used the following rule: 𝑛𝑛𝑡𝑡ℎ𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 𝑜𝑜 𝑏𝑏 𝑏𝑏 𝑡𝑡 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑠𝑠 = 𝑇𝑇 𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 𝑜𝑜 𝑏𝑏 𝑏𝑏 𝑒𝑒ℎ 𝑔𝑔𝑔𝑔 𝑔𝑔 𝑇𝑇 𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 𝑜𝑜 𝑏𝑏 𝑏𝑏 𝑡𝑡 𝑏𝑏 𝑠𝑠𝑠𝑠 𝑠𝑠 𝑠𝑠 For example, if there were 20 female second graders and we required 5 for the study, we selected every other girl from the line (20 ÷ 5 = 4). The team used the same rule to select students systematically from all sampled schools and both genders. In the absence of electronic class lists, this approach ensured sampling consistency across schools and achieved a random sample of students who were present on the day of data collection. IMPAQ International, LLC Page 4 LEARN Second Cohort Baseline Report 2.3 Modifications to the Sampling Design We received from SC a list of 85 schools targeted for program activities for the second cohort with 17 extra schools as replacements based on government-provided EMIS data. During the data collection, however, because some schools were inaccessible due to rainy seasons; students’ enrollment for some schools were inflated on the EMIS data; and information on the status of some schools were communicated inaccurately to our field team by district offices. Because of this, we ended up with a smaller sample of 61 schools instead of 85. The team was able to mitigate these challenges by finding replacement schools and students. However, the project sample might have been affected by the types of schools and students who remained in the sample as described above. 2.4 Data Sources For the second cohort baseline evaluation, IMPAQ administered the same studentsurvey, including the LBRA, which was developed, cognitively tested and used for the first round of data collection. In addition, we used the same school assessment to provide SC with the pre-implementation characteristics, enrollment, and attendance of the newly added LEARN schools. 2.4.1 Student Survey and Literacy Boost Reading Assessment The student survey collected data on five key topics from Grade 2 and Grade 6 students to set benchmark values for the project evaluation. Exhibit 2 presents an overview of the key survey topics. Exhibit 2. Overview of Student Survey Key Topics Topic Types of Questions Background information  Demographic information (e.g., students’ age, main language spoken at home, etc.) Hygiene and health knowledge and practices  Handwashing knowledge (when one should wash hands)  Handwashing practices (when students actually wash their hands) Nutrition knowledge  Knowledge of a healthy diet (what a balanced diet is) Sexual and gender-based violence  Knowledge of SGBV  Perceived gender norms (Grade 6 only) School environment  Attitudes toward their school  Teacher attendance Home environment  Home literacy activities (e.g., if anyone reads to students or tells a story)  Reading culture at home Disability  Difficulty in seeing, hearing, walking, etc. Along with the student survey, IMPAQ fielded the same LBRA, which we developed based on the Liberian second-grade textbooks, and the Ministry of Education validated it in a one-day workshop to measure Grade 2 students’ literacy skills. The test comprised four subtests, including letter recognition, reading familiar words, reading unfamiliar words, and reading or listening with comprehension. 2.4.2 School Assessment IMPAQ used the same school assessment, which we developed in the first round, for SC to assess the pre￾implementation characteristics, enrollment, and attendance in the newly added second cohort LEARN schools that were accessible for data collection. This school assessment included the following:  GPS coordinates  Student enrollment, based on school registers  Student attendance, based on teacher attendance logs  School structure (permanent or temporary) and building materials (mud, concrete, etc.)  Water, sanitation, and hygiene resources (availability of latrines, drinking water, handwashing stations) IMPAQ International, LLC Page 5 LEARN Second Cohort Baseline Report  Canteen and food storage status  Libraries IMPAQ developed this checklist to help the program establish pre-implementation targets and indicators based on schools’ characteristics, and to verify and update previously existing school data. Before each visit, CART (our data collection partner) and IMPAQ consultants coordinated with the school district office and principals. In collaboration with CART, we cleaned the collected data, identified the active selected schools (i.e., open and operating schools), and used geospatial information measures to locate them on maps of the three counties along with data on their relevant characteristics. IMPAQ shared the cleaned data with SC for analysis and use. 2.5 Data Analysis After completing field activities, we conducted a final review of the survey data, including the following activities:  Check for data completeness  Check for duplicate entries  Skip pattern logic of tablet survey programming  Data cleaning We then compiled the survey responses into a master file for the analysis. This baseline report provides summary statistics, as well as constructed outcomes (percentages and averages) using individual or multiple survey items through the statistical software package Stata, for the second cohort baseline evaluation. When possible, we also conducted subgroup analyses by grade, students’ gender, and county, highlighting emerging patterns. The project evaluation results are explained in Section 4. Project Evaluation Baseline Values. IMPAQ International, LLC Page 6 LEARN Second Cohort Baseline Report SECTION 3. FIELDWORK In this section, IMPAQ describes the activities that the evaluation team conducted before, during, and after data collection in the field. We also discuss challenges that the team faced during data collection. 3.1 Human Subjects Protection Prior to collecting data for the first round of baseline evaluation, IMPAQ sought IRB approval to ensure that the proposed evaluation complied with local and international rules and procedures. IMPAQ submitted protocol documents to Chesapeake Institutional Review Board (IRB), now merged with Schulman IRB as Advarra, on February 16, 2018 (Pro00024481), and to the University of Liberia Pacific Institute for Research and Evaluation IRB (UL-PIRE IRB, Protocol # 18-02-092) on February 15, 2018. IMPAQ also submitted research protocol to the SCUS Ethics Review Committee to confirm that the research is ethically sound and safeguards the rights, safety, and well-being of children. IMPAQ received approval from Chesapeake (February 16, 2018), UL-PIRE IRB (February 22, 2018) and SCUS (February 26, 2018). For the second round of baseline evaluation, to collect data, IMPAQ used the IRB-approved instruments, as well as informed consent and assent forms. Prior to administering the survey, IMPAQ trained enumerators on procedures for interviewing respondents, protecting respondents’ privacy and confidentiality, and securing the data. IMPAQ also reprised the training from SCUS on safeguarding children at school. During the data collection, the survey team first obtained teachers’ and principals’ consent to survey their students. The team then asked for students’ assent, assuring children that their participation was voluntary and that they could terminate the survey at any point. After data collection, the evaluation team protected the privacy and confidentiality of respondents by storing the data on secure servers and separating personally-identifiable information from the survey data. 3.2 Field Data Collection 3.2.1 Preparation Training and Pilot Testing In September 2018, the IMPAQ team trained 20 enumerators and 2 supervisors, recruited by our data collection partner CART, to conduct quantitative data collection. To enhance the efficiency of second cohort baseline data collection, CART rehired 11 of the 22 enumerators who collected first round baseline data. We conducted two days of classroom training to explain: (1) what each survey question intends to ask, (2) how to ask those questions of vulnerable respondents (students in Grade 2 and 6), (3) how to measure students’ literacy, and (4) how to use tablets to implement the in-person surveys offline without an internet connection. After the initial training, the IMPAQ team provided an opportunity for enumerators to practice with real respondents for a day in one school in Margibi county. Like the first round of baseline pilot testing, the school was not part of the LEARN evaluation, but its characteristics2 were similar to those schools in the evaluation sample. After this pilot, all enumerators regrouped with the IMPAQ team to debrief and discuss issues that they encountered. IMPAQ updated the tools to ensure smooth and consistent survey implementation. 3.2.2 Data Collection On September 24, 2018, CART organized the enumerators into 2 teams of 10 individuals and 1 supervisor. One team was sent to Grand Bassa, which had the largest sample, while the other team visited schools in the other two counties (Rivercess and River Gee). The data collection supervisors, in collaboration with the MOE, school district offices, and the local community coordinated their school visits with school principals. All enumerators regrouped with their supervisors several times during the data collection to debrief, submit 2 Some of the characteristics include proximity to main roads, accessibility to latrine, and availability of electricity. Students’ language skills in the pilot school were also similar to students in Grand Bassa, which contained the majority of sampled schools. IMPAQ International, LLC Page 7 LEARN Second Cohort Baseline Report daily paper-based data collection logs, submit electronic surveys, and review and plan for the next days of data collection. The CART fieldwork manager also worked closely with the survey teams to oversee data quality and provided enumerators with technical support. This manager updated the IMPAQ project director every other day. The team completed fieldwork in 18 working days. 3.2.3 Logistical Challenges Due to the rainy season and the inflated EMIS data, CART’s team leaders worked closely with the school district offices, local communities, and principals to get updates on the status of the targeted schools, their accessibility, and the availability of students. Where possible, CART, under the supervision of IMPAQ, oversampled students in larger schools. Inaccessible schools were replaced by accessible schools in all counties based on the replacement lists that SC provided. IMPAQ International, LLC Page 8 LEARN Second Cohort Baseline Report SECTION 4. PROJECT EVALUATION BASELINE VALUES In this section, IMPAQ presents summary statistics from the student survey, including the LBRA. We examine the baseline data for variations in outcomes by gender, grade, and county, highlighting differences of more than 10 percent.3 Appendix C: Additional Tables and Complementary Outcomes and Appendix D: Other Subtests of Reading Assessment provide additional detail. Note that self-reported data on culturally and socially sensitive topics, such as handwashing and hygiene, gender norms, and SGBV, may be subject to social desirability bias. Social desirability bias is the tendency of research subjects to give socially desirable responses instead of choosing responses that reflect their true feelings on socially sensitive issues. Therefore, these results should be interpreted with caution. Exhibit 3 presents a summary of indicators and, where applicable, disaggregates by grade, gender, and county, key project evaluation outcomes required by the performance monitoring plan. For a complete table of baseline levels for key McGovern-Dole performance indicators, please refer to Appendix B: McGovern￾Dole Performance Indicators. Exhibit 3. Baseline Levels for Key Project Indicators Required by PMP per County Gender Baseline Aggregate (Second Cohort) Grand Bassa Rivercess River Gee Percentage of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand grade level texta Girls 1% 1% 0% 3% Boys 1% 1% 0% 0% Percentage of students who, by the end of two grades of primary schooling, demonstrate proficiency in identifying lettersa Girls 51% 51% 58% 35% Boys 53% 57% 50% 35% Percentage of children in target schools who demonstrate improved knowledge and practices toward SGBV prevention and responsea Girls 60% 59% 70% 51% Boys 64% 65% 62% 59% Percentage of students in Grades 2 and 6 in target schools who can identify the components of a healthy diet Grade 2 Girls 0% 0% 0% 0% Boys 0% 0% 0% 0% Grade 6 Girls 0% 0% 0% 0% Boys 0% 0% 0% 0% Source: Student survey. IMPAQ calculation. Note: The survey included 614 second graders and 238 sixth graders, 489 males and 363 females. aOnly includes Grade 2 students per requirements of the PMP. We report details on these results in this section. First, we consider the characteristics of schools and students sampled for the project evaluation. We then discuss students’ reading outcomes, including factors such as home and school literacy environment in addition to LBRA results. We conclude the section by reviewing baseline values in handwashing and hygiene, nutrition, SGBV, and disability. 3 All the percentages in Section 4. Project Evaluation Baseline Values are rounded to the nearest whole number. IMPAQ International, LLC Page 9 LEARN Second Cohort Baseline Report 4.1 Project Evaluation Sample 4.1.1 Composition and Characteristics of School Sample To set benchmark values for performance indicators and to measure progress toward desired outcomes over time, the team selected 605 Grade 2 students and 234 Grade 6 students across 61 schools in 3 counties of Liberia: Grand Bassa, Rivercess, and River Gee. Although we aimed to survey 10 students in Grade 2 and 6 students in Grade 6 in each selected school, low enrollment and attendance rates and other challenges in the field required the survey team to oversample students in larger schools (regardless of their gender) and to replace selected schools that were inaccessible because of rain with accessible nearby program schools to reach the target. See 2.3 Modifications to the Sampling Design for more detail. Exhibit 4 summarizes the number of schools per county in the project sample; Exhibit 5 provides a map of these locations overlaid with the schools visited for the first cohort. Exhibit 4. Numbers of Sampled Schools by County Grand Bassa Rivercess River Gee Total Number of schools 48 9 4 61 Exhibit 5. Locations of Sampled Schools Source: IMPAQ calculation. 4.1.2 Composition and Characteristics of Student Sample Composition of Student Sample As explained in Section 2.2 Project Evaluation Methodology, the number of students that the team surveyed per school varied from the intended 10 students in Grade 2 and 6 students in Grade 6. Exhibit 6 shows the total numbers of students who participated in the student survey by county. Exhibit 6. Number of Students Surveyed by County Grade Grand Bassa Rivercess River Gee Total Grade 2 422 121 71 614 Grade 6 150 45 43 238 Total 572 166 114 852 Source: IMPAQ calculation. IMPAQ International, LLC Page 10 LEARN Second Cohort Baseline Report Further disaggregating this information by grade and gender, we see that the project sample has a slightly unbalanced gender ratio, as shown in Exhibit 7. Among the 605 Grade 2 students, 57 percent were male; and 58 percent of the 234 Grade 6 students were male. Exhibit 7. Gender and Grade Distribution of Students Surveyed Grade Male Female Total Percent Number Percent Number Grade 2 57% 352 43% 262 614 Grade 6 58% 137 42% 101 238 Total 57% 489 43% 363 852 Source: Student survey, IMPAQ calculation. Exhibit 8 shows the age distribution of surveyed students by grade. Grade 2 students averaged 12 years of age (both median and mean), with a range of 5 to 19 years. The large age gap and high average age could result from a government policy enacted in 2001 that mandated primary education for children and eliminated fees.4 Before the enactment of this law, the high price of education and 14 years of civil conflict deterred parents from sending their children to school.5 After the new law was passed, many parents enrolled their children in school regardless of age. The age spread for Grade 6 is slightly larger than for Grade 2. These age distributions do not vary considerably by county or gender. Exhibit 8. Age and Grade Distribution of Students Surveyed Grade Age Median Mean Youngest Oldest Grade 2 12 12 5 19 Grade 6 16 16 10 25 Source: Student survey, IMPAQ calculation. The survey asked students whether they had ever repeated a grade: 42 percent of students said they had, with no sizeable differences by grade, gender, or county. This statistic suggests that the high average age of students is not necessarily due to having repeated grades but also to starting school late. The correlation between age and repeating a grade was weak among all students. Additionally, a high percentage of students (86 percent) said they had attended school for 5 days in the past week. This rate remained high in each county although it ranged from 79 percent in River Gee to 93 percent in Rivercess. There were no large differences when disaggregated by grade or gender. Student Characteristics To better understand the children’s backgrounds, the survey asked about students’ household size, their primary caregiver, their socioeconomic status, and the language they spoke at home. The average household size, reported by second graders across counties was eight; River Gee had a slightly lower average size of about seven.6 The spread between minimum and maximum household sizes was quite large, between 2 and 24 people. However, this finding should be interpreted with caution. Large household sizes may be explained by students’ defining a household differently, and that, in rural areas, different families often live together as one community or household. On average, 70 percent of students said that their mother was their caregiver. Female students were slightly more likely to report their mother as caregiver (75 percent) than males (67 percent); meanwhile, 21 percent of males reported their father as caregiver, compared to 13 percent of females. Interestingly, among students who reported their mother as the caregiver, 46 percent said their caregiver went to school as a child; 4 Panapress. Liberia enforces free, compulsory primary education. 2004. Accessed July 2018. 5 UNICEF Liberia. Primary school years. Accessed July 2018. 6 This question was asked only to second-grade students as a mechanism to transition their mindset into thinking about their household literacy activities. IMPAQ International, LLC Page 11 LEARN Second Cohort Baseline Report however, this rate rose to 72 percent when students cited their father as the caregiver. Taken together, 53 percent of students said their caregiver had attended school when younger; there were no large differences by grade, gender, or county. To get a sense of household socioeconomic status, we asked students to report on whether certain durable goods were available at home. As shown in Exhibit 9, cell phones were at the top of the list in all counties (71 percent), but the prevalence of some other goods varied by county. In Rivercess and River Gee approximately half of the sample (49 and 53 percent respectively) had a generator while in Grand Bassa only 22 percent had one. The proportion of students who had televisions at home was lower in Rivercess compared to the other counties, at 8 percent. About one quarter (24 percent) of the students in Grand Bassa had none of the goods on the list. This is 11 and 13 percentage points higher than in Rivercess and River Gee, respectively. The prevalence of other durable goods showed no large differences by county. Exhibit 9. Household Possession of Durable Goods by County Does your home have a …. Grand Bassa Rivercess River Gee Overall Cell phone 69% 77% 71% 71% Current/light/generator 22% 49% 53% 32% Icebox 4% 6% 2% 4% Bicycle 5% 2% 5% 5% TV 14% 8% 18% 13% Motorbike 22% 25% 27% 23% Car 4% 1% 1% 3% Keh keh (a three-wheeled vehicle) 0% 0% 0% 0% None of the above 24% 14% 12% 21% Source: Student survey, IMPAQ calculation. Note: Students were told to select all that apply and therefore the total of the percentages does not add to 100 percent. N = 572 for Grand Bassa, 166 for Rivercess, and 114 for River Gee. The survey asked children about their primary language and any other languages they spoke at home. Among the whole sample, 52 percent of students reported that English was their primary language at home; 100 percent said they spoke another language as well, with Bassa the most prevalent second language at 44 percent. Grand Bassa is the only county in which the majority of respondents did not speak English as their main language. A 50 percent plurality reported speaking Bassa in Grand Bassa, followed by English at 37 percent. There were no significant differences in main language by gender or grade (see Exhibit 30 in Appendix C: Additional Tables and Complementary Outcomes for more detail). 4.2 Student Reading Outcomes To add depth to the analysis of second-grade students’ LBRA literacy outcomes, we examined second￾graders’ responses to survey questions about the literacy environment at school and at home. The questions focused on four key areas: (1) the availability of reading materials in and out of school, (2) students’ home literacy environment, (3) students’ attitudes toward schooling, and (4) the presence of teachers in schools. After reporting on the results of these survey questions, we outline findings from the LBRA. 4.2.1 Availability of Reading Materials The team asked second-grade students about the availability of reading materials at school, home, and outside of these spheres, in their community at large. The survey also asked how often students borrowed books from school, if available. At School As Exhibit 10 shows, despite some regional variations, 37 percent of Grade 2 students reported that their school had books other than textbooks. The proportion of students who reported being able to take these books home to read for free was higher in River Gee (33 percent) and Rivercess (34 percent) compared to Grand Bassa (20 percent). Grand Bassa also had a higher proportion of students who reported that their IMPAQ International, LLC Page 12 LEARN Second Cohort Baseline Report school did not have non-textbook reading materials at all at 70 percent. Just three percent of the overall sample reported having to pay a fee to take non-textbook reading materials home. Exhibit 10. Access to Non-Textbook Reading Materials in School Source: Student survey, IMPAQ calculation. N = 419 for Grand Bassa, 121 for Rivercess, and 71 for River Gee. Exhibit 11 below shows the frequency with which students borrowed books from school in each county. The team asked this question of the 161 students who said that their school allowed them to take books home either for free or at a cost. As the exhibit illustrates, a much lower proportion of students in Grand Bassa (17 percent) reported that they never borrowed books from school than in the other counties (63 percent in both Rivercess and River Gee). In other words, Grand Bassa, which makes up 55 percent of the sample size, significantly drives this result with 83 percent of students taking home non-textbooks at least once during the week, compared to Rivercess and River Gee at 37 and 38 percent, respectively. Exhibit 11. Frequency with Which Students Borrowed Non-Textbook Reading Materials to Take Home Response Grand Bassa Rivercess River Gee Overall Every day (5 days) 2% 2% 8% 3% A few times during the week (2–4 days) 33% 6% 17% 22% Once during the week 48% 29% 13% 37% Never 17% 63% 63% 38% Source: Student survey, IMPAQ calculation. N = 88 for Grand Bassa, 49 for Rivercess, and 24 for River Gee. At Home A majority of students (62 percent) said they had a holy book at home (see Exhibit 12). Storybooks and comics represented the next most frequently cited non-textbook reading material at home. Fewer students reported having newspapers (6 percent) or coloring books (14 percent). A larger proportion of students in Grand Bassa said they had no reading materials at home (22 percent) compared to the other counties (13 and 12 percent). Exhibit 12. Availability of Reading Materials in the Home by County Does your home have… Grand Bassa Rivercess River Gee Overall Holy book 62% 64% 57% 62% Textbooks/schoolbooks 40% 60% 65% 48% Storybooks/comics 22% 46% 42% 29% Coloring and drawing books 15% 13% 11% 14% Newspapers 5% 8% 11% 6% None of the above 22% 13% 12% 19% Source: Student survey, IMPAQ calculation. Note: Students were told to select all that apply and therefore the total of the percentages does not add to 100 percent. N = 572 for Grand Bassa, 166 for Rivercess, and 114 for River Gee. Outside of School or Home When we asked second-grade students whether they had read storybooks outside of school or home in the past week, only 21 percent stated they had done so (See Exhibit 31 in Appendix C: Additional Tables and 70% 45% 51% 8% 13% 15% 20% 35% 33% 1% 8% 1% 0% 10% 20% 30% 40% 50% 60% 70% 80% Grand Bassa Rivercess River Gee No Yes, but can't take off campus Yes, and we can take home for free Yes, and we can take home for a cost IMPAQ International, LLC Page 13 LEARN Second Cohort Baseline Report Complementary Outcomes for more details). There were only minute differences when disaggregated by county or gender. Of the students who reported they read storybooks outside of school or home, 59 percent said they went to their friends or relatives for reading materials. One percent or fewer reported using each of the following: reading clubs, religious buildings, or community libraries. Interestingly, 56 percent of respondents said they did not know where to go to read or borrow books, which could possibly explain the low average proportion of students who reported extracurricular reading. Presumably, if students are unaware how or where to obtain reading materials, then they would less frequently engage in this activity. However, we cannot state with confidence that lack of understanding of how to obtain books accounts for the low proportion of Grade 2 students who responded that they read storybooks outside of school or home. 4.2.2 Home Literacy Environment Study of household literacy practices can illuminate the level of children’s exposure to learning outside of school. Students exposed to literacy activities at home have better opportunities to acquire literacy.7 Numerous studies point to the role of the home literacy environment in influencing early reading skills, and in particular, children’s exposure to print materials at home and opportunities to engage in reading with other household members.8 As such, the assessment included questions to measure the home literacy environment, which we conceptualized as the presence of print materials at home and reading habits of family members, as reported by students. Specifically, we asked students whether they saw anyone reading at home and whether anyone in their household had encouraged them to study, read to them, or told them a story. Exhibit 13 shows how students reported their household literacy activities by county. In the overall sample, the majority of students (55 percent) said that someone in their household helped them study; 44 percent stated that someone read to them and 40 percent that they saw someone reading. There were large differences by county. A smaller proportion of students in Grand Bassa reported household literacy activities at home in each category compared to the other counties. However, the level of literacy activities, reported by students, was more or less the same in River Gee and Rivercess counties. Exhibit 13. Household Literacy Activities in the Past Week Source: Student survey, IMPAQ calculation. Students were told to select all that apply and therefore the total of the percentages do not add to 100 percent. N = 421 for Grand Bassa, 121 for Rivercess, and 71 for River Gee. To further understand the household learning environment, the survey asked students to report on the specific family member involved in the four activities listed in Exhibit 13. Students tended to state that it was an older sibling, especially a brother or father (34 and 25 percent on average, respectively), who engaged in 7 Kim, Y. S. (2009). The relationship between home literacy practices and developmental trajectories of emergent literacy and conventional literacy skills for Korean children. Reading and Writing, 22(1), 57-84. 8 Hess, R. D. & Holloway, S. D. (1984). Family and school as educational institutions. Review of Child Development Research, 7, 179-222. Dowd, A.J., Pisani, L. & Borisava, I. (2016). “Evaluating Early Learning from Age 3 to Grade 3” in Understanding What Works in Oral Reading Assessments. Montreal: UNESCO Institute for Statistics (UIS). 32% 61% 55% 47% 70% 72% 37% 60% 61% 23% 31% 31% 0% 20% 40% 60% 80% Grand Bassa Rivercess River Gee Saw someone reading Helped with studies Read to student Told student a story IMPAQ International, LLC Page 14 LEARN Second Cohort Baseline Report these activities. (See Exhibit 32 in Appendix C: Additional Tables and Complementary Outcomes.) No major differences were found by gender or county. 4.2.3 Student Attitudes Toward Schooling To assess students’ perceptions of their education, we asked Grade 2 students the reasons they liked and disliked school. (The details are shown in Exhibit 33 and Exhibit 34 in Appendix C: Additional Tables and Complementary Outcomes.) A large majority of second-grade students (70 percent) said they liked schools because they found their lessons and learning enjoyable. Students in Rivercess were particularly enthusiastic about lessons and learning: 82 percent reported them as a reason they like school. Additionally, students in Grand Bassa were less enthusiastic about their teachers: 26 percent reported teachers as a reason they like school compared to 46 percent and 44 percent respectively for Rivercess and River Gee. There were no major differences in students’ responses between boys and girls. No clear patterns emerged in the reasons students disliked school. A fairly large proportion of students (53 percent) did not respond or know the answer; 17 percent of students reported that they disliked their school because of their teacher’s punishments or physical violence (from teachers or other students). Moreover, 10 percent mentioned “other” options for each question, but the survey did not ask the students to specify. 4.2.4 Presence of Teachers in School A large body of literature suggests a positive association between teachers’ attendance and students’ achievement.9 Thus, we asked second-graders about the consistent presence of teachers in school, as teacher attendance is important when considering students’ reading outcomes. Overall, 89 percent of students stated that teachers came to class every day. The frequency of student￾reported daily teacher attendance by county is shown in Exhibit 14. The proportion is lowest in River Gee at 83 percent, followed by 87 percent in Grand Bassa, and 97 percent in Rivercess. There were no noticeable differences in teacher attendance by grade, reported by students. Exhibit 14. Teacher Attendance by County Source: Student survey, IMPAQ calculation. N = 362 for Grand Bassa, 116 for Rivercess, and 66 for River Gee. Even if teachers come to class every day, they may arrive late or leave earlier than they should, and this behavior could affect student education outcomes. For that reason, we asked students who said their teacher did attend class at least once a week how often their teacher was tardy. Exhibit 15 shows the results by county. Across all three counties, 80 percent of students said their teacher never arrived late to class. Teacher 9 Ahn, T., & Vigdor, J. (2010). The impact of incentives on effort: Teacher bonuses in North Carolina. PEPG 10-06. Miller, Raegen. (2012). Teacher Absence as a Leading Indicator of Student Achievement: New National Data Offer Opportunity to Examine Cost of Teacher Absence Relative to Learning Loss. Center for American Progress. Woods, Robert. (1990). The effect of teacher attendance on student achievement in two selected school districts. 87% 97% 83% 11% 0% 8% 0% 2% 0% 3% 2% 8% 0% 20% 40% 60% 80% 100% 120% Grand Bassa Rivercess River Gee Every day (5 days) A few times during the week (2-4 days) Once during the week Never IMPAQ International, LLC Page 15 LEARN Second Cohort Baseline Report tardiness was reported slightly more frequently in River Gee than in the other two counties: only 7 percent of students reported that teachers arrived late at least once per week in Rivercess, compared to 23 and 29 percent of students in Grand Bassa and River Gee, respectively. However, the results for both attendance and tardiness should be interpreted with caution for three main reasons. First, the team measured these outcomes based on self-reported responses of young children in Grade 2. Second, the team collected data at the beginning of the school academic year when some schools had not completed enrollment. Finally, the team collected data during the rainy season when road conditions could have influenced teachers’ attendance and tardiness. Exhibit 15. Teacher Tardiness by County Source: Student survey, IMPAQ calculation. Note: The percentages do not add up to 100 percent because students who reported that their teacher never came to class did not answer this question. N = 336 for Grand Bassa, 113 for Rivercess, and 61 for River Gee. Percentage are rounded to the nearest whole number. 4.2.5 Reading Outcomes As mentioned in Section 2.4.1 Student Survey and Literacy Boost Reading Assessment, the evaluation team used the same modified version of the LBRA that we developed for data collection in April-May 2018. Similar to the first cohort of baseline evaluation, the evaluation team administered the LBRA in English. The LBRA consists of the following four subtests: 1. Letter knowledge: The number of letter sounds the student could identify out of 26 2. Word recognition: The number of words, out of 20 most-used words from leveled textbooks, that the student could read correctly. Recognition is defined as the student’s ability to read the word. 3. Decoding (Invented word recognition): The number of invented words, out of 20, that students could decode correctly. 4. Reading comprehension • Reading aloud: Using a short story of 155 words, we assessed: o Fluency: The number of words read correctly in a minute o Accuracy: The percentage of words read correctly (untimed) • Comprehension: Ten comprehension questions related to the short story were asked orally in one of three conditions: o Reading comprehension, which applied to children who could read at least 5 words in the story correctly in 30 seconds. These children were identified as “readers.” o Listening comprehension, which applied to children who could not read 5 words in the story correctly in 30 seconds. The enumerator read the story aloud to these children, identified as “non-readers.” o Listening comprehension for “readers,” which applied to students who read at least 5 words correctly but gave up before attempting to read a significant portion of the passage or could not finish the passage on their own. The enumerator read the rest of the story to them. 7% 2% 5% 10% 3% 12% 6% 2% 12% 77% 94% 72% 0% 20% 40% 60% 80% 100% Grand Bassa Rivercess River Gee Every day (5 days) A few times during the week (2-4 days) Once during the week Never IMPAQ International, LLC Page 16 LEARN Second Cohort Baseline Report Before examining each of the four LBRA subtests in depth, Exhibit 16 shows a summary of second-grade students’ literacy skills by county. In the overall sample, the evaluation team identified only 12 percent of the sample as readers. In particular, the key reading outcomes were fairly inconsistent among the counties. Rivercess appears to have a relatively larger group of readers. However, among the readers, students in River Gee outperformed students in other counties in reading comprehension. These few students who could read performed better in reading comprehension and listening comprehension than non-readers did in listening comprehension. In general, children were successful at recognizing the letters of the alphabet but struggled to recognize full words or decode nonsense words. More specifically, on average, the students were able to identify 31 percent of words (out 20); only 1 percent of students were able to decode all 20 nonsense words. Exhibit 16. Second-Grade Students’ Literacy Skills by County Outcomes Grand Bassa Rivercess River Gee Overall Foundational Literacy Skills Letter knowledge (# correct out of 26) 23 23 21 22 Letter knowledge (% correct) 87% 87% 80% 86% Word recognition (# correct out of 20) 7 5 3 6 Word recognition (% correct) 35% 27% 17% 31% Invented word recognition (# out of 20) 0 0 0 0 Invented word recognition (% correct) 2% 1% 1% 2% Reading Skills Students classified as readers (5+ words correct in 30 seconds) 9% 20% 14% 12% Accuracy (% words correct in passage), readers only 8% 8% 5% 7% Fluency (words correct per minute), readers only 9 10 9 9 Comprehension Skills Reading comprehension questions correct (%), readers only 44% 0% 100% 42% Listening comprehension questions correct (%), non-readers only 17% 4% 8% 14% Listening comprehension questions correct (%), readers only 17% 14% 33% 18% Source: Student survey, IMPAQ calculation. Note: Only 12 students were eligible for reading comprehension. Readers who were not eligible for reading comprehension were considered for listening comprehension.. N = 413 for Grand Bassa, 121 for Rivercess, and 71 for River Gee. In the next sections, IMPAQ document the benchmark values of the key literacy outcomes in detail. IMPAQ provides a brief overview of other subtests and their desired outcomes in Appendix D: Other Subtests of Reading Assessment. Letter Knowledge. Letter Knowledge To assess students’ letter knowledge, enumerators showed them a chart of 26 letters in English and asked them to name each letter. Most students (81 percent) could identify 21 to 26 letters, with an average of 22 letters. Only 13 percent were able to identify all 26 letters, but there was no student who could not identify a single letter. The letter “L” was the most challenging for students to identify (40 percent), while “O” was the easiest (98 percent). Exhibit 17 shows the distribution of letter recognition scores for the sample of second-grade students by county. There were no major differences in the outcomes by gender, primary language at home, or county. IMPAQ International, LLC Page 17 LEARN Second Cohort Baseline Report Exhibit 17. Distribution of Letter Recognition Scores Source: Student survey, IMPAQ calculation. N = 422 for Grand Bassa, 121 for Rivercess, 71 for River Gee. Reading Skills and Reading Comprehension Only 12 percent of sampled second-graders were identified as readers; that is, they read at least 5 words correctly in 30 seconds. Rivercess had the highest percentage of readers with 20 percent, followed by River Gee with 14 percent, and Grand Bassa with 9 percent. Only minor inconsistencies emerged when the team disaggregated data by gender and language. Notably, out of the 73 readers, 10 percent did not consider English their first language compared to 48 percent of the overall sample. Among the 73 readers, 84 percent of students either gave up before attempting to read a significant portion of the passage or could not read the rest of the passage. The enumerators read the rest of the passage aloud to these students and assessed them on listening comprehension. Only 16 percent of identified readers qualified for assessment on reading comprehension. We also looked at the students’ reading fluency and accuracy. Overall, students were able to read an average of three words per minute while those determined to be readers averaged nice words per minute. The students’ accuracy was low; on average only 46 percent of the words they attempted were read correctly. That number rose to only 53 percent when only considering the readers. Readers could read only 7 percent of the entire passage accurately; however, this result should be considered with caution because as mentioned above most of the readers stopped part-way through the passage without even attempting to read the entire passage. The accuracy of readers goes up to 52 percent when measured as the percent of the words they attempted to read. After almost all students listened to the whole passage, enumerators asked 10 comprehension questions:  Summary: One question that tests students’ ability to identify the main ideas of a reading passage.  Literal: Five questions in which the answer is clearly and explicitly stated in the passage.  Inferential: Three questions in which the answers are usually implied, rather than clearly stated in the passage.  Evaluative: One question that requires some level of cognitive and/or emotional judgment. To answer such a question, a child needs to use his/her personal opinion. The team defined competency on the comprehension assessment as ability to answer at least 80 percent of the questions correctly. In general, readers were more successful on the comprehension questions than listeners (both non-readers and readers who did not finish the passage). Fully 42 percent of readers (5 students) and just 14 percent of listeners (82 students) answered at least 80 percent of the comprehension questions correctly. Among the listeners, 14 percent of non-readers and 19 percent of readers met the 80 percent competency standard. We also examined the reading outcomes based on students’ primary language at home. Interestingly, those who reported English as their main language spoken at home scored similarly on the comprehension test than non-English speakers, with gaps of just one percent on the reader and listening comprehension 0% 5% 10% 15% 20% 25% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Grand Bassa Rivercess River Gee IMPAQ International, LLC Page 18 LEARN Second Cohort Baseline Report assessments.10 The only space where we see any gaps between English and non-English speakers is in terms of accuracy. English speakers were able to read 56 percent of the words they attempted correctly compared to 48 percent of non-English speakers (see Exhibit 38 in Appendix C: Additional Tables and Complementary Outcomes for more details). There were no major differences in the results by county or gender. We also analyzed the comprehension results by question types. As shown by Exhibit 18, there were substantial differences between counties on different question types. Respondents in River Gee outperformed on the summary question, while respondents in Grand Bassa scored more strongly on the literal and evaluative questions. The disproportionate sample size in Grand Bassa causes these results to weigh heavily on the overall averages. Exhibit 18. Comprehension Subtests by County Question Types Grand Bassa Rivercess River Gee Total Summary 5% 5% 11% 5% Literal 31% 16% 17% 26% Inferential 70% 71% 68% 70% Evaluative 73% 60% 59% 69% Source: Student survey, IMPAQ calculation. N = 422 for Grand Bassa, 121 for Rivercess, and 71 for River Gee. Associations between School Characteristics and Students’ Literacy Skills As the literature suggests that schools’ characteristics may matter in improving children’s learning or their willingness to attend school,11 the evaluation team used multivariate regression analyses to examine the relationship between students’ literacy skills and schools’ characteristics, collected from the school assessment. We looked at the following selected variables: 12  Students’ enrollment (by gender)  Number of latrines available in the school  Presence of handwashing stations  Availability of drinking water resources at the school  Presence of school meals  Availability of libraries and book banks  Structure of the school (temporary vs permanent) Overall, the regression analysis showed a strong relationship between school enrollment and second-grade students’ literacy skills. However, the association was negative for male and positive for female students. Although this information could be helpful for implementation purposes, they should be interpreted with caution as we were only able to collect data from 61 schools where not all of them had enrollment data available. Appendix E: Regression Analyses presents the regression results, taking into consideration performance by county, gender, and grade across the program schools. 4.3 Other Key Student Outcomes This section describes key project evaluation outcome indicators pertaining to hygiene and handwashing, nutrition, SGBV and gender norms, as well as disability. The enumerators asked questions about these topics of both Grade 2 and Grade 6 students, except for questions about gender norms, from which Grade 2 students were excluded because of limited cognitive ability to handle such questions. 10 Reading comprehension numbers for non-English speakers should be analyzed with caution because only five non￾English-speaking students and seven English speaking students qualified for the reading comprehension assessment. The listening comprehension results can be considered more robust. 11 Glewwe, P. (2002). Schools and Skills in Developing Countries: Education Policies and Socioeconomic Outcomes. Journal of Economic Literature, XL, 436-482. 12 Only variables that had enough variations in their responses were selected for the regression analysis. IMPAQ International, LLC Page 19 LEARN Second Cohort Baseline Report 4.3.1 Hygiene and Handwashing Practices To capture information on hygiene practices, enumerators first asked students whether they had washed their hands at all in the day prior to the survey, and with what. Moststudents (88 percent) reported that they had washed their hands, and 87 percent of those studentssaid they washed with water and soap. There were no gender, grade, or county differences in those outcomes. For a deeper understanding ofstudents’ knowledge and handwashing practices, we also developed questions to compare students’ knowledge of appropriate handwashing behavior to their actual conduct. Survey questions focused on handwashing at critical moments, defined as: (1) after using the toilet to defecate, (2) after using the toilet to urinate, and (3) before consuming food. The data show that students’ knowledge of washing hands at critical moments is more or less consistent with their actual practices at nine and six percent, respectively. But both low percentages could be due to lack of proper infrastructure and facilities in the school. Handwashing knowledge and practice varied considerably by county, as shown in Exhibit 19. Interestingly, students in Grand Bassa had the lowest reported handwashing knowledge but the highest reported handwashing behavior, while their knowledge and actual practices were consistent. Students in River Gee and Rivercess reported very low hygiene knowledge and practices with a large gap between them. When looking at each of the three critical moments separately by county, the data show that this large gap in River Gee and Rivercess was mostly driven by having a higher knowledge of washing hands after urinating than students’ actual practices compared to other critical times for handwashing. (See Exhibit 35 in Appendix C: Additional Tables and Complementary Outcomes for details.) Exhibit 19. Student Knowledge vs. Practice of Critical Handwashing Moments by County Indicator Grand Bassa Rivercess River Gee Overall Handwashing knowledge 8% 13% 11% 9% Handwashing self-reported behavior 7% 4% 3% 6% Source: Student survey, IMPAQ calculation. N = 572 for Grand Bassa, 166 for Rivercess, and 114 for River Gee. Additionally, 70 percent of Grade 2 students reported that they washed their hands after defecating while 80 percent of Grade 6 students reported doing so. The gap widens slightly to 12 percentage points when students were asked if they should be washing their hands after defecating. Boys and girls showed similar patterns in their responses to questions about when they should wash their hands. 4.3.2 Nutrition Practices and Knowledge Practices To determine whether schools had canteens and provided meals before LEARN implementation, the survey inquired whether students had eaten a free meal prepared at school the day prior to the survey. We gave the students the option of responding ‘no’ to having eaten a meal prepared at school, ‘yes’ if they had eaten a meal, and ‘no food was prepared’ if they had previously eaten a meal at school but the canteen was currently inactive. Though, overall, eight percent said that no food was prepared in their school a day before the survey, the results varied by county, as Exhibit 20 shows. In Grand Bassa, 91 percent of students reported that they had not eaten a meal at school compared to 83 and 82 percent in Rivercess and River Gee, respectively. A negligible difference (less than five percent) was found between Grade 2 and Grade 6 students and between boys and girls in their responses to whether they consumed a free meal at school. However, the school observations show lower proportions of school had food preparation at school. In Grand Bassa, 51 percent of schools reported that there was meal preparation; while, in both Rivercess and River Gee every school reported that food preparation took place. This contradiction should be interpreted with caution as we collected data during the first month of school and from young children. IMPAQ International, LLC Page 20 LEARN Second Cohort Baseline Report Exhibit 20. Availability of Free School Meals by County Source: Student survey, IMPAQ calculation. N = 564 for Grand Bassa, 166 for Rivercess, and 114 for River Gee. To obtain a better understanding of students’ eating habits, we asked Grade 2 and Grade 6 students how frequently they ate each day. As shown in Exhibit 21, 59 percent said they ate twice per day and 20 percent reported eating 3 times daily. A higher percentage of Grade 6 students than Grade 2 reported eating 3 times per day (29 percent and 17 percent respectively). Negligible differences emerge when the data are disaggregated by gender or county. Exhibit 21. Students’ Eating Frequency Source: Student survey, IMPAQ calculation. 6 students responded that they did not know. N = 846. When asked how food should be divided by gender in the household, over half of the sample (60 percent of girls and 54 percent of boys) believed boys and girls should receive equal amounts. However, as shown in Exhibit 22 male students tended to believe boys should be given more food than girls at a higher rate than females (36 percent compared to 23 percent). This pattern was generally repeated across counties. No notable difference was found when comparing the responses between grades. 2% 20% 59% 2% 16% 0% More than three times per day Three times per day Twice per day Sometimes two times, sometimes one time Once per day I eat once a day and sometimes not eat at all 91% 83% 82% 5% 1% 2% 4% 17% 16% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Grand Bassa Rivercess River Gee No Yes No food was prepared IMPAQ International, LLC Page 21 LEARN Second Cohort Baseline Report Exhibit 22. Students’ Perceptions of How Food Should Be Allocated to Boys and Girls Source: Student survey, IMPAQ calculation. N = 446 for boys and 339 for girls. Knowledge To determine whether students could identify the components of a healthy diet, the survey asked students to identify the three components of a balanced diet, defined as go, glow, and grow foods.13 Only 2 percent of students (20 students out of 816) stated that they knew the definition of a balanced diet, and of those 20 none could successfully identify all three components of a healthy diet. 4.3.3 Sexual and Gender-Based Violence and Gender Norms As noted at the beginning of this section, we surveyed both grades regarding SGBV, but directed gender norm questions only to Grade 6 students. To assess students’ willingness and ability to report incidents of violence, the survey inquired whether rules existed to guide teachers’ behavior in school and, if so, to specify those rules. Specifically, it asked how teachers disciplined students and whether students knew to whom they could go if they were being harassed. Regarding gender norms, we asked Grade 6 students only whether they agree or disagree with a series of statements about relationships between males and females. Knowledge of Rules for Teachers A large majority, of students (78 percent) said that rules existed for how teachers should treat students at school. An examination of the data by county and gender shows no major differences. But it seemed a higher proportion of Grade 6 students (85 percent) knew about the rules, compared to second-grade students (75 percent). A greater level of knowledge among Grade 6 students shows they have a more complete understanding of school operations and the rules in place that teachers ought to follow compared to second￾grade students, which could be due to the fact that they had attended school for a longer period of time. Students tended to know that rules prevented teachers from physically harming students. Differences by gender and region are negligible. As Exhibit 23 shows, 56 percent of students stated that teachers were not allowed to beat students, while 17 percent said that teachers were not allowed to beat students “too much.” As explained in the first cohort baseline evaluation, we made the distinction between beating students and doing so “too much” because the cognitive testing showed that in the Liberian context beating students remains a common disciplinary strategy in primary schools. Therefore, we distinguish between the severity of beating so as to better capture information on this front. 13 “Go” foods are defined as foods that give you energy to play and learn; “Glow” foods are defined as foods that protect your body from disease; and “Grow” food are defined as foods that help your body grow. 36% 9% 54% 23% 17% 60% 0% 10% 20% 30% 40% 50% 60% 70% Boys should get more Girls should get more Boys and girls should get equal amounts Girls Boys IMPAQ International, LLC Page 22 LEARN Second Cohort Baseline Report Exhibit 23. Student Identification of Rules to Guide Teacher Behavior Source: Student survey, IMPAQ calculation. Students were told to select all that apply and therefore the total of the percentages do not add to 100 percent. For example, 31 students responded that they did not know the answer even though they said rules exist for how teachers should treat students in school; 36 students responded “Other”. N = 568. Reports of Disciplinary Practices When asked about discipline at school, 47 percent of students said teachers forced them to clean or work at school if they behaved poorly, and 43 percent reported physical violence. Exhibit 24 shows high variability among counties in reports of teachers’ disciplinary practices, particularly physical violence, work at school, and extra schoolwork. The team’s analysis of the data by grade also revealed some differences in students’ experience of discipline. Grade 2 students were more likely to mention physical violence (47 percent) as a school disciplinary practice than were Grade 6 students (32 percent). Meanwhile, 56 percent of Grade 6 students identified cleaning or working at school as a type of punishment, while only 44 percent of Grade 2 students mentioned this tactic as a disciplinary strategy. In addition, Grade 6 students were more likely to be given extra schoolwork (51 percent) compared to Grade 2 students (39 percent). (See Exhibit 36 in Appendix C: Additional Tables and Complementary Outcomes.) These results suggest that teachers’ disciplinary strategies varied according to students’ grade. Exhibit 24. Types of School Discipline by County Source: Student survey, IMPAQ calculation. Students were told to select all that apply and therefore the total of the percentages do not add to 100 percent. N = 572 for Grand Bassa, 166 for Rivercess, and 114 for River Gee. The incidence of physical violence varied by grade across counties. For instance, the disparity between Grade 2 and Grade 6 students was far greater in River Gee than in other counties: 31 percent of Grade 2 students cited physical violence compared to 12 percent of Grade 6 students. No large differences emerged in an examination of the data by gender. 15% 17% 20% 21% 56% 0% 10% 20% 30% 40% 50% 60% Teachers are not allowed to ask students for money Teachers are not allowed to beat students too much Teachers are not allowed to be in a relationship with students Teachers are not allowed to use humiliating language on students Teachers are not allowed to beat students 44% 38% 52% 15% 1% 5% 63% 46% 25% 11% 0% 4% 42% 55% 24% 10% 1% 8% 0% 20% 40% 60% 80% 100% Made to clean or work at school Give Extra Work Physical violence Dismiss students Humiliating language Other Grand Bassa Rivercess River Gee IMPAQ International, LLC Page 23 LEARN Second Cohort Baseline Report Willingness to Report Finally, the enumerators asked students about their knowledge of actions to take if they are teased or touched at school in a way they do not like, which left open ended the actor involved in such action (a teacher, another child, an administrator, etc.). A large majority of students (91 percent) reported that they would speak to their teacher, followed by 24 percent who said they would go to the principal or registrar. Synthesizing all this information to create an index for SGBV knowledge and practice, we developed three measurements to gauge students’ willingness and ability to report SGBV incidents: (1) proportion of students who understand school rules and codes of conduct; (2) proportion of students who indicated they would report cases of bad behavior; and (3) proportion of students who reported any type of corporal or psychological teacher discipline. We considered students to be knowledgeable about codes of conduct if they reported that rules exist to guide teachers’ behavior and could describe at least one of these rules to the enumerator. Across the sample, 63 percent of students stated that their schools had a code of conduct regulating teachers’ behavior. This proportion generally remained above 50 percent when data were disaggregated by county, grade, and gender. Further, 75 percent of Grade 6 students understood the code of conduct compared to 58 percent of Grade 2 students. There was little to no difference between male and female respondents. We considered students to be willing to report SGBV incidents, defined in the survey as being teased or touched in an uncomfortable way, to see if they could identify the person they would speak to in such cases. Though simply knowing whom to contact does not guarantee that the student would actually contact the individual, the survey could not ask students directly if they would report an incident because of the sensitivity of the topic. We, therefore, assume that students who could readily name a contact person may be inclined to report SGBV incidents. In this context, a vast majority of students (98 percent) in our sample were willing to report inappropriate situations at school that they witnessed or in which they were personally involved. No differences were found by county, grade, or gender. For the third measurement in our index of willingness to report, we analyzed students’ responses to questions regarding teachers’ disciplinary practices. We considered teachers as having engaged in corporal or psychological punishment if students reported them as having taken part in any of the disciplinary tactics shown in Exhibit 24. By this definition, across the board, regardless of gender, grade, or county, more than 98 percent of students stated that their teachers used some form of corporal or psychological punishment. In summary, our index suggests that students in all grades would willingly report inappropriate teasing or touching in school. Although teachers regularly disciplined students using corporal or psychological disciplinary strategies, students largely understand that their teachers are subject to a code of conduct. However, such knowledge of a code of conduct does not guarantee that students would report their teachers should they violate the code. But this level of knowledge regarding guidelines could influence students’ perceptions of the school climate and permissible behavior. Studies have shown that students feel more encouraged to share their thoughts if they hold positive perceptions of relationships with their teachers in the classroom, and such sentiments may depend on whether their teachers abide by a code of conduct.14 Gender Norms To obtain information on students’ perceptions of gender norms, we asked Grade 6 students only whether they agreed or disagreed with a series of five statements. Exhibit 25 presents the results. 14 Wentzel KR. (1997). Student motivation in middle school: The role of perceived pedagogical caring. Journal of Educational Psychology, 411–419. IMPAQ International, LLC Page 24 LEARN Second Cohort Baseline Report Exhibit 25. Student Perceptions of Gender Norms Statement Disagreed Male Female Total If a boy touches a girl at school, it’s because the girl did something to attract him. 63% 69% 66% There are times when a boy needs to beat his girlfriend. 58% 72% 64% Girls like to be teased by boys. 45% 62% 52% When girls wear short skirts, they are telling boys or men to touch them. 45% 49% 47% For girls to get good grades, they sometimes have to let their teachers touch them or love them. 66% 77% 71% Source: Student survey, IMPAQ calculation. N = 137 for males and 100 for females. A high percentage of Grade 6 students (71 percent) disagreed with the statement that girls must allow teachers to touch them or love them to earn good grades. Similarly, a large proportion of students (66 percent) disagreed with the statement that if a boy touches a girl, he has done so because the girl did something to attract him. As for physical abuse, 64 percent of students disagreed with the statement that sometimes a boy needs to beat his girlfriend. However, respondents were divided regarding whether girls like to be teased by boys and whether wearing short skirts invites boys to touch girls. When examining the answers to these statements by gender, we found some large differences in male and female perceptions of these norms. In general, females were more likely to disagree with the statements than males. For example, 72 percent of females believe that girls do not like to be teased while only 58 percent of males agreed. Disaggregating the data by county also found some large differences. In particular, counties had a wide variation when asked whether if a boy touches a girl at school, it is because the girl did something to attract him. Among respondents, 74 percent of students in Grand Bassa disagreed with that statement, followed by 57 and 47 percent of students in Rivercess and River Gee who disagreed, respectively. See Exhibit 37 in Appendix C: Additional Tables and Complementary Outcomes for a full breakdown by county. Next, we defined a threshold for students holding less biased gender norms as disagreement with at least four of the five statements on gender stereotypes. Forty-two (42) percent of studentsreached the threshold, composed of a slightly higher percentage of female (50 percent) than male (37 percent). As Exhibit 26 illustrates, there were significant variations by county: a much lower percentage of students in River Gee disagreed with at least four out of five gender norms statements. However, these results should be interpreted with caution due to social desirability of self-reported responses to such questions (especially in Liberia where this subject is one of the known challenges in the country). 15 Exhibit 26. Students who Disagreed with 4 of 5 Gender Norms by County Source: Student survey, IMPAQ calculation. N = 150 for Grand Bassa, 45 for Rivercess, and 43 for River Gee. 15 Parkes, J. (2016). The Evolution of Policy Enactment on Gender-based Violence in School. Prospects, 93-107. 47% 40% 26% 0% 10% 20% 30% 40% 50% Grand Bassa Rivercess River Gee Disagreed with 4 of 5 gender norms IMPAQ International, LLC Page 25 LEARN Second Cohort Baseline Report 4.3.4 Disability We also assessed visual, auditory, or physical impairments that may impede students’ ability to learn in the classroom by asking the same set of questions from the Washington Group that we asked in the first cohort baseline data collection. These questions reflect current thinking and measurement of child functioning.16 Although this leading conceptual framework assesses a multitude of areas in which children may experience functional difficulties, in our survey we asked students only questions related to difficulties seeing, hearing, or walking. We directed these questions to students in both Grade 2 and Grade 6. Our results showed that across grades, gender, and counties, the vast majority of children (more than 90 percent) reported that they did not have any kinds of disability in these domains. We did not find any notable differences by gender, grade, or county. Exhibit 27. Proportion of Students Reporting No Visual, Oratory, or Physical Difficulties Source: Student survey; IMPAQ calculation. N = 852. 16 The Washington Group/UNICEF Module on Child Functioning, finalized in 2016, covers children between 2 and 17 years of age and assesses functional difficulties in different domains including hearing, vision, communication/comprehension, learning, mobility and emotions. See: https://data.unicef.org/topic/child￾disability/module-on-child-functioning/ for more information. 97% 97% 96% 80% 82% 84% 86% 88% 90% 92% 94% 96% 98% 100% No difficulty seeing No difficulty hearing No difficulty walking or climbing steps IMPAQ International, LLC Page 26 LEARN Second Cohort Baseline Report SECTION 5. PROPOSED APPROACH TO MIDLINE AND ENDLINE After a careful review of the data from first and second cohorts for the project evaluation, we observed a few differences in the key average performance outcomes. To understand these differences, we implemented tests for comparing the equality of means of indicators across relevant sub-samples, such as testing whether differences in outcomes remain if we compare subsamples of students from each grade across the two cohorts or subsamples of each region across the two cohorts. In this Section, we summarize these differences and their potential explanations. We also discuss IMPAQ’s proposed strategy for midline and endline sampling and evaluation based on these findings. We will provide SC with more details in the evaluation plan at midline and endline, separately. 5.1 Comparing Baselines Our findings confirm that the differences in the outcomes between the two cohorts arise due to variation in sample size across regions (especially in Grand Bassa), timing of data collection, and/or different demographic information of the two samples. The first difference arises from differences in the timing of data collection. To avoid delays in program implementation for the second cohort, we had to collect data at the beginning of the school year. Whereas, data were collected at the end of the school year for the first cohort. This distinction likely contributes to the lower reading skills in the second cohort. Comparison of the results showed that although there were statistically significant differences between the two cohorts in terms of fluency, accuracy, letter knowledge, and word recognition, the difference in magnitude of these outcomes was only between 5-10 percentage points. Moreover, we found no statistically significant difference in reading comprehension or listening comprehension for readers. The second difference in data collection is the proportion of Grand Bassa schools sampled in each cohort. In the list of 85 schools that we received from SC for the second cohort evaluation, the majority of beneficiary schools were in Grand Bassa, thereby contributing to a higher proportion of Grand Bassa schools in the second cohort sample – 67 percent of the sample consisted of students from Grand Bassa schools in the second cohort as compared to just 31 percent in the first cohort. When disaggregating the two samples by county, the data shows fairly large and statistically significant differences, especially in the proxies for income. Thus, the second cohort students appears to come from poorer households than the first cohort. Additionally, a much lower proportion of the second cohort speaks English as their primary language (36 percent) compared to the first cohort (81 percent) in Grand Bassa. This finding was closer to IMPAQ’s initial expectation because Bassa is the predominant language spoken in the county. When we limited the data to compare the two cohorts’ responses from Rivercess and River Gee, we find almost no statistically significant differences between the two samples in terms of their socioeconomic status, language, and overall home environment across the two cohorts. This suggests that the systematic differences between first and second cohort backgrounds are most likely driven by the Grand Bassa sample. 5.2 Midline and Endline Sampling Plan The midline and endline sample of schools will be constructed using a random representative sample of the 130 schools visited for project evaluation during both rounds of data collection, which will be stratified by region (69 schools drawn from Grand Bassa, Rivercess, and River Gee, in addition to the 16 schools from Grand Gedeh which are included in both the impact17 and performance evaluation sample). We will use a probability proportional to size sampling method to account for Grand Bassa systematic differences between the first and second cohort. We will implement robustness checks to ensure the midline schools drawn from 17 This school selection would not affect the impact evaluation sample and design. We will visit all 55 baseline schools that are part of the impact evaluation. The addition of second cohort of schools may only change the number of schools that overlap between the impact and project evaluation samples. The details of the impact evaluation design are provided in the first baseline evaluation report. IMPAQ International, LLC Page 27 LEARN Second Cohort Baseline Report each cohort are representative of the baseline schools from the respective cohort. Then, at endline, IMPAQ and CART will resurvey these selected 85 schools from midline. 5.3 Midline and Endline Analysis Plan The additional sample from the second cohort evaluation provides us greater variation in the data and allows the evaluation team to select a more robust and representative sample to compare baseline values with those at midline and endline. At midline (2020) and endline (2022), we will collect the data at the end of the school year during dry season. Assuming that the program rollout for both cohorts is approximately the same and both cohorts receive the same level of exposure to the program interventions, we will compare the literacy scores of midline and endline samples with the cohort that best matches the timing of data collection to maximize comparability of outcomes. This means only the first cohort reading outcomes will be used for assessing the progress of the program at midline/endline. We will employ this strategy because the reading outcomes of the first and second cohort samples were statistically significantly different and this variation likely arises from differences in the timing of data collection. Therefore, for fully capturing changes in reading outcomes between baseline and midline, we want to hold all else constant by comparing reading outcomes measured at the same time of the year. On the other hand, to analyze the progress on other outcomes, we will use the aggregated values from the representative random sample of 85 schools as described above. The midline and endline analysis plan is shown in detail below. Exhibit 28. Midline and Endline Analysis At midline, we will work closely with SC to finalize this plan based on the timing of each activity’s implementation. If the exposure to different cohort of students to the intervention activities varies significantly, we will separate their analysis to report on the progress of each cohort, separately. As described in the first cohort evaluation report, the desired sample size based on the power calculations was determined to be 820 for capturing changes in reading outcomes and 384 for measuring changes in health knowledge and practices. Implementing separate analysis for the two cohorts may then influence the statistical power for capturing progress in outcomes, especially outcomes related to reading and literacy. However, we should be able to capture changes in outcomes related to health knowledge and practices in the separate cohort analysis scenario. Our recommendation to SC would be to ensure that both cohorts receive the same duration of exposure to program activities. At midline, our team will work closely with the SC team to ensure that our evaluation design mitigates challenges imposed by potential variation in exposure to program activities and loss in statistical power. IMPAQ International, LLC Page 28 LEARN Second Cohort Baseline Report SECTION 6. CONCLUSION In this report, IMPAQ presents the assessment of baseline levels of an additional 61 schools added to the LEARN project in Grand Bassa, Rivercess, and River Gee. Although the IMPAQ team attempted to visit all 85 of the project’s new schools, 16 were inaccessible because of heavy rains. Additionally, five schools were reported as closed, three were kindergartens, and one was not a primary school. One replacement school was accessible and is included in the data. The team collected data to benchmark baseline values, confirm project design assumptions, and identify potential threats to the project’simplementation in these additional schools. At midline and endline, for the project evaluation, IMPAQ will select a representative sample of 85 schools from among 130 schools visited in both rounds of the baseline evaluation (69 from the first round and 61 from the second round) in Grand Bassa, River Gee, and Rivercess. Since we have set benchmark values in a similar manner for both sets of schools at baseline, we will be able to measure changes in the performance indicators at midline and endline using the combined set of 85 schools. As with the first baseline assessment, here, our findings are based on data from surveys of students, which included reading assessments and questions to probe respondents’ understanding of health, hygiene, nutrition, and SGBV knowledge and practices. We collected data from 852 primary school students in Grades 2 and 6 and administered the reading assessment to 614 second-grade students. In this section, we summarize key findings in response to the main research questions, highlight the study’s limitations and potential challenges, and provide recommendations for the project as a whole and for the evaluation. 6.1 Summary of Key Outcomes Below are key findings from the project evaluation related to students’ literacy outcomes, nutrition knowledge, hygiene practices, and knowledge and practices regarding SGBV and gender norms. Project Evaluation Key Outcomes  Literacy. The evaluation team found that 81 percent of Grade 2 students could identify 21 to 26 letters, with an average of 22. However, students struggled with reading proficiency and comprehension, regardless of county or gender. Only 12 percent of surveyed second-graders were identified as readers, i.e., could read at least five words correctly in 30 seconds; 42 percent of the second-grade students were able to answer at least 80 percent of the comprehension questions correctly (reading with comprehension). These results confirmed the low proficiency of Grade 2 students at grade level at the end of the school year, before the intervention was implemented.  Home environment. The majority of students (55 percent) said that someone in their household helped them study; 44 percent stated that someone read to them; and 40 percent responded that they saw someone reading.  Nutrition. Students lacked sufficient knowledge of a healthy diet; none of them could correctly identify the three types of foods defined as constituting a healthy diet, defined by the project as go, glow, and grow foods.  Handwashing. A high proportion of sampled students (88 percent) said that they had washed their hands during the day prior to the survey. The survey also gathered information on student knowledge and practice of handwashing at three critical moments: after using the toilet to defecate, after using the toilet to urinate, and before consuming food. Although nine percent of students said they should wash their hands at these moments, only six percent responded that they actually did.  Sexual and gender-based violence. To gauge students’ understanding of SGBV, as well as their willingness and ability to report such incidents, we examined the proportion of students who reported that they understood school rules and codes of appropriate conduct; said that they would report any cases of inappropriate teasing or touching; and listed any type of corporal or psychological teacher discipline. Our data showed that 78 percent of students responded that rules exist for how teachers should treat students at school. Furthermore, students in all grades stated that they would willingly report inappropriate teasing or touching. IMPAQ International, LLC Page 29 LEARN Second Cohort Baseline Report  Gender norms. We established a benchmark that indicated that students were aware of gender norms if they disagreed with at least four of five statements related to stereotypical gender perceptions. Less than half of students (42 percent) disagreed with at least four statements, but this percentage was different for girls (50 percent) and boys (37 percent). A regional analysis of the data revealed county variations. A smaller proportion of students in River Gee disagreed with the gender statements regarding stereotypes that were read to them, compared to Grand Bassa and River Gee. 6.2 Limitations The second cohort baseline evaluation has several limitations, some of which are in common with the first cohort baseline that IMPAQ implemented. The main limitation is that the team used a survey instruments designed for both second- and sixth-graders, who have different developmental levels. The IMPAQ team recognized this limitation and, after having conducted cognitive interviews before collecting the data in the first round, made corrections to the survey instrumentsto adapt them to the Liberian context and to mitigate any issues that were likely to result in unreliable data. Another limitation is the inability to triangulate student reports on such factors as parents’ education or the availability of reading materials in the home with those of an informed adult such as a parent or a teacher. Working with our partner CART, we phrased and updated the questions (in the first round of baseline data collection) to the greatest extent possible to ensure that children regardless of their grade could understand the questions. Another limitation arose from sampling students who were present at school rather than drawing a sample from full classroom lists. The possibility of systematic student absences, especially during the rainy season, could have resulted in sampling bias. For example, students from vulnerable socioeconomic backgrounds might have had more health-related absences and might have been more likely than more affluent students to have been excluded from the study because they were absent on the day of data collection. For the project evaluation, there was no reduction in sample size (and thus no reduction in power) as the schools we could not reach were replaced with alternative schools that were more accessible. However, the resulting sample may be less representative than originally designed; for example, the inaccessible schools were more remote, and the schools in the final sample may be closer to major roads. Similarly, far fewer students than expected were found attending sampled schools given the inflated EMIS enrollment numbers. The final sample of students for the project evaluation may also be a more motivated subset of students, living in close proximity to their school. While these limitations are important to mention and keep in mind when interpreting the results, they do not undermine the validity and rigor of the study. Another limitation is getting participants to be fully honest when answering sensitive questions (for example, asking if teachers coerce students to have sex for grades). SGBV is a “known-secret” in Liberia, yet no teachers, principals, or parents would admit any history of sexual abuse of students at their schools. 6.3 Best Practice Recommendations Overall, except for the reading outcomes, almost all of the quantitative results in the second baseline evaluation were similar to those in the first round. This suggests that the team’s recommendations from the first cohort are applicable to the second cohort. However, based on our experience in the field, and our analysis of the second cohort’s baseline data with respect to the first cohort’s data, we recommend the following to SC to enhance future evaluations, data collections, and program implementation.  Ensure enough heterogeneity across reading outcomes for midline and endline sample. Most of the indicators that IMPAQ measured in the second round of data collection showed similar results to the first round, except for the reading outcomes. IMPAQ will take into account variation in the reading outcomes in selecting the 85 schools (among the 130 visited in first and second rounds) for the project evaluation at midline and endline to ensure a more representative sample in measuring the reading outcomes over time. IMPAQ International, LLC Page 30 LEARN Second Cohort Baseline Report  Ensure the midline and endline data collection will be conducted during the dry season. High water on rivers and impassible roads caused by heavy rains in both rounds of baseline data collection severely limited the team’s ability to collect data efficiently and effectively. Even in the accessible schools, the rain limited students’ attendance, which impeded the team’s ability to meet the evaluation target sample. IMPAQ with SC should collect data for both midline and endline during the dry season.  Consider including strategies for teaching decoding skills into the BLA training. Almost none of the second-grade students in both rounds of baseline evaluations were able to decode the made-up words, which could explain the low results on word recognitions. Decoding and encoding of words are the basic skills for reading with comprehension. To improve children’s reading outcomes at earlier grades, teachers should be trained on these techniques.  Educate parents as well as teachers on positive discipline. Although, no qualitative assessment was performed for this report as agreed with SC, the data from the second cohort reaffirms our recommendation in the first report for a focus on educating teachers and parents on the benefits of positive discipline. In both baseline evaluation cohorts, a large majority of students, especially in second grade, reported teachers’ use of physical violence as a school discipline. We recommend the program to focus on improving teachers’ attitudes to create an atmosphere conducive to learning for students, in addition to strengthening teachers’ pedagogical practices. IMPAQ International, LLC Page 31 LEARN Second Cohort Baseline Report REFERENCES Ahn, T., & Vigdor, J. (2010). The impact of incentives on effort: Teacher bonuses in North Carolina. Program on education policy and governance working papers series (PEPG 10-06). Cambridge, MA: Program on Education Policy and Governance. Harvard University, Kennedy School of Government. Dowd, A.J., Pisani, L. & Borisava, I. (2016). “Evaluating Early Learning from Age 3 to Grade 3” in Understanding What Works in Oral Reading Assessments. Montreal: UNESCO Institute for Statistics (UIS). Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement, 33(3), 613-619. Glewwe, P. (2002). Schools and Skills in Developing Countries: Education Policies and Socioeconomic Outcomes. Journal of Economic Literature, XL, 436-482. Hess, R. D., & Holloway, S. D. (1984). Family and school as educational-institutions. Review of child development research, 7, 179-222. Kim, Y. S. (2009). The relationship between home literacy practices and developmental trajectories of emergent literacy and conventional literacy skills for Korean children. Reading and Writing, 22(1), 57-84. Miller, Raegen. (2012). Teacher Absence as a Leading Indicator of Student Achievement: New National Data Offer Opportunity to Examine Cost of Teacher Absence Relative to Learning Loss. Center for American Progress. Panapress. (2004). Liberia enforces free, compulsory primary education. http://www.panapress.com/Liberia-enforces-free,-compulsory-primary-education--13- 549649-17-lang2-index.html Parkes, J. (2016). The Evolution of Policy Enactment on Gender-based Violence in School. Prospects, 93-107. UNICEF Liberia. Primary school years. https://www.unicef.org/liberia/children_7916.html Wentzel KR. (1997). Student motivation in middle school: The role of perceived pedagogical caring. Journal of Educational Psychology, 411–419. Woods, Robert. (1990). The effect of teacher attendance on student achievement in two selected school districts. IMPAQ International, LLC Page 32 LEARN Second Cohort Baseline Report APPENDICES A. LEARN Results Framework B. McGovern-Dole Performance Indicators C. Additional Tables and Complementary Outcomes D. Other Subtests of Reading Assessment E. Regression Analysis F. Inter-Rater Reliability G. Evaluation Instruments IMPAQ International, LLC Page A1 LEARN Second Cohort Baseline Report APPENDIX A: LEARN RESULTS FRAMEWORK IMPAQ International, LLC Page A2 LEARN Second Cohort Baseline Report IMPAQ International, LLC Page A3 LEARN Second Cohort Baseline Report Exhibit 29. Program Activity Packages School Feeding Base Package (SF) Literacy Boost (LB) School Health & Nutrition (SHN)  Provide school meals  Provide take-home rations  Distribute deworming medications, vitamins, and minerals  Institute teacher recognition  Build/rehabilitate storerooms, kitchens, stoves, latrines  Establish PTAs  Provide training on PTAs, food preparation & storage, good health & nutrition, commodity management  Establish activities to promote literacy  Train teachers  Establish libraries  Produce books & reading materials  Promote increase community awareness on SGBV  Establish school gardens  Improve health and nutrition practices Source: SC Terms of Reference (TOR). IMPAQ International, LLC Page B1 LEARN Second Cohort Baseline Report APPENDIX B: McGovern-Dole Performance Indicators McGovern-Dole Indicators Data Collection methods Data Source Observations Baseline (Percentage or Number) MGD 26: Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Evaluation LBRA 352 Boys: 1% 262 Girls: 1% 614 Overall: 1% Custom: Percent of students who, by the end of two grades of primary schooling, demonstrate proficiency in identifying letters Evaluation LBRA 352 Boys: 53% 262 Girls: 51% 614 Overall: 52% MGD 27: Number of individuals benefiting directly from USDA-funded interventions SC/Monitoring SC 0 MGD 28: Number of individuals benefiting indirectly from USDA-funded interventions SC/Monitoring SC 0 MGD 1: Number of students regularly (80%) attending USDA supported classrooms/schools SC/Monitoring SC 22,766 MGD 19: Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance SC/Monitoring SC 0 MGD 21: Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance SC/Monitoring SC 0 Custom: Percentage of teachers in target schools who attend and teach at least 90% of the scheduled school days SC/Monitoring SC 51.12% IMPAQ International, LLC Page B2 LEARN Second Cohort Baseline Report McGovern-Dole Indicators Data Collection methods Data Source Observations Baseline (Percentage or Number) MGD 2: Number of textbooks and other teaching and learning materials provided as a result of USDA assistance SC/Monitoring SC 0 MGD 5: Number of teachers/educators in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance SC/Monitoring SC 0 MGD 6: Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance SC/Monitoring SC 0 MGD 15: Number of daily school meals (breakfast, snack, lunch) provided to school-age children as a result of USDA assistance SC/Monitoring SC 0 MGD 16: Number of school￾age children receiving daily school meals (breakfast, snack, lunch) as a result of USDA assistance SC/Monitoring SC 0 MGD 13: Number of take￾home rations provided as a result of USDA assistance SC/Monitoring SC 0 MGD 14: Number of individuals receiving take￾home rations as a result of USDA assistance SC/Monitoring SC 0 MGD 17: Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance SC/Monitoring SC 0 Custom: Number of daily school meals provided that include fruits, vegetables and/or animal-sourced proteins in addition to USDA commodities SC/Monitoring SC 0 Custom: Number of schools with a strengthened support structure for a code of conduct policy SC/Monitoring SC TBD IMPAQ International, LLC Page B3 LEARN Second Cohort Baseline Report McGovern-Dole Indicators Data Collection methods Data Source Observations Baseline (Percentage or Number) Custom: Percentage of children in target schools who demonstrate improved knowledge and practices towards SGBV prevention and response Evaluation Student survey 322 Boys: 64% 530 Girls: 60% 852 Overall: 62% MGD 12: Number of educational policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance. SC/Monitoring SC 0 MGD 24: Number of students receiving deworming medication(s) SC/Monitoring SC 29,026 Custom: Number of energy￾saving stoves provided as a result of USDA assistance SC/Monitoring SC 0 MGD 7: Number of educational facilities (i.e., school buildings, classrooms, and latrines) rehabilitated/ constructed as a result of USDA assistance SC/Monitoring SC 0 Custom: Number of primary school-age children in targeted communities who participated in a reading camp in the past year SC/Monitoring SC 0 Custom: Number of government officials trained in commodity management practices SC/Monitoring SC 0 MGD 10: Number of public￾private partnerships formed as a result of USDA assistance SC/Monitoring SC 0 MGD 9: Number of Parent￾Teacher Associations (PTAs) or similar school governance structures supported as a result of USDA assistance SC/Monitoring SC 0 Custom: Percentage of Grades 2 and 6 students in target schools who can identify the components of a healthy diet Evaluation Student survey 322 Boys: 0% 530 Girls: 0% 852 Overall: 0% IMPAQ International, LLC Page B4 LEARN Second Cohort Baseline Report McGovern-Dole Indicators Data Collection methods Data Source Observations Baseline (Percentage or Number) MGD 20: Number of individuals trained in safe food preparation and storage as a result of USDA assistance SC/Monitoring SC 0 Custom: Number of schools equipped with food preparation and storage materials SC/Monitoring SC 139 MGD 11: Value of new public and private sector investments leveraged as a result of USDA assistance SC/Monitoring SC 0 MGD 18: Number of individuals trained in child health and nutrition as a result of USDA assistance SC/Monitoring SC 0 MGD 23: Number of schools with improved sanitation facilities SC/Monitoring SC 132 IMPAQ International, LLC Page C1 LEARN Second Cohort Baseline Report APPENDIX C: ADDITIONAL TABLES AND COMPLEMENTARY OUTCOMES Exhibit 30. Main Language by County Source: Student survey; IMPAQ calculation. Note: N=571 for Grand Bassa, 166 for Rivercess, and 114 for River Gee. Exhibit 31. Proportion of Students who Read Non-Textbooks in the Last Week Outside School by County Source: Student survey; IMPAQ calculation. Note: N=413 for Grand Bassa, 121 for Rivercess, and 71 for River Gee. Exhibit 32. Type of Household Members Providing Literacy and Learning Support to Students Family Member Who did you see reading last week? Who helped you study? Who read to you? Who told you a story? Mother 4% 6% 6% 16% Father 25% 27% 25% 22% Older sister 27% 14% 16% 19% Younger sister 2% 1% 1% 3% Older brother 36% 39% 38% 21% Younger brother 1% 1% 1% 2% Grandmother 0% 0% 0% 7% Grandfather 2% 1% 1% 3% 36% 12% 1% 0% 50% 0% 1% 78% 1% 0% 0% 21% 0% 1% 90% 0% 10% 1% 0% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% English Kpelle Grebo Krahn Bassa Kru Other Grand Bassa Rivercess River Gee 21% 18% 24% 0% 5% 10% 15% 20% 25% 30% Grand Bassa Rivercess River Gee IMPAQ International, LLC Page C2 LEARN Second Cohort Baseline Report Family Member Who did you see reading last week? Who helped you study? Who read to you? Who told you a story? Other female relative 2% 4% 4% 2% Other male relative 6% 10% 10% 6% Female non-relative 1% 0% 1% 2% Male non-relative 0% 2% 2% 3% Total Responses 247 333 272 154 Source: Student survey; IMPAQ calculation. Note: Proportion of the 852 Grade 2 students only. Grade 6 students did not answer this question. Students were told to select all that apply and therefore the total of the percentages do not add to 100 percent. Exhibit 33. Reasons Students Like Their School and Class Source: Student survey; IMPAQ calculation. Note: Students were told to select all that apply and therefore the total of the percentages do not add to 100 percent. N=614. Exhibit 34. Reasons Students Dislike Their School and Class Source: Student survey; IMPAQ calculation. Note: Negligible difference (<7 percent) among both male and females. Students were told to select all that apply and therefore the total of the percentages do not add to 100 percent. N = 614. 3% 0% 0% 1% 7% 7% 10% 32% 70% 0% 10% 20% 30% 40% 50% 60% 70% 80% Other Food is Provided Access to water Access to clean toilet Participate in classroom games and activities Being with my friends Playing a sport at school Like teacher Learning new things/enjoy lessons 10% 0% 0% 1% 1% 1% 2% 2% 2% 3% 3% 5% 10% 17% 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20% Other Lessons are boring Lessons are boring No access to water Teacher asks for money Not learning much at school Lessons are difficult to understand/learn Teacher is mean to me/other students Poor toilet conditions/lack of toilets Lack of learning materials No food is provided/the food is bad Lack of uniform Other students tease me/fight with me/other students Teacher punishes me/hits me/other students IMPAQ International, LLC Page C3 LEARN Second Cohort Baseline Report Exhibit 35. Students’ Knowledge vs. Practice of Critical Handwashing Practice by County (Breakdown) Indicator Grand Bassa Rivercess River Gee Overall Reported should wash hands after urinating 16% 25% 19% 18% Washed hands after urinating 15% 14% 11% 14% Reported should wash hands after defecating 73% 77% 71% 74% Washed hands after defecating 73% 73% 71% 73% Reported should wash hands before eating 40% 54% 47% 43% Washed hands before eating 40% 46% 47% 42% Reported should do all of the above 8% 13% 11% 9% All of the above 7% 4% 3% 6% Source: Student survey; IMPAQ calculation. Note: Students were told to select all that apply and therefore the total of the percentages do not add to 100 percent. N = 505 for Grand Bassa, 138 for Rivercess, and 100 for River Gee. Exhibit 36. Types of School Discipline by Grade Source: Student survey; IMPAQ calculation. N = 614 for Grade 2 and 238 for Grade 6. Exhibit 37. Perceptions of Gender Norms by County Grand Bassa Rivercess River Gee If a boy touches a girl at school it's because the girl did something to attract him Disagree 74% 57% 47% There are times when a boy needs to beat his girlfriend Disagree 66% 58% 62% Girls like to be teased by boys Disagree 51% 47% 60% When girls wear short skirts they are telling boys or men to touch them Disagree 51% 49% 28% For girls to get good grades, they sometimes have to let their teachers touch them or love them Disagree 75% 78% 49% Source: Student survey; IMPAQ calculation. N = 149 for Grand Bassa, 42 for Rivercess, 43 for River Gee. 47% 44% 39% 12% 1% 6% 32% 56% 51% 18% 0% 4% 0% 10% 20% 30% 40% 50% 60% Physical violence Made to clean or work at the school Give extra work Dismiss students Humiliating language Other Grade 2 Grade 6 IMPAQ International, LLC Page C4 LEARN Second Cohort Baseline Report Exhibit 38. Comprehension and Reading Skills by Language Indicator English Non-English Overall Reading comprehension (overall % of students who passed) 1% 1% 1% Listening comprehension (overall % of students who passed) 14% 13% 14% Accuracy (% words correct in passage out of attempted words), readers only 56% 48% 52% Accuracy (% words correct in passage out of total words), readers only 8% 6% 7% Fluency (words correct per minute), readers only 9 9 9 Source: Student survey, IMPAQ calculation. N = 439 for English and 413 for Non-English, N = 41 for English Readers and 32 for Non-English Readers. IMPAQ International, LLC Page D1 LEARN Second Cohort Baseline Report APPENDIX D: OTHER SUBTESTS OF READING ASSESSMENT As mentioned in Section 4.2.5 Reading Outcomes, IMPAQ also tested students on other literacy skills, including word recognition, and invented word recognition. In this appendix, IMPAQ presents the outcomes of these subtests to illuminate further children’s literacy outcomes. Word Recognition To assess children’s word recognition skills, enumerators gave students a chart of 20 words that we developed based on the most frequently used words from their textbooks. Exhibit 39 shows the ability of second graders to read these words. In comparison to their ability to identify letters, students struggled to read full words. Additionally, there were some large disparities between counties, as more students struggled to read the words in River Gee compared to Grand Bassa and Rivercess. As seen in Exhibit 39, overall, students were able only to identify 31 percent of the 20 words but students from River Gee could only identify 17 percent correctly. Almost a third of River Gee students (31 percent) could not identity a single word correctly. Exhibit 39. Most Recognized Word by County Indicator Grand Bassa Rivercess River Gee Overall Total number of correctly read words 7 5 3 6 % of words read correctly 35% 27% 17% 31% % identified hardest word (uncle) 8% 7% 0% 7% % identified easiest word (we) 73% 74% 46% 70% % identified zero words 13% 17% 31% 16% Source: Student survey; IMPAQ calculation. N = 422 for Grand Bassa, 121 for Rivercess, and 71 for River Gee. Exhibit 40 also shows that the overall distribution of the number of words identified has a downward trend with the plurality of students naming just one to five words correctly. Exhibit 40. Distribution of Most Used Words Identified by County Source: Student survey, IMPAQ calculation. N=614. Decoding (Invented Word Recognition) We also included a decodable word test in the LBRA to measure the students’ ability to recognize the basic sounds and phonemes. We rearranged the 20 most common words (from the word recognition test) to form “pseudo words” and asked students to decode them. Students struggled with this task: they identified zero words correctly on average. Only 11 percent were able to decode even the easiest invented word. Exhibit 0 5 10 15 20 25 30 35 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Grand Bassa Rivercess River Gee IMPAQ International, LLC Page D2 LEARN Second Cohort Baseline Report 41 shows that 83 percent of the sample could not read even one word. There were no large differences in these numbers between county, gender, or first language. Exhibit 41. Invented Word Recognition Indicator Mean/Percent Total number of correctly read invented words 0 % of invented words read correctly 2% % identified hardest invented word (phe and yill) 0% % identified easiest invented word (ne) 11% % identified zero invented words 83% Source: Student survey, IMPAQ calculation. N =614. Word and Invented Word Recognition Similar to the letter knowledge subsection, the word recognition subsection asked students to identify words from a list of 20 real words followed by a list of 20 invented words. Compared to the letter knowledge section, Grade 2 students performed poorly on this section, identifying only 31 percent of the real words correctly. Overall, 16 percent of the sample could not read a single word correctly. IMPAQ International, LLC Page E1 LEARN Second Cohort Baseline Report APPENDIX E: REGRESSION ANALYSES Exhibit 42 summarizes associations between students or household characteristics and students’ literacy skills. The table shows either positive or negative associations that are statistically significant (p < 0.05). A “+” indicates that the factor is positively associated with the respective literacy outcome, while a “-“ indicates a negative association. Detailed regression results can be found in the full OLS regression table in Exhibit 42 below. Exhibit 42. Summary of Predictors of Literacy Skills at Baseline Letter Knowledge Word Recognition Invented Word Recognition Reading – Accuracy (total words) Reading – Accuracy (attempted words) Reader Listening Comprehension Age – – – Female – – + English – – + Repeated a grade – Caregiver attended school – – Reading materials Home literacy index + SES Index + + Source: Student survey; IMPAQ calculation. Note: Only statistically significant predictors (p-value < 0.05) are presented in the table. IMPAQ International, LLC Page E2 LEARN Second Cohort Baseline Report Exhibit 43. Predictors of Literacy Skills at Baseline Letter Knowledge Word Recognition Invented Word Recognition Reading – Accuracy (total words) Reading – Accuracy (attempted words) Reader Listening Comprehension Age 0.001 -0.003*** -0.000*** 0.000 0.000 -0.000*** -0.000 (0.001) (0.001) (0.000) (0.000) (0.000) (0.000) (0.000) Female 0.099 -1.074*** -0.156 -0.010** 0.028*** -0.010 -0.006 (0.301) (0.396) (0.104) (0.005) (0.010) (0.022) (0.033) English -1.086** -0.441 -0.279** 0.013** 0.026 0.022 -0.000 (0.484) (0.955) (0.139) (0.006) (0.018) (0.026) (0.042) Repeated a grade -0.418 -0.538 0.115 -0.012** 0.003 -0.024 0.008 (0.376) (0.494) (0.086) (0.004) (0.017) (0.032) (0.037) Caregiver attended school -0.001 -0.002** -0.000 0.000 0.000 -0.000 -0.000* (0.001) (0.001) (0.000) (0.000) (0.000) (0.000) (0.000) Reading materials 0.122 0.131 0.059 0.002 0.006 0.011 0.004 (0.109) (0.216) (0.056) (0.002) (0.006) (0.010) (0.012) Home literacy index 0.317*** 0.132 0.013 0.000 -0.007 0.009 0.009 (0.117) (0.144) (0.038) (0.002) (0.004) (0.007) (0.009) SES Index 0.220 0.409 -0.028 0.007** -0.001 0.028** -0.035* (0.170) (0.306) (0.063) (0.003) (0.010) (0.013) (0.019) Constant 22.240*** 6.916*** 0.415*** 0.026*** 0.434*** 0.087*** 0.124*** (0.703) (0.945) (0.107) (0.005) (0.017) (0.029) (0.035) R2 0.057 0.039 0.029 0.055 0.025 0.028 0.015 N 614 614 614 609 609 614 580 Source: Student survey; IMPAQ calculation. *p-value <0.1, ** p-value < 0.05, *** p-value < 0.01 Note: Robust standard errors are clustered at the school level and shown in parentheses below the coefficients. IMPAQ International, LLC Page E3 LEARN Second Cohort Baseline Report Exhibit 44. Associations Between School Characteristics and Literacy Skills (Performance Sample) Letter Knowledge Word Recognition Invented Word Recognition Reading – Accuracy (total words) Reading – Accuracy (attempted words) Reader Listening Comprehension Number of boys enrolled in Grade 2 -0.118 -0.357** -0.051** -0.001 0.005** -0.001 -0.010** (0.103) (0.139) (0.019) (0.001) (0.003) (0.003) (0.004) Number of girls enrolled in Grade 2 0.101 0.331** 0.050*** 0.001 -0.006*** 0.002 0.010** (0.094) (0.126) (0.017) (0.001) (0.002) (0.002) (0.004) Number of toilets 0.171 0.559* 0.110** 0.002 -0.014*** 0.006 0.024 (0.113) (0.284) (0.046) (0.002) (0.005) (0.008) (0.016) Presence of handwashing station 1.021* 1.561 0.024 0.002 -0.023 0.034 0.014 (0.535) (1.062) (0.150) (0.007) (0.019) (0.032) (0.064) Water available for drinking -0.134 0.036 0.080 0.006 -0.026 -0.028 0.001 (0.389) (0.726) (0.121) (0.006) (0.019) (0.026) (0.052) Food preparation at school 0.337 1.068 0.113 0.018** 0.029 0.035 -0.096 (0.657) (1.273) (0.164) (0.008) (0.026) (0.035) (0.067) Library or book bank for students to take books home -2.762* -3.458** -0.402** -0.007 0.058 -0.065 -0.084 (1.524) (1.540) (0.193) (0.015) (0.038) (0.040) (0.055) Temporary vs Permanent School -0.949 -0.544 -0.131 -0.018 0.035 -0.014 0.014 (0.606) (1.366) (0.242) (0.012) (0.026) (0.052) (0.083) Constant 22.971*** 5.076*** 0.119 0.018*** 0.510*** 0.107*** 0.152** (0.531) (0.900) (0.134) (0.006) (0.025) (0.037) (0.061) R2 0.081 0.121 0.057 0.039 0.064 0.011 0.052 N 556 556 556 553 553 556 525 Source: Student survey; IMPAQ calculation. *p-value <0.1, ** p-value < 0.05, *** p-value < 0.01 Note: Robust standard errors are clustered at the school level and shown in parentheses below the coefficients. IMPAQ International, LLC Page F1 LEARN Second Cohort Baseline Report Appendix F: INTER-RATER RELIABILITY Reading Assessment To measure the reliability and level of homogeneity of enumerators’ scores on children’s literacy skills, two different enumerators simultaneously assessed 6 percent of the respondents of the overall second-grade sample (47 out of 852). IMPAQ used Long one-way Analysis of Variance (ANOVA) techniques, which is used to determine whether the mean of a dependent variable is the same in two or more unrelated and independent groups, to calculate the intra-class correlation within pairs of assessors for a measure of inter-rater reliability. Adapted from Fleiss et al. (1973), we interpreted the intra-class correlations as it follows:  Less than .40 – Poor  Between .40 and .75 – Good or fair  Greater than .75 – Excellent Exhibit 45 shows the percent of agreement between the raters, as well as inter-rater reliability ratings for the project evaluation sample. Overall, the inter-rater reliability (IRR) across the project evaluation sample was excellent for most of the literacy skills measures and good for two of them, showing high internal validity of the scores. For readers and reading comprehension, however, there were no variations in the proportion of children who were able to answer at least 80 percent of comprehension questions. Therefore, the ANOVA test could not calculate the IRR. Exhibit 45. IRR by Literacy Skill Subtests for Performance Sample Literacy Skill Sub-test IRR Rating Letter Knowledge 93% Excellent Word Recognition 99% Excellent Reader n/a n/a Fluency 92% Excellent Accuracy (out of the whole passage) 99% Excellent Accuracy (out of the words attempted) 79% Excellent Reading Comprehension n/a n/a Listening Comprehension 79% Excellent Source: Student survey, IMPAQ calculation. N = 61 Grade 2 students. Overall, the IRR was good or excellent. To maintain the good internal validity of the scores, and improve the administration and scoring of the LBRA, we will provide further training at midline and endline. IMPAQ International, LLC Page G1 LEARN Second Cohort Baseline Report APPENDIX G. EVALUATION INSTRUMENTS BASELINE DATA COLLECTION FOR USDA FOOD FOR EDUCATION (LEARN) IN LIBERIA IMPACT AND PROJECT EVALUATION Student Survey Start Time Date INTRODUCTION County Grand Bassa Rivercess River Gee Districts Enter the name of the district ----------------------------- school name Enter the school name ------------------ enum Enter your name -------------------------------- Consent Has the principal given consent for the child to participate in this survey? 0. No  thank them and terminate the survey and select the next child on your list. 1. Yes  ” assent” I__I If teacher says No, thank them, and terminate the survey and proceed to the next child on your list. Student Code stcode1 Please enter the student code CAREFULLY------------ stcode2 Please enter the student code CAREFULLY again ------------ Reliab Is this an individual assessment or a pair assessment? 0. Individual  “nickname” 1. Pair assessment  ”reliabtype” I__I IMPAQ International, LLC Page G2 LEARN Second Cohort Baseline Report Reliabtype Talking enumerator or observing enumerator? 0. Observing 1. Talking I__I Please get the student code from the team leader. It is very important to use the correct student code, so please enter the code twice. If you are unsure, please check again with the team leader Dear student: Hi, my name is ___, and I am with Center Action and Research Training. I am here asking some questions from children like you to understand more about a reading program. Your answers will help us make Liberia's education system better. Your parents, your classmates and your teachers will not know your answers to the questions. Everything you say will be kept a secret. There aren’t any right or wrong answers. I want you to answer honestly and as best as you can. It will take only 30 to 35 minutes. Do you have any questions for me? You can interrupt me to ask a question at any time. Also, if you don't know the answer to a question or don't want to answer it, just let me know and we can skip it. I will just start with a few questions to know you better, and then we will play a reading game. Are you ready to begin? assent Do you agree to answer the questions I have? 0. No  thank him/her, terminate the survey, and proceed to the next child on your list. 1. Yes  continue to the background section. I__I *Select only one option If child says No, thank him/her, terminate the survey, and proceed to the next child on your list. Background information [DON’T READ TO THE CHILD] Fname What is your first name? Lastname What is your last name? Caregivername What is the name of the person that takes care of you at home most of the time? Caregiver Who is (caregivername)’s to you? 1. Mother 2. Father 3. Older sister 4. Older brother 5. Grandmother 6. Grandfather 7. Other female relative 8. Other male relative IMPAQ International, LLC Page G3 LEARN Second Cohort Baseline Report 9. Female non-relative 10. Male non-relative 888. Don’t know Caregiverschool Did (caregivername) go to school when she/he was small? 0. No 1. Yes 888. Don’t know/No response *Select only one option gender 0. Male 1. Female I__I *Ask only if necessary age How old are you? …… *RECORD AGE >=5 & <25 *Mark 888 if no response/don’t know grade Which grade/class are you in? 1. Grade 2 2. Grade 6 3. Other  Thanks the child and terminate the survey I__I *Select only one option everrpt Did you repeat any grades? 0. No 1. Yes 888. Don't know/ No response I__I *Select only one option studattend During the last week of school, how many days did you attend school? 1. 1 2. 2 3. 3 4. 4 5. 5 888. Don't know/No response I__I *Select only one option mainlang What language do you speak at home most often? 1. English 2. Kpelle 3. Grebo 4. Krahn 5. Bassa 6. Kru 7. Lorma 8. Belleh 9. Sapo 10. Other I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Do not read options *Select only one option IMPAQ International, LLC Page G4 LEARN Second Cohort Baseline Report 888. Don’t Know otherlang At home, do you speak any other languages? 1. English 2. Kpelle 3. Grebo 4. Krahn 5. Bassa 6. Kru 7. Lorma 8. Belleh 9. Sapo 10. Other 11. No 888. Don’t Know I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I *Select all that apply *Do not read the options ses In your home, do you have any of the following items that I will read to you? 1. CELL PHONE 2. CURRENT/LIGHT/GENERATOR 3. ICE BOX 4. BICYCLE 5. TV 6. MOTORBIKE/PEMPEM 7. CAR 8. KEHKEH 9. None 888. Don't know I__I I__I I__I I__I I__I I__I I__I I__I I__I *Please read all the options to the child and select all that apply book At home do you have : 1. TEXTBOOKS/SCHOOLBOOKS 2. NEWSPAPERS 3. STORYBOOKS/COMICS 4. COLORING AND DRAWING BOOKS 5. HOLY BOOK (BIBLE OR KORAN) 6. None 888. Don't know I__I I__I I__I I__I I__I I__I I__I *Please read all the options to the child and select all that apply WASH [DON’T READ TO THE CHILD] Okay, now I have some questions about hygiene. IMPAQ International, LLC Page G5 LEARN Second Cohort Baseline Report hand1 Did you wash your hands at all yesterday? 0. No hand4 1. Yes 888. Don’t know I__I *Select only one option hand2 When did you wash your hands yesterday? 1. After using the toilet (poo poo) 2. After using the toilet (pee pee) 3. Before eating food 4. When they were dirty 5. After eating 6. After playing 7. Before preparing food 8. After helping someone else use the toilet 9. Other 888. Don't know/ No response I__I I__I I__I I__I I__I I__I I__I I__I * Probe if the child refers to the time s/he washed he/his hands, ask them why they washed their hands at that time *Do not read the options to the child. *Select all that apply. hand3 What did you use to wash your hands yesterday? 1. Water only 2. Water and soap 3. Water and ash 4. Other 888. Don't know/ No response I__I *Do not read the options to the child. *Select only one option hand4 When should you wash your hands? 1. After using the toilet (poo poo) 2. After using the toilet (pee pee) 3. Before eating food 4. When they were dirty 5. After eating 6. After playing 7. Before preparing food 8. After helping someone else use the toilet 9. Other 888. Don't know/ No response I__I I__I I__I I__I I__I I__I I__I I__I *Do not read the options to the child. *Select all that apply. Food Security [DON’T READ TO THE CHILD] Thank you! Now, I would like to ask you some questions about food. eatfreq How many times do you eat per day? 1. More than three times per day 2. Three times per day 3. Twice per day 4. Sometimes two times, sometimes one time 5. Once per day I__I *Select only one option IMPAQ International, LLC Page G6 LEARN Second Cohort Baseline Report 6. I eat once a day and sometimes not eat at all 888. Don't know/ No response diet1 Do you know what does a "balanced diet" mean? 0. No  diet3 1. Yesdiet2 888. Refuse to answer  diet3 I__I *Do NOT probe if the child does not understand probe *Select only one option diet2 Can you explain to me what a balanced diet is? 1. Eating foods that give us energy to play, work, learn (Go) 2. Eating foods that help us grow (Grow) 3. Eating foods that protect us from disease (Glow) 4. None of the above 888. Don't know/ No response I__I I__I I__I I__I I__I *Probe if needed but do NOT read the options to the child * Select all that apply * For programming purpose - restrict selection of None of the above and 888 with other options. diet3 Can you name foods that give you energy to play and learn? 1. Grains like maize (corn), rice, fufu, bulgur, or pasta 2. Sweet foods like sugarcane, sugar, or honey 3. Roots like potato, yam, cassavas, eddos, or sweet potato 4. Fats like margarine (butter), or oils 5. Other 888. Don't know/ No response I__I I__I I__I I__I I__I I__I *Probe if needed but do NOT read the options to the child * Select all that apply * For programming purpose - restrict selection of None of the above and 888 with other options. diet4 Can you name foods that help your body grow? 1. Dairy products like milk, yogurt, and cheese 2. Red meat 3. Poultry (chicken) 4. Fish 5. Eggs 6. Beans, peas, legumes/pulses like seeds and nuts 7. Other 888. Don't know/No response I__I I__I I__I I__I I__I I__I *Probe if needed but do NOT read the options to the child * Select all that apply * For programming purpose - restrict selection of None of the above and 888 with other options. diet5 Can you name foods that protect your body from disease? 1. Green leafy vegetables like potato greens, spinach, collard green, cassava greens, watergreens I__I I__I I__I I__I I__I I__I * Do NOT read the options to the child * Select all that apply * For programming purpose - restrict selection of None of IMPAQ International, LLC Page G7 LEARN Second Cohort Baseline Report 2. Fruits like mango, banana, pawpaw, oranges, pineapple, watermelon, or cucumber 3. Okra 4. Cauliflower 5. Pumpkin 6. Other 888. Don't know/ No response the above and 888 with other options. diet6 How do you think the food should be divided between boys and girls? 1. Boys should get more 2. Girls should get more 3. Boys and girls should get equal amounts 888. Don't know/ No response I__I *Select only one option That's great! You did a good job! Now I want to ask you a couple of questions about your school. canteen1 Did you eat a meal that was prepared at school for free yesterday? 0. No 1. Yes 2. No food was prepared 888. Don't know/ No response I__I *Select only one option *Probe if necessary *If the interview is on Monday, ask the child about Friday or the last time the child was at school. If the child was absent yesterday, ask about the last time the child was at school. SCHOOL ENVIRONMENT AND PARTICIPATION [DON’T READ TO THE CHILD] The following questions are only for Grade 2 students. enviro1 What do you like best about your class and school? 1. Like teacher 2. Learning new things/enjoy lessons 3. Participate in classroom games and activities 4. Playing a sport at school 5. Access to water 6. Access to clean toilet 7. Food is provided 8. Being with my friends 9. Other (Specify ...) 888. Don't know/ No response I__I I__I I__I I__I I__I I__I I__I I__I I__I *Select all that apply. *Do not read the options to the child. IMPAQ International, LLC Page G8 LEARN Second Cohort Baseline Report enviro2 What do you not like about your class and school? 1. Teacher is mean to me/other students 2. S/he punishes me/ hits me/other students 3. Teacher asks for money 4. Lessons difficult to understand/learn 5. Not learning much at school 6. Poor toilet conditions/lack of toilets 7. No access to water 8. No food is provided/the food is bad 9. Other students tease me/fight with me/other students 10. Lack of uniform 11. Lack of learning materials 12. Lessons are boring 13. Other (specify) 888. Don't Know/ No response I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I I__I * Do not read the options to the child * Select all that apply *Note to enumerators: Mean can be yelling, laughing at students, or humiliating them, etc. enviro3 How many times in the last week did your teacher come to class? 1. Every day (5 days) 2. A few times during the week (2-4 days) 3. Once during the week 4. Never  enviro5 888. Don’t know/No reponse *Read the list to the respondent, but don't read 'don't know' *Select only one enviro4 How many times in the last week did your teacher come late or miss a portion of the class? 1. Every day (5 days) 2. A few times during the week (2-4 days) 3. Once during the week 4. Never 888. Don’t know *Read the list to the respondent, but don’t read “don’t know” *Select only one enviro5 Does your school have books other than textbooks/schoolbooks for you to borrow? If yes, is it free, or do you have to pay money? 0. No  nhhold 1. Yes, we can take books, but not off campus  nhhold 2. Yes, we can take books home and it is free  enviro5a 3. Yes, we can take books home but it costs money  enviro5a I__I *Select only one option IMPAQ International, LLC Page G9 LEARN Second Cohort Baseline Report 888. Don't know enviro5a How many times in the last week did you borrow books other than textbooks/school books from school to take home to read? 1. Every day 2. A few times during the week; 3. Once during the week; 4. Never 888. Don't know I__I *Read the list to the respondent, but don't read 'don't know' *Select only one Household Environment [DON’T READ TO THE CHILD] We are almost done! We have a few more questions about your home. Nhhold How many people are there in your household, including yourself? ……. *Define the household for the child as a place where its members live with each other, eat out of the same pot *Record the number > 0 & < 30 hh1 In the last week, did you see anyone in your house reading? 0. No hh2 1. Yes hh1a 888. Don't know I__I *Select only one option hh1a Who did you see reading last week? 1. Mother 2. Father 3. Older sister 4. Younger sister 5. Older brother 6. Younger brother 7. Grandmother 8. Grandfather 9. Other female relative 10. Other male relative 11. Female non-relative 12. Male none-relative 888. Don't know *Select all that apply IMPAQ International, LLC Page G10 LEARN Second Cohort Baseline Report hh2 In the past week, did anyone in your household help you with your studies/school work? 0. No hh3 1. Yes hh2a 888. Don't know I__I *Select only one option hh2a Who helped you study? 1. Mother 2. Father 3. Older sister 4. Younger sister 5. Older brother 6. Younger brother 7. Grandmother 8. Grandfather 9. Other female relative 10. Other male relative 11. Female non-relative 12. Male none-relative 888. Don't know I__I *Select all that apply hh3 In the past week, did anyone in your house read to you? 0. No hh4 1. Yes hh3a 888. Don't know I__I *Select only one option. hh3a Who read to you? 1. Mother 2. Father 3. Older sister 4. Younger sister 5. Older brother 6. Younger brother 7. Grandmother 8. Grandfather 9. Other female relative 10. Other male relative 11. Female non-relative 12. Male none-relative 888. Don't know *Select all that apply hh4 In the past week, did anyone in your house tell you a story? 0. No readout1 1. Yes hh4a 888. Don't know I__I *Select only one option. hh4a Who told you a story? *Select all that apply IMPAQ International, LLC Page G11 LEARN Second Cohort Baseline Report 1. Mother 2. Father 3. Older sister 4. Younger sister 5. Older brother 6. Younger brother 7. Grandmother 8. Grandfather 9. Other female relative 10. Other male relative 11. Female non-relative 12. Male none-relative 888. Don't know readout1 During the last week, did you read books other than textbooks/schoolbooks outside of school? 0. No 1. Yes 888. Don't know I__I *Select only one option readout2 Outside of your school or home, where else can you go to read or borrow books (other than textbooks)? 1. Community library 2. Church/Mosque or any other religious building 3. Reading clubs 4. Friends or relatives 5. Other 888. Don't know/ No response I__I I__I I__I I__I I__I *Select all that apply Sexual and Gender-based Violence [DON’T READ TO THE CHILD] Thank you! Now, I would like to ask your opinion about something. There is no right or wrong answer. sgbv1 Are there rules for the ways that teachers should treat students in school? 0. No sgbv3 1. Yes sgbv2 888. Don't know I__I Probe if needed sgbv2 What are they? 1. Teachers are not allowed to be in a relationship with students 2. Teachers are not allowed to beat students 3. Teachers are not allowed to beat students too much * Do not read the options to the child * Select all that apply IMPAQ International, LLC Page G12 LEARN Second Cohort Baseline Report 4. Teachers are not allowed to use humiliating language on students 5. Teachers are not allowed to ask students for money 6. Other (specify) 888. Don't know sgbv3 How do teachers discipline students at school? 1. Give extra work/assignments 2. Dismiss students from class 3. Physical violence (hitting students) 4. Humiliating language 5. Made to clean or work at the school 6. Other (specify) 888. Don't know/No response I__I I__I I__I I__I I__I * Probe if needed *Do not read the options to the child * Select all that apply sgbv4 If children are teased or touched in a way they don't like at school, what do they do? 1. Tell their teacher 2. Tell the principal or registrar 3. Tell their parents 4. Nothing 5. Other (specify) 888. Don't know/No response * Probe if needed *Do not read the options to the child * Select all that apply I'm going to read you things that some children agree with and some children disagree with. After I read each one, please tell me if you yes you agree or no you disagree. The following questions are only for Grade 6 students. gender1 If a boy touches a girl at school, it’s because the girl did something to attract him 1. Disagree 2. Agree 888. No response/Not sure I__I *Select only one option gender2 There are times when a boy needs to beat his girlfriend 1. Disagree 2. Agree 888. No response/Not sure I__I *Select only one option gender3 Girls like to be teased by boys 1. Disagree 2. Agree 888. No response/Not sure I__I *Select only one option gender4 When girls wear short skirts they are telling boys or men to touch them I__I *Select only one option IMPAQ International, LLC Page G13 LEARN Second Cohort Baseline Report 1. Disagree 2. Agree 888. No response/Not sure gender5 For girls to get good grades, they sometimes have to let their teachers touch them or love them 1. Disagree 2. Agree 888. No response/Not sure I__I *Select only one option Disability [DON’T READ TO THE CHILD] Thank you! You are doing a great job! We are almost done! Then we can play the reading game! dis1 Do you have difficulty seeing? For example, is it difficult to see the chalkboard when you are at school, even if you sit near the front of the classroom, or when you wearing your glasses (mention this example if they wear glasses)? What about when you sit at the back of the classroom? 0. No – no difficulty 1. Yes – some difficulty 2. Yes – a lot of difficulty 3. Cannot do at all 888. Don't know I__I *Select only one option ***Make sure difficulty is not because students are blocked by taller students in front of them dis2 Do you have difficulty hearing? For example, if you were in the main room of your house, could you hear someone talking in a normal voice on the other side of the room, or even when you wearing your hearing aid (only ask if you see they have hearing aid)? 0. No – no difficulty 1. Yes – some difficulty 2. Yes – a lot of difficulty 3. Cannot do at all 888. Don't know I__I *Select only one option dis3 Do you have difficulty walking or climbing steps? For example, is it difficult to move around in your home? 0. No – no difficulty 1. Yes – some difficulty 2. Yes – a lot of difficulty 3. Cannot do at all 888. Don't know I__I *Select only one option IMPAQ International, LLC Page G14 LEARN Second Cohort Baseline Report LITERACY BOOST ASSESSMENT: Understanding Letters 1. Give the child the list of letters and say to the child: 2. Say: Let’s look at some letters. Can you start here (point to first letter) and tell me what these letters are moving in this direction? (indicate left to right direction) Do you understand? Ok, you can begin. 3. Mark the letters correct or incorrect as the child reads. 4. Correct letters are: • the letter name in the home language or language of instruction • any sound that is acceptable for in the home or instructional language • a response which says “It begins like…” giving a word for which the letter is the initial letter 5. If the child read the letters out of order, then remember to bring his/her attention to the ones they might have skipped. 6. Make sure you marked all of the letters 7. Move to the Most Used Words section. What to do if a student is struggling: • If the student is struggling, and hesitates at any letter for five seconds, ask follow up questions: Do you know its name? What sound does it make? Do you know a word that starts with this letter? • If the student still hesitates for five seconds, ask: Can you tell me any of these letters? • If the student still hesitates for five seconds, then stop and thank him/her for trying his/her best. • Mark letters not identified or not attempted as incorrect. • Move to the Most Used Words section. x v s o a k g c f b p l h d z t q m i e w u r n j y Most Used Words 1. Give the pupil the laminated copy of the "Most Used Words" list. 2. Say: I would like you to read some words to me. They are words from your textbook. Please point to and say each of these words starting here (point to first word) and moving across each line like this (indicate left to right direction). Do you understand? Ok, you can begin. IMPAQ International, LLC Page G15 LEARN Second Cohort Baseline Report 3. Mark the words correct or incorrect as the child reads 4. Remember that pronunciations of words in local dialects are acceptable. 5. If the child read the words out of order then remember to bring his/her attention to the ones they might have skipped. 6. Make sure you marked all of the words. 7. Move to the Decoding Section. What to do if a student is struggling: • If the student is struggling, and hesitates at any words for five seconds ask the child, Are there any words on the list that you know? Tell me or say the words you know. Repeat the request to encourage the child to continue. • If the student still hesitates for five seconds, then stop and thank him/her for trying his/her best. • Mark words not identified or not attempted as incorrect. your his uncle we school girls want help and said story room go she will ask not was mother did IMPAQ International, LLC Page G16 LEARN Second Cohort Baseline Report Invented words 1. Give the pupil the laminated copy of the "Invented Words" list. 2. Say: I would like you to read another list of words to me. These words are not real words, rather they are words that we made up ourselves. But they can still be read. Please point to and say each of these words starting here (point to first word) and moving across each line like this (indicate left to right direction). Do you understand? Ok, you can begin. 3. Mark the words correct or incorrect as the child reads. 4. Remember that pronunciations of words in local dialects are acceptable. 5. If the child read the words out of order then remember to bring his/her attention to the ones they might have skipped. 6. Make sure you marked all of the incorrect words. 7. Move to the Reading Passage section. What to do if a student is struggling: • If the child hesitates at any word for five seconds, ask the child, Are there any words on the list that you know? Tell me or say the words you know. Repeat the request to encourage the child to continue. • If the student still hesitates for five seconds, then stop and thank him/her for trying his/her best. • Mark words not identifies or attempted as incorrect. • Move to the Reading Passage section. jour mir undle ne sprood kirls vakt gelb alt baid flory koom vo phe yill asb dok sar rothem thu IMPAQ International, LLC Page G17 LEARN Second Cohort Baseline Report COMPREHENSION PASSAGES AND QUESTIONS 1. Give the pupil the reading passage. 2. Say: I am going to give you a reading passage to read. When I say 'begin,' start reading aloud from the title on this page. Try to read each word. If you come to a word you don't know, I'll tell it to you. Be sure to try to do your best reading. Do you understand what I want you to do? 1. Say: 'Begin' and when the pupil begins to say the first word of the title press START. 2. As the pupil reads, follow along on your screen. Click on words read incorrectly (they will turn with a line through them). 3. If the pupil stops reading before the end of the passage, encourage the pupil to keep reading. Show the pupil where he/she stopped, if necessary. Follow along on your copy. If the child does not want to or cannot read anymore, stop the timer and select the last word the child read. Thank the child for reading it and read it out to him/her. 4. After 30 seconds, a message will flash, “Please mark the item being attempted.” Mark the word that the child was reading when the message came, and a blue box will appear around it. 5. When the screen flashes at the end of 30 seconds, do a quick count of the correct words. • If the pupil has read less than 5 words correctly, then: o Politely stop the child and Press “Finish” box to stop the timer. Say: Thank you. o Read the passage to them. o On the next page, mark NONREADER o And ask them comprehension questions. • If the pupil has read 5 or more words correctly, then: o Select the box under the word being read/attempted by the child at 30 seconds. o Allow the pupil to finish the passage. o Continue marking which words are read incorrectly by clicking on them. o As soon as the pupil finishes the last word of the passage, click the FINISH button. Say: Thank you. o On the next page, for the question, ‘Was the student a reader or nonreader?’ mark READER. o Move to the Reading Comprehension questions What to do if a student is struggling: • If the pupil is struggling and fails to correctly pronounce a word within five seconds, tell him/her the word and mark it as an error by clicking on it (the word should appear with a line through it). IMPAQ International, LLC Page G18 LEARN Second Cohort Baseline Report The Lone Star Kite! One hot day, all the children were outside playing. Many were flying kites high in the sky. Flomo looked at the kite that his older brother Moses made for him. It had red and white stripes and a blue lone star at the top. It looked great. Flomo was proud of his kite. He ran up the hill. Flomo ran so fast that he fell down and broke his kite. Flomo began to cry. Moses came down from the hill. “Why are you crying?” he asked. “My kite is broken,” said Flomo. “I will fix it,” said Moses. Flomo trusted his brother. Moses fixed the kite with glue. He handed it to Flomo. “Try it now!” Flomo ran and the wind carried the kite in the air. All the children came running to look at the beautiful Lone Star kite. Flomo was right – his big brother always knew what to do. reader Is child a reader or a non-reader? 0. A non-reader read fewer than 5 words accurately 30 seconds) 1. A reader (read correctly 5 per 30 seconds) I__I Select only one option Comprehension Questions Comp1 What happened in the story? 1. Flomo wants to fly the kite that his brother made 2. Flomo falls and breaks his kite 3. Flomo’s brother fixes the kite 4. Flomo is able to fly the kite 5. None I__I mark every main point mentioned by the child Comp2 Who made the kite for Flomo? (His older brother, Moses) 0. False 1. True I__I Don’t read the answer to them Comp3 What did the kite look like? (Lone Star/red and white stripes with blue star) 0. False 1. True I__I Don’t read the answer to them Comp4 How did the kite break? (Flomo tripped and dropped it) 0. False 1. True I__I Don’t read the answer to them Comp5 Who fixed Flomo’s kite? (his brother, Moses) 0. False 1. True I__I Don’t read the answer to them Comp6 How did Moses fix the kite? (with glue) 0. False 1. True I__I Don’t read the answer to them Comp7 Does the kite fly at the end of the story? (yes) 0. False 1. True I__I Don’t read the answer to them IMPAQ International, LLC Page G19 LEARN Second Cohort Baseline Report Comp8 Why was Flomo proud of his kite? (his brother made it for him/it was a Lone Star kite) 1. Student could explain their answer with information from the story 2. Student could NOT explain their answer with information from the story I__I Don’t read the answer to them Comp9 How did Flomo feel after he broke his kite? (Sad or depressed) 0. False 1. True I__I Don’t read the answer to them Comp10 Why do you think Moses was a good brother? 0. False 1. True True if student can support opinion with details from story Thank you very much for answering my questions. End time ………………….. Comment ………………………. IMPAQ International, LLC Page G20 LEARN Second Cohort Baseline Report Parents: Focus Group Discussion ___________________________________________________________________ • Here as researchers for a project that Save the Children is starting. We are not funding any programs or school, we are providing feedback to from parents to STC to improve planned activities • Everything is confidential, we will not record anyone’s name or share anything they say with teachers or principals. Respect each other and do not repeat this conversation outside of here • No right or wrong answers – it’s ok to disagree, because we want everyone’s opinion. Everyone should speak freely, and respect each other • You do not have to answer a question if you do not want • Can we record the discussion for notes? • Do you have any questions for us before we begin? 1. Do you think children need to go to school? For how long? Are there any differences between boys and girls? 2. How do children benefit from school? What are both the immediate benefits and future benefits? Are there different benefits for boys and girls? 3. Do you encourage your children to study at home? Why? Why not? 4. Do you tell your children stories? How often? Which children – how old? Both girls and boys? If not, why not? 5. Do you have storybooks or other reading materials such as newspapers at home? [If no] why not? [If yes], do your children read them, or do you read to your children? How often? Which children – how old? Both girls and boys? [If parents say yes] Is this typical of families in your community? 6. Does your children’s school have books that the children can borrow to take home? Is there another place in your community where children can borrow books? [If yes] how often do your children bring books home? Do you encourage them to borrow books? Why or why not? [If no] what do you think of the idea? 7. In some communities, not all children are able to attend school on a regular basis. Does this happen in your community? Are there some children who attend school more than others? What prevents some children in this community from going to school? Are there different reasons that prevent boys and girls from going to school? What usually happens when children are unable to attend regularly? 8. What do you think would encourage children in your community to go to school more often? What are the specific encouragements needed for girls? For boys? (Spend time on this question and probe – for example, if they parents say “money” ask specifically what the costs are, and why they are prohibitive. Try to get a lot of responses here – keep following up, “is there anything else?” Encouraging stories of real children in the community might be helpful.) 9. Do your children go willingly and look forward to attending school? What do they like/dislike about going to school? (Probe for specific examples of their own children and what they like / dislike) 10. What do you like about the school your child attends? What, if anything, could be better? Do you know your child’s teacher or teachers? What do you like (or not like) about your child’s teacher or teachers? IMPAQ International, LLC Page G21 LEARN Second Cohort Baseline Report 11. Do you know if there are any rules in place for how teachers should treat students in school? Do you think these rules are followed, or sometimes broken? (probe for beating, sex for grades, teacher/student relationships) How do you go about handling any issues that arise in the school? If there are problems, what is the main barrier keeping it from being addressed? 12. Are you part of a parent group, such as a parent teacher association that collaborates with your school? a. If yes, what kind of activities does this group do? Do you have any suggestions on how to improve collaboration? What would a good PTA be able to do in this school? b. If no, do you think a group like this would benefit your school? What activities would you expect from such a group? Would you be interested in joining? Who from your community do you expect would want to join? 13. Do you think teachers will accept if the parent body has a role to monitor their attendance, their behaviour with the children, rewarding good teachers? 14. Is there anything that I did not ask about that you would like to share with me, or do you have any additional thoughts about what we have discussed today? Thank you all very much for your sharing with me today, your feedback is much appreciated! IMPAQ International, LLC Page G22 LEARN Second Cohort Baseline Report Principals: Key Informant Interview ___________________________________________________________________ • Here as researchers for a project that Save the Children is starting. We are not funding any programs or school, we are providing feedback to from parents to STC to improve planned activities • Everything is confidential, we will not record anyone’s name or share anything they say with teachers or with anyone else. Respect each other and do not repeat this conversation outside of here • No right or wrong answers – please speak freely • You do not have to answer a question if you do not want • Can we record the discussion for notes? • Do you have any questions for us before we begin? 1. Let’s start by talking a little bit about your background – how long have you been a principal? How long have you been at this school? 2. What are the challenges families in this community face in sending their children to school and supporting their education? (Probe to understand if there are different reasons that prevent boys and girls from going to school, and if there are different challenges by age group or class level) 3. Do you feel that boys and girls have equal access to education in your community? By ‘equal access’ we mean: do girls and boys have the same chance to go to school, or do girls or boys have more responsibilities in the household that would prevent them from going to school, or maybe there could be differences in parent’s idea of the importance of education being more or less important for boys as opposed to girls. There may be other reasons that there could be different levels of opportunity for boys and girls in attending school. 4. What factors do you think encourage families to send their children to school? (Probe if there are gender and age differences) 5. Does your school currently have place where meals are cooked for students and teachers? How do children eat during the day (bring food from home; don’t eat; school garden; other donations, etc.)? Do you think if a meal was cooked at school for them, would it increase the attendance of students at your school? What about the attendance of teachers? Do you think providing take home rations to girls as an incentive will increase girls’ attendance at your school? Why or why not? 6. Is there a place in your community, either at school or somewhere else, for children to read or take books home from school, unrelated to their homework? If yes, do students use this? Are there are certain types of students who take books home (gender, age, others)? 7. Are parents involved at your school? Do you have any parent groups, such as a parent teacher association that collaborate with your school? a. If yes, what kind of activities does this group do? Do you have any suggestions on how to improve collaboration? What would a good PTA be able to do in this school? b. If no, do you think a group like this would benefit your school? What activities would you expect from such a group? 8. Do you think teachers will accept if the parent body has a role to monitor their attendance, their behaviour with the children, rewarding good teachers? IMPAQ International, LLC Page G23 LEARN Second Cohort Baseline Report 9. How many teachers are there at your school? On average, how long does each teacher stay at this school (do the teachers change every year?) 10. Aside from the usual reasons for missing school (being sick, taking care of family members), do teachers regularly skip coming to school, or come late/leave early? If not in your school, what about nearby schools? Why do you think that is? 11. What do you think encourages or motivates teachers to come to school, or stay engaged? What do you think would encourage teachers to have better performance? 12. Do the teachers in this school or in nearby schools hit or beat students in this school? What about humiliating language? Why do you think that is? Do you think there are any alternatives that could work better? If so, how could we convince teachers to use them? 13. Is there a problem in your school or local schools with teacher behaviour with children – e.g. asking children to do their housework; sex for grades, persuading girls (or boys) to have sex with them, or become their girlfriends? If so, why do you think that is? What could you and we do about it? 14. What are your priorities as a leader in terms of developing the school? What are the most important things you would like to see improve with regard to education in this community? 15. Is there anything that I did not ask about that you would like to share with me? Thank you for taking the time to speak with us today, your feedback is very valuable! IMPAQ International, LLC Page G24 LEARN Second Cohort Baseline Report Teachers: Focus Group Discussion ___________________________________________________________________ • Here as researchers for a project that Save the Children is starting. We are not funding any programs or school, we are providing feedback to from parents to STC to improve planned activities • Everything is confidential, we will not record anyone’s name or share anything they say with principals. Respect each other and do not repeat this conversation outside of here • No right or wrong answers – it’s ok to disagree, because we want everyone’s opinion. Everyone should speak freely, and respect each other • You do not have to answer a question if you do not want • Can we record the discussion for notes? • Do you have any questions for us before we begin? 1. Let’s start by talking a little bit about your background – how long have you been a teacher? How long have you been at this school? 2. What are the challenges families in this community face in sending their children to school and supporting their education? (Probe to understand if there are different reasons that prevent boys and girls from going to school, and if there are different challenges by age group or class level) 3. Do you feel that boys and girls have equal access to education in your community? By ‘equal access’ we mean: do girls and boys have the same chance to go to school, or do girls or boys have more responsibilities in the household that would prevent them from going to school, or maybe there could be differences in parent’s idea of the importance of education being more or less important for boys as opposed to girls. There may be other reasons that there could be different levels of opportunity for boys and girls in attending school. 4. What factors do you think encourage families to send their children to school? (Probe if there are gender and age differences) 5. Does your school currently have place where meals are cooked for students and teachers? How do children eat during the day (bring food from home; don’t eat; school garden; other donations, etc.)? Do you think if a meal was cooked at school for them, would it increase the attendance of students at your school? What about the attendance of teachers? Do you think providing take home rations to girls as an incentive will increase girls’ attendance at your school? Why or why not? 6. Is there a place in your community, either at school or somewhere else, for children to read or take books home from school, unrelated to their homework? If yes, do students use this? Do you encourage students to take home books? Are there are certain types of students who take books home (gender, age, others)? 7. Are parents involved at your school? Do you have any parent groups, such as a parent teacher association that collaborate with your school? a. If yes, what kind of activities does this group do? Do you have any suggestions on how to improve collaboration? What would a good PTA be able to do in this school? IMPAQ International, LLC Page G25 LEARN Second Cohort Baseline Report b. If no, do you think a group like this would benefit your school? What activities would you expect from such a group? Would you be interested in joining? Who from your community do you expect would join? 8. Would you, or do you think other teachers will accept if the parent body has a role to monitor their attendance, their behaviour with the children, rewarding good teachers? 9. Do you, or other teachers ever miss coming into school (other than occasionally for being sick or family reasons)? Why? What encourages you to come to school every day? Is there anything that would help encourage you to attend more frequently? 10. What are some situations when you need to punish students at school? What do you do as punishment? (Probe if they don’t mention: Do some of the teachers in this school beat students as punishment? How about use humiliating language?) Do you think there are any alternatives that could work better? 11. Is there a problem in your school or local schools with teacher behaviour with children – e.g. asking children to do their housework; sex for grades, persuading girls (or boys) to have sex with them, or become their girlfriends? If so, why do you think that is? What could you and we do about it? 12. Do you want to stay at this school, or would you prefer to move to another school? Why? 13. What are the key challenges you face in your teaching? 14. What do you like most about teaching? 15. Is there anything that I did not ask about that you would like to share with me? Thank you for taking the time to speak with us today, your feedback is very valuable!