1 February 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Social Impact, Inc. through Andrew Epstein, Miguel Albornoz, Mike Duthie, Mohammed Dawuda, Monica Gadkari, and Tommie Thompson. This publication was made possible by the support of the American people through the United States Agency for International Development (USAID). Ghana Early Grade Reading Program Impact Evaluation ii GHANA EARLY GRADE READING PROGRAM IMPACT EVALUATION 2018 MIDLINE REPORT February 2019 Cover photo courtesy of John Nkaw, Social Impact. Taken with consent, June 2018. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iii CONTENTS Acronyms ..................................................................................................................................... vii Executive Summary...................................................................................................................... 1 How Were the Results Measured? .......................................................................................................................................... 1 Findings and Conclusions............................................................................................................................................................ 1 Introduction and Background ...................................................................................................... 5 Language of Instruction in Ghana.............................................................................................................................................. 5 Project Background ...................................................................................................................... 7 Summary of the Intervention..................................................................................................................................................... 7 Project Objectives........................................................................................................................................................................ 7 Target Beneficiaries...................................................................................................................................................................... 9 Evaluation Purpose and Questions............................................................................................ 10 Evaluation Purpose.....................................................................................................................................................................10 Evaluation Questions.................................................................................................................................................................10 Evaluation Theory of Change and Measurement Approach..............................................................................................11 Evaluation Methods and Limitations......................................................................................... 13 Evaluation Methodology............................................................................................................................................................13 Impact Evaluation........................................................................................................................................................................................13 Performance Evaluation (Outcomes)......................................................................................................................................................13 Implementation Fidelity Evaluation (Inputs and Outputs)................................................................................................................13 Sampling........................................................................................................................................................................................14 Selection of Teachers and Pupils within Schools .................................................................................................................................16 Sample Realized...........................................................................................................................................................................................17 Data Collection Methods.........................................................................................................................................................17 EGR Assessment (EGRA) ...........................................................................................................................................................................17 Stallings Classroom Observation System...............................................................................................................................................19 Implementation Fidelity Observation ......................................................................................................................................................19 Interviews.......................................................................................................................................................................................................20 Data Collection...........................................................................................................................................................................21 Supervisor and Enumerator Training......................................................................................................................................................21 Data Quality Assurance..............................................................................................................................................................................21 Data Analysis...............................................................................................................................................................................22 Sample Weights...........................................................................................................................................................................................22 Principal Component Analysis...................................................................................................................................................................23 Impact Analysis.............................................................................................................................................................................................23 Limitations.................................................................................................................................................................................... 23 Selection Bias ................................................................................................................................................................................................23 Response Bias...............................................................................................................................................................................................24 Contamination and Compliance...............................................................................................................................................................24 Spillovers.........................................................................................................................................................................................................25 iv External Validity............................................................................................................................................................................................25 Impact Analysis ...........................................................................................................................................................................27 Question 1: To what extent did the EGR program improve reading skills in the Ghanaian language of instruction and prereading skills in English?.......................................................................................................................................................................27 Outcome Assessment ...............................................................................................................................................................35 Question 2: To what extent did the EGR program improve teacher and pupil classroom time on task? ..........................35 Question 3: To what extent did the EGR program increase the likelihood that teachers apply program-aligned best practices in teaching early grade reading?............................................................................................................................................40 Question 4: To what extent did the EGR program increase the likelihood that teachers engage in adaptive teaching? ..........................................................................................................................................................................................................................42 Question 5: To what extent did the EGR program increase the likelihood that teachers incorporate coach and mentor feedback?.......................................................................................................................................................................................................44 Question 6: To what extent did the EGR program increase the likelihood that pupils engage with books both inside and outside the classroom?................................................................................................................................................................................46 Fidelity Assessment....................................................................................................................................................................50 Question 7: To what extent was the program implemented according to plan?.......................................................................50 Context, Risks, and Assumptions Analysis ...........................................................................................................................58 Question 8: How have contextual factors affected program success?..........................................................................................58 Factors that Predict Success.................................................................................................................................................... 64 Question 9: What program components and pupil- and school-level characteristics were most strongly associated with program success?.........................................................................................................................................................................................64 Summary Conclusions ................................................................................................................ 70 Recommendations ...................................................................................................................... 72 Recommendations to Support Program Improvement......................................................................................................72 Recommendations for Future EGR Programming...............................................................................................................72 Annex 1. Theory of Change ....................................................................................................... 75 Annex II. Sampling Procedures ................................................................................................. 77 Annex III. Full Impact Analysis Regression Outputs ............................................................... 92 Annex IV. Full Program Predictors of Success Regression Output ....................................... 99 Annex V. Outcomes and Measurement Framework............................................................. 109 Annex VI. Data Collection Tools and Consent Forms .......................................................... 113 Annex VII. Statement of Work ............................................................................................... 267 Annex VIII. Disclosure of Conflict of Interest Forms ............................................................ 268 Annex IX. Evaluation Team..................................................................................................... 276 v TABLE OF FIGURES Figure 1: Comparative Effect Sizes by EGR Program................................................................................................................. 1 Figure 2: Results Framework for the Ghana EGR program ..................................................................................................... 8 Figure 3: Summary of School Sample Selection Procedure Using CEM...............................................................................16 Figure 4: Listening Comprehension by Treatment and Time (%) ........................................................................................27 Figure 5: Letter Sounds by Treatment and Time (CLPM).......................................................................................................28 Figure 6: Nonword Reading by Treatment and Time (CWPM)............................................................................................28 Figure 7: Oral Reading Fluency by Treatment and Time (CWPM).......................................................................................29 Figure 8: Reading Comprehension by Treatment and Time (%) ........................................................................................... 29 Figure 9: Percentage of EGRA Zero Scores at Midline, Ghanaian LOI by Subtask and Treatment Status...................30 Figure 10: Midline Level on National Reading Scale, by Treatment Status and Language ................................................30 Figure 11: EGR Program Effect Size on Ghanaian LOI Assessment, by Subtask and Gender.........................................33 Figure 12: EGR Program Effect Size on English Assessment, by Subtask and Gender......................................................33 Figure 13: Benchmarking Ghana EGR Program Effect Size Relative to Other Interventions..........................................34 Figure 14: Teacher Use of Reading Class Time (Baseline Left, Midline Right)....................................................................35 Figure 15: Comparison of Classroom Activities by Survey Wave ........................................................................................36 Figure 16: Use of Class Time by Activity and Survey Wave...................................................................................................37 Figure 17: Materials Use by Pupil Group Size............................................................................................................................38 Figure 18: Materials Use Small Group.........................................................................................................................................39 Figure 19: Materials Use by Large Group/Everyone.................................................................................................................39 Figure 20: Extent to Which Teachers Employ Phonics Methods (Self-Reported).............................................................40 Figure 21: Extent to Which Teachers Employ Phonics Methods (Observed, Midline Only) ..........................................41 Figure 22: Teacher Frequency of Assessing Pupil Reading Performance (Self-Reported)................................................42 Figure 23: Methods for Adjusting Teaching Based on Individual Assessment (Self-Reported) .......................................43 Figure 24: Source of Coaching/Mentoring Given in Last School Year..................................................................................45 Figure 25: Pupils Who Reported Reading on Most Recent School Day ..............................................................................47 Figure 26: Pupil-Reported Frequency of Being Read To at Home, Midline.........................................................................48 Figure 27: Reasons Pupils Do Not Take Library Books Home, Midline ..............................................................................48 Figure 28: Proportion of Pupils in Observed Classrooms with Their Own Workbooks................................................52 Figure 29: Supplemental Classroom Materials Observed .......................................................................................................52 Figure 30: Curriculum Pacing by Observed Classroom (Number of Days Behind Schedule).........................................53 Figure 31: Implementation Fidelity Index Structure .................................................................................................................54 Figure 32: Average Implementation Fidelity Score by Component ......................................................................................56 Figure 33: Sub-Index Distribution ................................................................................................................................................57 Figure 34: Overall Implementation Fidelity ................................................................................................................................58 Figure 35: Pupil and Teacher Language Match with Ghanaian LOI, by Category...............................................................60 Figure 36: Overall Language Match Between Pupils, Teachers, and the Official Ghanaian LOI (Based on Proportion of Pupils)............................................................................................................................................................................................61 Figure 37: Percentage of Teachers Using Official Ghanaian Language of Instruction........................................................62 Figure 38: Percentage of Classrooms with Materials in the Ghanaian Language of Instruction......................................62 Figure 39: Pupil Reasons for Being Sometimes or Regularly Absent (Self-Reported).......................................................63 Figure 40: Pupil Reasons for Being Tardy (Self-Reported)......................................................................................................63 Figure 41: Differential Program Effects on Ghanaian Subtask Scores, Demographics......................................................66 Figure 42: Differential Program Effects on Ghanaian Subtask Scores, Language Match....................................................66 Figure 43: Differential Program Effects on Ghanaian Subtask Scores, Implementation Fidelity......................................67 Figure 44: Program Component Association with Treatment Effects..................................................................................68 Figure 4545: Power versus Number of Subjects per Cluster ................................................................................................86 Figure 46: MDES versus Total Number of Clusters.................................................................................................................87 Figure 47: MDES versus Total Number of Clusters.................................................................................................................93 Figure 48: OLS Impact Analysis Model (Female Subgroup) ....................................................................................................94 Figure 49: OLS Impact Analysis Model (Male Subgroup).........................................................................................................95 Figure 50: Tobit Impact Analysis Model......................................................................................................................................96 Figure 51: Tobit Impact Analysis Model (Female Subgroup) ..................................................................................................97 Figure 52: Tobit Impact Analysis Mode (Male Subgroup)........................................................................................................98 Figure 53: OLS Predictors of Assessment Scores Model........................................................................................................99 vi Figure 54: OLS FOI Predictors of Assessment Scores Model..............................................................................................100 Figure 55: OLS Predictors of Assessment Scores Model (Rural Subgroup)......................................................................101 Figure 56: OLS Predictors of Assessment Scores Model (Urban Subgroup)....................................................................102 Figure 57: OLS Predictors of Assessment Scores Model (Pupils with Low Proficiency in Official Ghanaian LOI Subgroup) ........................................................................................................................................................................................103 Figure 58: OLS Predictors of Assessment Scores Model (Pupils with Medium Proficiency in Official Ghanaian LOI Subgroup) ........................................................................................................................................................................................104 Figure 59: OLS Predictors of Assessment Scores Model (Pupils with High Proficiency in Official Ghanaian LOI Subgroup) ........................................................................................................................................................................................105 Figure 60: OLS Predictors of Assessment Scores Model (Female Subgroup) ..................................................................106 Figure 61: OLS Predictors of Assessment Scores Model (Male Subgroup).......................................................................107 Figure 62: OLS Predictors of FOI Index and Sub-Indices......................................................................................................108 TABLE OF TABLES Table 1: Midline Sample.................................................................................................................................................................... 1 Table 2: Current versus Target Practices for Teaching Reading in Ghanaian Classrooms................................................ 9 Table 3: Program Theory of Change/Logic Model Performance Summary.........................................................................12 Table 4: Sample Size for Schools, Grades, Teachers, and Pupils........................................................................................... 14 Table 5: Updated Minimum Detectable Changes, by EGRA Subtest, Language, and Grade............................................14 Table 6: Midline Sample Realized, by Treatment and Comparison.......................................................................................17 Table 7: Reading Skills Evaluated by the EGRA.........................................................................................................................18 Table 8: Mean Scores of Top Performers on Ghanaian LOI Assessments, Midline Only................................................31 Table 9: EGR Program Impact on EGRA Scores.......................................................................................................................32 Table 10: Positive Teaching Adaptations in Response to SBI, Coaching, and Mentoring.................................................45 Table 11: Dimensions of Implementation Fidelity..................................................................................................................... 50 Table 12: Teacher Training, Coaching, and Support Input Targets.......................................................................................51 Table 13: Materials Input Targets.................................................................................................................................................52 Table 14: Curriculum Implementation Output Targets...........................................................................................................53 Table 15: Fidelity Sub-Index and Component Scoring Table..................................................................................................55 Table 16: Fidelity Index Distribution ...........................................................................................................................................57 Table 17: Direction of Effect, Commonly Predictive Covariates on Assessment Scores................................................65 Table 18: Program ToC/Logic Model Performance Summary ...............................................................................................70 Table 19: Program ToC Risk Assessments ................................................................................................................................71 Table 20: Program Theory of Change/Logic Model and Instrument Mapping – Training and Materials.......................75 Table 21: Program Theory of Change/Logic Model and Instrument Mapping – Teacher Coaching and Supervision 76 Table 22: Performance Indicators Used by CPTD to Classify Districts as Performing, Transitioning, or Low Transitioning ..................................................................................................................................................................................... 77 Table 23: District Classification Using CTPD Method.............................................................................................................78 Table 24: Selection of the 100 Districts to Receive the EGR Program ...............................................................................78 Table 25: Variables Selected for CEM Based on Ability to Predict Treatment and/or EGRA ........................................80 Table 26: Distribution of Sample, by Language Group ............................................................................................................81 Table 27: External Validity Check of Final Sample....................................................................................................................81 Table 28: Final Sample Balance Checks.......................................................................................................................................82 Table 29: Balance Between Treatment and Comparison on Proportion of Zero Scores...............................................83 Table 30: Balance Between Treatment and Comparison EGRA Scores..............................................................................83 Table 31: Balance Between Treatment and Comparison on Other Variables that Predict Assessment Performance in Similar Contexts, Baseline .........................................................................................................................................................84 Table 32: Intra-Cluster Correlations for EGRA Subtasks in Ghanaian Language and English.........................................85 Table 33: Minimum Detectable Change by EGRA Subtasks - Ghanaian Language ............................................................88 Table 34: Minimum Detectable Change by EGRA Subtasks - English Language.................................................................88 Table 35: Minimum Detectable Change for Teacher Practices .............................................................................................89 Table 36: Pre- and Post-Pruning Balance on EGRA Scores....................................................................................................90 Table 37: Pre- and Post-Pruning Balance on Language Match................................................................................................90 vii ACRONYMS ASER Annual Status of Education Report BECE Basic Education Certificate Examination CEM coarsened exact matching CL curriculum lead CLPM correct letters per minute CPTD Committee on Performing and Transitioning Districts CS circuit supervisor CWPM correct words per minute DTST district teacher support team DWAS district weighted average score EGR early grade reading EGRA Early Grade Reading Assessment EMIS Education Management Information System ES Evaluating Systems Activity FOI fidelity of implementation GES Ghana Education Service GPEG Ghana Partnership for Education Grant HFC high-frequency check HT head teacher ICC intra-cluster correlation ILC ILC Africa INSET in-service teacher training IRB Institutional Review Board IRR inter-rater reliability KG1 Kindergarten 1 KG2 Kindergarten 2 LOI language of instruction LS literacy specialist MDES minimum detectable effect size MOE Ministry of Education MT master trainer NALAP National Literacy Acceleration Program ODK open data kit OLS ordinary least squares ORF oral reading fluency P1 Primary 1 P2 Primary 2 PCA principal component analysis QA quality assurance RAMSRI-ERC Radiological and Medical Sciences Research Institute and Ethical Review Committee SBI school-based in-service teacher training SI Social Impact SoT structured observation tool ToC theory of change ToT training of trainers USAID U.S. Agency for International Development 1 EXECUTIVE SUMMARY This report presents the findings from the midline impact evaluation of the Early Grade Reading (EGR) program, implemented under the U.S. Agency for International Development (USAID) Partnership for Education Learning Activity. The evaluation findings will inform Ministry of Education (MOE) and Ghana Education Service (GES) decision makers on the effectiveness of a phonics-based, mother-tongue intervention implemented at scale. The evaluation examined the EGR program’s effectiveness by analyzing impact on EGRA scores; pupil and teacher outcomes; implementation fidelity; contextual factors, risks, and assumptions; and predictors of program success. Findings show a statistically significant impact of considerable magnitude on all subtasks of the Early Grade Reading Assessment (EGRA) in Ghanaian language of instruction and on prereading skills in English. HOW WERE THE RESULTS MEASURED? This midline impact evaluation, conducted one year into the two-year program, used an independent, quasi￾experimental design. Using data from treatment and comparison schools, teachers, and pupils, the evaluation quantified the magnitude of changes at multiple levels and their attribution to the program as distinct from other factors that may influence reading scores (Table 1). FINDINGS AND CONCLUSIONS Impact on Reading Performance The EGR program significantly improved Primary 1 (P1) reading skills in the Ghanaian language of instruction and prereading skills in English. The Ghana EGR program demonstrated one of the largest effect sizes of any early grade reading program worldwide (Figure 1). Figure 1: Comparative Effect Sizes by EGR Program Average Ghanaian reading scores rose significantly. 0 0.5 1 1.5 2 2.5 3 Oral Reading Fluency Letter Sounds Midline Sample Treatment Comparison Schools 235 235 Teachers Interviewed 470 469 Teachers Observed 235 N/A Students Interviewed 2,247 2,257 Students Assessed 2,238 2,257 Table 1: Midline Sample 2 The EGR program significantly raised EGRA scores across all subtasks in the Ghanaian language assessment and across the letter-sound identification and nonword reading subtasks in the English language assessment. The program improved pupils’ ability to read more than tenfold—from an average of 0.64 words per minute to 6.90 words per minute. Letter￾sound identification rose from 6.24 letter sounds per minute to 25.54 letter sounds per minute. Higher-achieving pupils experienced substantially larger reading gains. The top 10 percent of program pupils were reading 33.76 words per minute at midline, compared with 8.20 words in the top 10 percent of the control group. The top 25 percent of program pupils were reading 22.72 words per minute, compared with 0.85 words in the top 25 percent of the control group. The percentage of zero scores fell. In letter-sound identification, nonword reading, and oral reading fluency, zero scores fell by 30–35 percent. In reading comprehension, zero scores fell by 13 percent. Average reading levels were lower than expected by USAID and the government of Ghana. The main driver of low average scores was the prevalence of zero scorers. Although the program significantly reduced the number of pupils unable to score on the test, at midline the pool of zero scorers was still high—85 percent for reading comprehension, 57 percent for oral reading fluency, and 64 percent for nonword reading. The continuing prevalence of zero scorers, combined with the larger gains experienced by high achievers, indicates that many pupils are not benefiting fully from the program. Pupil and Teacher Outcomes Program pupils were more likely to engage with books. Program pupils were 22 percent more likely to read books, enjoy reading more, and have greater access to books. However, control pupils were more likely to read at home, and the groups had an equal likelihood of being read to at home. These findings indicate that when books are made available (as they were in the EGR program), teachers often disallow pupils from taking the books home for fear of their being lost or damaged. Productive teaching time in program classrooms increased. Program teachers spent 7 percent more time on instruction, as opposed to time off task and dealing with classroom management. This gain of just a few minutes in each class could lead to as many as five or six extra hours of instruction over the year. Teachers also began lessons earlier and applied more program-aligned teaching practices, likely because of the availability of program curriculum lesson plans, which provided detailed contents and pacing of teaching and learning activities. Program pupils were significantly less likely than control pupils to spend time in the classroom without any learning materials, indicating that program teachers had more tools to draw on while guiding pupils through their instruction. 22% increase in children reading in school 30-35% decrease in zero scores across three subtests 7% increase in teacher time on lessons Top 25% of students read 20 more words per minute than the comparable controls More than tenfold increase in average word per minute reading 3 Program teachers believed they were demonstrating nearly all the program-aligned best practices for teaching early grade reading. However, observation showed mixed performance. The team observed greater use of phonics methods for reading instruction, and the use of lesson plans rose from 80 percent to 97 percent. However, despite increases in program-aligned beliefs in the importance of learning syllables before words (52–74 percent increase) and of learning in the mother tongue before transitioning to English (51–82 percent increase), the team observed declines in the implementation of some best practices, such as engagement of learners in reading activities or games, asking prereading questions, asking pupils to read aloud, and explaining the meaning of new words. Program teachers were more likely to engage in targeted adaptive teaching methods. Program teachers were more likely than control teachers to use adaptive teaching practices targeted to individual pupils who were struggling, as opposed to generalized adaptive practices such as repeating or rushing through a lesson, as at baseline. This finding indicates that program teachers became more efficient in their use of assessment and their time spent on assessment. Program teachers were substantially more likely to incorporate coaching and mentoring feedback. Program teachers were 28 percent more likely to receive both school-based, in-service teacher training (SBI) and coaching and mentoring from head teachers (HTs), curriculum leads (CLs), district teacher support teams (DTSTs), and circuit supervisors (CSs). These teachers were also 45 percent more likely to get coaching and mentoring based on observed lessons rather than generalized training and coaching. Finally, the adaptation guidance provided based on observation was more comprehensive, specific, and actionable than generalized guidance and coaching. Implementation Fidelity More than 80 percent of program schools were fully or almost fully implementing the program according to plan. Across the nine program components, 16.7 percent of schools demonstrated full fidelity to program targets, 65.5 percent near-full fidelity, and 17.1 percent partial fidelity. Fewer than 1 percent of schools displayed low fidelity. Schools adhered most faithfully to the program in both having available and using program materials for teachers, closely following scripted lesson plans, undergoing CS monitoring at the prescribed frequency, and providing coaching and mentoring from HTs and CLs at the prescribed frequency. The lower-scoring program components included providing SBI at the prescribed frequency and having a full suite of supplemental materials available in the classroom. Time spent on reading instruction declined. Although more teaching time was spent on task, shorter lessons and more frequent teacher absenteeism reduced time spent on reading instruction for program pupils from 1.90 hours a week at baseline to 1.47 hours a week at midline. Pupils (at both baseline and midline) thus received significantly less instructional time than the target 3.4 hours a week set by the MOE and GES, potentially hindering program impact. Predictors of Program Success Program fidelity was most predictive of program success. Program fidelity outweighed characteristics such as pupil hunger, teacher education level, and the urban/rural divide. The program component that most drove impact was adherence to the prescribed lesson plans. Schools where teachers followed the prescribed lesson plans most closely with the most pupils engaged had the highest-performing pupils. In addition, greater implementation fidelity correlated with more school resources, more teachers, better educated teachers, fewer pupils dropping out, and longer hours. 4 RECOMMENDATIONS The one-year EGR program impacts suggest exceptional promise. The following recommendations for improvement could lead to even stronger outcomes: Recommendations to Support Program Improvement Review and improve SBI trainings, assessment for adaptation, pacing, and materials provision. These program components were implemented with the lowest fidelity. Provide more targeted support to nonreaders. Assessments and teaching strategies should target zero￾score pupils, who constitute the majority of P1 and P2 pupils. Address school resource levels, pupil assets, and teacher use of punishment. Adjust program training to inform teachers of the associations between learning outcomes and school and pupil home resource levels and discuss potential mitigation strategies within the program’s scope. Teacher training should also promote positive methods of classroom discipline rather than pupil punishment. Abusive and coercive forms of discipline—verbal, emotional, or physical—correlate with lower pupil reading outcomes. Improve data quality for strong implementation. Higher-quality data on school language of instruction, teacher language skills, enrollment, and school resources would support stronger program implementation. Recommendations for Future EGR Programming Review reading outcomes holistically to understand progress. Average scores can mask progress. Policymakers should review reading outcomes for low-, middle-, and high-performing pupils separately. Integrate economic and health considerations into EGR design. Given the impact a pupil’s home environment can have on reading performance, MOE policymakers should continue to work with implementers and partner government agencies to design programs that integrate economic or heath support programs with education programming. Ensure an implementation fidelity system is in place for program adoption. The EGR program succeeds most with high implementation fidelity. Policymakers should build on the EGR program’s implementation tracking mechanisms to ensure success. 5 INTRODUCTION AND BACKGROUND Ghana has in recent years made significant strides in expanding education access, with net primary enrollment rising from 61 percent to 91 percent between 1999 and 2015 and net secondary enrollment rising from 32 percent to 55 percent with the introduction of Free and Compulsory Universal Basic Education.1 However, these significant gains in education access are not necessarily mirrored by gains in education quality. The 2015 national Early Grade Reading Assessment (EGRA) and 2016 National Education Assessment (NEA) showed that the majority of Primary 2 (P2) pupils struggled with foundational reading skills and were well behind government targets. Researchers have identified several potential causes for this lag in reading instruction, including the language in which instruction is being conducted in lower primary school.2 LANGUAGE OF INSTRUCTION IN GHANA Until recently, many children in Anglophone Africa were expected to learn to read and write in English, as the colonial era encouraged dependence on English models of instruction, curriculum, and content.3 The Republic of Ghana presents a powerful case study, having vacillated between English and Ghanaian language medium of instruction in the six decades following its independence.4 In 2002, English was reinstated as the language of primary-level instruction for the first time since 1969.5 Following a change in government in 2008, momentum started to build for a return to mother-tongue reading instruction at the primary level. This momentum led to the introduction of the current language￾of-instruction (LOI) policy and the National Literacy Acceleration Program (NALAP) in 2009. These policies stipulate that one of 116 Ghanaian languages must serve as the instructional medium for grades Kindergarten 1 (KG1) through Primary 3 (with English as a subject) and establish Primary 4 (P4) as the transition point from a Ghanaian language to English. Currently, policymakers are looking critically at the implementation of the Ghanaian LOI policy and are considering English-based instruction models in their current review of pedagogical approaches. Theoretical Underpinnings of Language-of-Instruction Policies There is growing international consensus that children best acquire basic literacy skills in the language of their home environments.7 A large body of research shows that children should achieve a level of fluency in their first language before learning an additional language. This “threshold hypothesis”8 posits that this approach allows pupils to use their first-language literacy as a foundation for eventual fluency in additional 1 UNESCO Institute for Statistics. 2 The Mitchell Group. A Look at Learning in Ghana: The Final Evaluation of USAID/Ghana's Quality Improvement in Primary Schools (QUIPS) Program. 2005 3 Bamgbose, A. (2004). Language-of-instruction policy and practice in Africa. Dakar, Senegal: Regional Office for Education in Africa, UNESCO. Retrieved June, 5, 2009. 4 Owu-Ewie, C. (2006, April). The language policy of education in Ghana: A critical look at the English-only language policy of education. In Selected proceedings of the 35th annual conference on African linguistics (pp. 76-85). Somerville, MA: Cascadilla Proceedings Project. 5 Owu-Ewie, Charles. “The Language Policy of Education in Ghana: A Critical Look at the English-Only Language Policy of Education.” www.lingref.com/cpp/acal/35/paper1298.pdf 6 Asante Twi, Akuapem Twi, Fante, Nzema, Ga, Dagbani, Dangme, Ewe, Dagaare, Gonja and Kasem 7 Pflepsen, A., Benson, C., Chabbott, C., & van Ginkel, A. (2015). Planning for language use in education: Best practices and practical steps to improve learning outcomes. 8 See Cummins, Linguistic Interdependence and the Educational Development of Bilingual Children, 1979. Brisbois J. (1995). Connections between first and second language reading. Journal of Literacy Research, 27 (4), 565–584. Bernhardt E.B., & Kamil M.L. (1995). Interpreting relationships between L1 and L2 reading: Consolidating the linguistic threshold and the linguistic interdependence hypotheses. Applied Linguistics, 16, 15–34. ELIZABETH B BERNHARDT, MICHAEL L KAMIL; Interpreting Relationships between L1 and L2 Reading: Consolidating the Linguistic Threshold and the Linguistic Interdependence Hypotheses, Applied Linguistics, Volume 16, Issue 1, 1 March 1995, Pages 15–34 6 languages. Transitioning from their first language to a second language too early forces pupils to learn to read while simultaneously learning a new language. The same body of research indicates that children who begin their education in their first language and then transition to a second language show higher achievement in reading than children who learn in the second language from the beginning. This finding is especially important in countries such as Ghana, where learning a second language is an essential part of a primary education that actively prepares its pupils to be productive in formal economies where a second language is required.9 Practical Challenges to Implementing Mother-Tongue Language-of-Instruction Policies at Scale Although the theoretical underpinnings of mother-tongue LOI policies are strong, they are sometimes complex, expensive, and difficult to implement, especially at scale, and require system-level cooperation, coordination, and management. Mother-tongue LOI requires the effective development and distribution of a more diverse set of pupil and teacher materials, as well as a more robust matching of teacher and pupil language skills. These challenges are complicated by the fact that although more than 73 indigenous languages are spoken in Ghana,10 only 11 serve as official LOIs. In addition, any new instructional approach presents implementation challenges. For example, teachers and administrators require institutionalized monitoring and guidance support to ensure they understand the new techniques, are using the materials as intended, and are implementing the approach according to plan. Meeting the Demand for Evidence More evidence is needed to support decision making on the best approach to improve early grade reading performance in Ghana, with two main gaps. First, policymakers need to understand whether the international research supporting the use of mother-tongue LOI holds true in the complex Ghanaian linguistic environment and, if so, the extent of the potential gains in both the mother-tongue language and English. Second, evidence is needed on the effectiveness of these programs at scale. In Ghana, empirical research on education programs is often conducted on small-scale and pilot programs. Although small￾scale approaches allow for more troubleshooting and hands-on management in pilot areas, national policy decisions need to consider how programs would be implemented throughout the country. To generate rigorous evidence on the effective implementation of phonics-based, mother-tongue instruction in Ghanaian classrooms at scale, the U.S. Agency for International Development (USAID)/Ghana Partnership for Education designed the Early Grade Reading (EGR) program and the Evaluating Systems (ES) evaluation in tandem. This independent, quasi-experimental impact evaluation can inform Ministry of Education (MOE) and Ghana Education Service (GES) policy decisions on EGR. This evaluation can also inform the practical implementation of mother-tongue instruction in linguistically complex environments across the African continent and around the world. 9 Pflepsen, A., Benson, C., Chabbott, C., & van Ginkel, A. (2015). Planning for language use in education: Best practices and practical steps to improve learning outcomes. 10 "Ghana," in: Lewis, M. Paul, Gary F. Simons, and Charles D. Fennig (eds.). 2014. Ethnologue: Languages of the World, 17th ed.Murica Texas: SIL International. 7 PROJECT BACKGROUND The EGR program, a core component of the USAID/Ghana Learning Activity, which aims to improve, expand, and sustain learning outcomes for 900,000 primary pupils in Ghana, works to achieve measurable improvements in reading outcomes for KG2, P1, and P2 pupils through phonics-based instruction in the 11 official Ghanaian instruction languages. The program provides in-service teacher training (INSET) and develops and distributes teacher guides (including scripted lesson plans), pupil activity books, and supplementary classroom materials. In addition, the program builds the capacity of school-, circuit-, and district-level actors to provide ongoing teacher monitoring, coaching, and mentoring. The program does not include out-of-classroom activities. The evaluation will cover the full intervention period over the 2017–18 and 2018–19 school years. SUMMARY OF THE INTERVENTION The Ghana EGR program emphasizes best practices in teaching reading and using explicit instruction in phonemic awareness, phonics, vocabulary, comprehension, fluency, and writing.11 Following this approach, the program developed teacher guides with scripted lesson plans and pupil reading books in the 11 official Ghanaian LOIs. Head teachers (HTs) and curriculum leads (CLs) are provided with INSET on using these resources by EGR program-trained master trainers (MTs) consisting of a district teacher support team (DTST) and literacy specialists (LSs). In addition, teachers receive training and resources on continuous learner assessment and DTSTs, national core trainers, and CSs receive supplementary e-learning coursework to maintain the quality and content of the core training. School-based HTs and CLs are provided with additional training on coaching and instructional leadership so that they can engage in biweekly teacher observations and coaching sessions and facilitate weekly teacher INSET meetings. CSs are trained to monitor teacher implementation fidelity using an electronic data collection platform and arrange visits from the DTST and LSs for struggling teachers. The EGR program materials are modeled on the stages of literacy acquisition and are therefore customized to each grade level. KG2 materials focus on teacher-scripted lesson plans only and emphasize reading readiness, including letter recognition, and oral vocabulary and comprehension. P1 materials include both scripted lesson plans and pupil books, which feature decodable text and emphasize the development of initial reading skills, including decoding, comprehension, and early writing skills. P2 scripted lesson plans and pupil books emphasize leveled texts and aim to produce readers who can read basic text with fluency and comprehension. PROJECT OBJECTIVES The results framework shows the progression of intermediate outcomes that lead to measurable improvements in pupil reading achievement (Figure 2). To achieve these results, the EGR program aims to develop teacher behavior and practice in the five “T”s—time, text, testing, teaching, and tongue—and in leadership (Table 2). 11 National Reading Panel (US), National Institute of Child Health, & Human Development (US). (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. National Institute of Child Health and Human Development, National Institutes of Health. 8 Figure 2: Results Framework for the Ghana EGR program12 Develop teaching and learning materials Develop and implement trainings of teachers, HTs, and CLs Develop and implement training of MTs (DTST) Availability of teaching and learning materials increased Cadre of early grade teachers trained in new reading program and of HTs/CLs trained in coaching Cadre of MTs (DTST) trained in reading, coaching, and supervision Increased use of teaching and learning materials Capacity of teachers in teaching reading and of HTs/CLs in coaching increased Support for coaching by DTST improved Enhanced reading instruction Pupil reading performance improved 12 FHI360 (2017). EGR Program Plan: January 30, 2017. 9 Table 2: Current versus Target Practices for Teaching Reading in Ghanaian Classrooms Current Practice EGR Program Objective Time 90-minute reading lessons, but little time on task 90-minute reading lessons used productively by teachers and pupils Text Some materials available in the 11 Ghanaian languages of instruction Development of full pupil books with teacher guides (scripted lessons) for each of the 11 official Ghanaian languages of instruction Testing Use of the EGRA by NGOs and donors Teacher and school use of the Annual Status of Education Report (ASER); 13 periodic teacher evaluations with guidance for review and remediation Teaching Whole-language instructional approach with little training and ongoing support Phonics-based approach with in-service teacher professional development Tongue Insufficient materials for full reading program in the 11 Ghanaian languages; teachers unfamiliar with teaching in the 11 Ghanaian languages Full reading program in each of the 11 official Ghanaian languages of instruction, with support to teachers in use of Ghanaian language and bilingual techniques in practice Leadership HTs have oversight and are expected to manage resources and attendance HTs and CLs trained to be leaders of learning, facilitating weekly school-based INSET meetings and observing and coaching teachers in using new teaching materials and practices TARGET BENEFICIARIES The EGR program benefits 900,000 KG2, P1, and P2 pupils by training and capacitating approximately 20,000 teachers, more than 14,000 HTs and CLs, 1,000 LSs, and 800 CSs. The program targets more than 7,000 schools across the 10 regions of Ghana. Target districts were narrowed down to 165 based on their performance on several district-level education indicators, with higher-performing districts being prioritized. Given the program’s emphasis on mother-tongue instruction, districts were further narrowed to 100 based on an assessment of the degree of match between pupil language and the official Ghanaian LOIs. The methods for district selection are elaborated in the Sampling section, as well as Annex I of this report. 13 The ASER is a simple assessment that is useful for broad identification and tracking of reading levels. Originally developed in India as a citizen-led assessment, it is simple enough for a rapidly trained adult or youth to learn to administer. 10 EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE This midline evaluation measured the EGR program’s impact on pupil reading outcomes at each grade level after one year. The evaluation also rigorously examined the EGR program’s theory of change (ToC) to identify barriers and catalysts to effective implementation at scale. The evaluation findings will enable implementers and stakeholders to adjust the program in response to observed successes and obstacles and allow policymakers to assess the program’s intermediate progress toward its goals. In addition, the evaluation supports learning among the broader community of education policymakers and practitioners by: • Generating evidence on the effectiveness of a structured pedagogy program. • Producing novel research on the effectiveness of mother-tongue instruction in early education environments where multiple local languages are spoken. • Contributing to the growing body of global evidence on phonics-based approaches to literacy acquisition and reading fluency. EVALUATION QUESTIONS This evaluation is guided by the following evaluation questions aligned with the program’s ToC: Impact/Goals Question 1. To what extent did the EGR program improve reading skills in the Ghanaian language of instruction and prereading skills in English? Outcome/Intermediate Goal Questions 2. To what extent did the EGR program improve amount and use of teacher and pupil classroom time on task? 3. To what extent did the EGR program increase the likelihood that teachers apply program-aligned best practices in teaching early grade reading? 4. To what extent did the EGR program increase the likelihood that teachers engage in adaptive teaching? 5. To what extent did the EGR program increase the likelihood that teachers incorporate coach and mentor feedback? 6. To what extent did the EGR program increase the likelihood that pupils engage with books both inside and outside the classroom? Output/Input Questions 7. To what extent was the EGR program implemented according to plan? Context and Sustainability Questions The evaluation team addressed the following questions that may inform program scale-up: 8. How have contextual factors affected program success? 9. What program components and pupil- and school-level characteristics were most strongly associated with program success? 11 EVALUATION THEORY OF CHANGE AND MEASUREMENT APPROACH To guide the evaluation design, the evaluation team operationalized the EGR program logic into a detailed ToC (Table 3 and Annex 1).14 This ToC aims to reveal the causal links and embedded assumptions that connect inputs/activities to outputs, outputs to outcomes, and outcomes to goal impacts. Although the evaluation aimed primarily to measure the EGR program’s impact on pupil reading skills (goal impact), it also examined the inputs/activities leading to impact and tested the causal links and assumptions underlying observed changes. The impact evaluation thus contains an embedded process/implementation fidelity evaluation that explores mediating factors to help determine reasons for any limited or null effects. Mapping each component of the ToC to specific data source(s) ensured that the data collection approach aligned with the program logic and provided a starting point for structuring, selecting, and designing the data collection tools. Each of nine outputs, implemented together, are meant to lead to five outcomes (or changes in pupil and teacher behavior). These outcomes, in turn, lead to two impacts (changes in learning outcomes in Ghanaian LOI and English). The risks are factors outside the program’s scope that may affect outputs, outcomes, and impact. 14 Given the complexity of the EGR program, the team organized the program logic model in Annex 1 into two broad categories: 1) teacher training and materials and 2) coaching and mentoring. Although these components are part of an integrated package, breaking down the logic model in this way allowed a clear display of the causal links and assumptions for each component, which was useful for designing the evaluation. Although the inputs/activities and outputs differ between these two categories, they converge on outcomes and impacts and there are thus notable overlaps between the two tables at these stages. Beginning with the assumptions that link inputs/activities to outputs and all the way through goal impact, the team also used color-coded bullets to identify the data source(s) that will be used to triangulate that component. 12 Table 3: Program Theory of Change/Logic Model Performance Summary Outputs Outcomes Goal Impact Pace: Progress on scripted lesson plans in relation to program￾prescribed pace Lesson Performance: Teacher fidelity to steps in the lesson plan and pupil engagement with the lesson plan Teacher Textbook: Use of teacher textbook by teacher Pupil Workbook: Use of pupil workbook by teacher and pupils Supplemental Materials: Presence of materials such as teacher guide, alphabet chart, flash cards, Big Books, and conversation posters Reading Instruction and Training: Compliance with expectation for eight or more days of in-service training on reading instruction Observation and Coaching: Compliance with expectation for biweekly observation and coaching by HT and/or CL SBI: Compliance with expectation for one hour-long SBI session a week CS Monitoring: Compliance with expectation for monthly visits from CS to monitor implementation Pupils are more likely to engage with books and learning materials inside and outside classroom Teachers demonstrate mastery of phonics-based approach to teaching reading in Ghanaian LOI Classroom time on task improves for pupils and teachers Teachers adapt instructional approach to pupil needs through continuous assessment Teachers adapt instructional approach based on SBI trainings, coaching, and monitoring Improved reading outcomes in Ghanaian language for P1 Improved prereading outcomes in English for P1 Risks Pupils miss class or are tardy to class Teachers miss class Pupils do not understand the LOI Teachers cannot teach in the LOI Pupils have a low level of assets Schools have a low level of resources*(added) Teachers use abusive or degrading forms of discipline and punishment for poor performance*(added) 13 EVALUATION METHODS AND LIMITATIONS EVALUATION METHODOLOGY This midline evaluation, conducted in May and June 2018, combined an impact evaluation, a performance evaluation, and an implementation fidelity evaluation to track progress through the EGR program’s ToC. The midline evaluation built on work from the 2017 baseline and will inform the 2019 endline. Impact Evaluation The impact evaluation measured the change that can be attributed to the EGR program by establishing a credible counterfactual to estimate what would have happened to beneficiaries in the absence of this intervention. By comparing participants (i.e., the treatment group) with a comparison group that represents the counterfactual, the evaluation “subtracted away” the contextual changes that affect both program participants and the comparison group. If program participation is the only substantive difference between participants and the comparison group, the difference in outcomes can be attributed to the program. Without a rigorous estimate of a counterfactual, there is a risk of over- or unde-estimating program impact, which could misguide policymakers. The gold standard in impact evaluation involves random assignment to treatment or comparison conditions so that each group, on average, is statistically similar at baseline; thus, any observed differences between the two groups at endline can be attributed to the intervention. However, because the selection of treatment sites for the EGR program was done purposively (rather than randomly) in collaboration with the GES, this evaluation instead used a quasi-experimental approach, with a counterfactual group constructed in 2017 using advanced statistical matching techniques. Specifically, the evaluation used coarsened exact matching (CEM) to ensure that comparison schools were similar to treatment schools at baseline on a wide range of observable characteristics. These observables were selected based on (a) their ability to predict treatment status or selection into the treatment group and (b) the extent to which they were significant predictors of the primary outcomes of interest (EGRA scores). The specific approach to constructing a credible counterfactual is discussed briefly in the Sampling section of this report and at length in Annex I. A full discussion of the sampling approach and the balance statistics demonstrating the comparability of the treatment and comparison group is provided in the baseline report. In the analysis of the results, the evaluation controlled for time-invariant differences at endline using difference-in-differences analysis,15 which subtracted any remaining baseline differences between treatment and comparison units. Performance Evaluation (Outcomes) The performance evaluation identified changes in behaviors and attitudes and differences between the groups that received the program and those that did not. Implementation Fidelity Evaluation (Inputs and Outputs) For the implementation fidelity evaluation, data from 232 classroom observations, 230 CL surveys, 183 CS surveys, and 235 HT and teacher interviews in 241 treatment schools were used to measure how closely the program achieved its intended inputs/activities, outputs, and outcomes. An implementation fidelity index was constructed and calculated for each observed school, based on EGR program implementation targets; this index will be described in detail in the implementation fidelity portion of the Findings section. The evaluation team collected data to test the assumptions underlying the program 15 Difference-in-differences attempts to calculate the treatment effect in non-experimental evaluations by comparing the average change over time in the outcome variable for the treatment group to the average change over time for the control group, thus attempting to mitigate the effects of extraneous factors and selection bias. 14 theory/logic model. In addition to allowing us to explore reasons underlying any positive, limited, or null effects, these process data helped the team disaggregate results by degree of fidelity and understand the extent to which the program is on course to achieving its goals. SAMPLING Sampling for the midline evaluation was conducted using the same sampling process and protocol established in the baseline. However, the class sample was restricted to P1. This section outlines the key components of the sampling process, including the decisions about sample size, the sample frame and selection of individual schools, and the sampling of individual children at each school. (For a full discussion of the sampling approach, please refer to the baseline report.) Sample Size and Power Analysis To determine the sample size—both the number of schools and the number of pupils within schools— the evaluation team conducted detailed statistical power analysis during the evaluation design phase, the results of which are presented in Annex I. The total required sample size for schools, grades, teachers, and pupils is provided in Table 4. Table 4: Sample Size for Schools, Grades, Teachers, and Pupils Total schools sampled: 470 schools (235 treatment and 235 comparison) Grades sampled per school: 1 grade (P1) Classes sampled per grade: 1 class per grade Pupils sampled per grade: 10 pupils (5 boys and 5 girls to allow for sex disaggregation) Teachers sampled per grade: 1 teacher (reading) Total pupil sample P1: 4,700 pupils Total teacher sample: 940 teachers To maximize balance between treatment and comparison, the team sampled an additional 50 schools at baseline (approximately 10 percent of the required sample), evenly distributed across treatment and comparison groups. The required sample of 470 schools, to be used throughout the rest of the evaluation, was then selected from these 520 schools through a purposive pruning procedure developed to improve matching between treatment and comparison schools. The pruning strategy, as well as pre- and post￾pruning balance statistics, is detailed in Annex I. Using actual baseline statistics (i.e., EGRA subtest scores, intracluster correlations [ICCs], and R-squared values), the team then recalculated the minimum detectable effect sizes (MDESs) that the evaluation is powered to detect with the above sample. Adopting standard assumptions of 0.8 for statistical power (β) and a partial Bonferroni-adjusted value of 0.0216 for significance level (α), the team presented updated MDESs (Cohen’s d) for each of the EGRA subtests (Table 5). Table 5: Updated Minimum Detectable Changes, by EGRA Subtest, Language, and Grade EGRA Subtest ICC R2 MDES 16 Bonferroni correction is applied by dividing the standard α value of 0.05 by the number of hypotheses being tested (we use the number of local language EGRA subtasks) while factoring in the mean correlation between these subtasks (0.47). Bonferroni adjustment is designed to correct for the increased likelihood of false positives when multiple hypotheses are being tested. For more information, see http://www.quantitativeskills.com/sisa/calculations/bonhlp.htm. 15 Ghanaian Language Listening Comprehension 0.24 0.14 0.16 Letter Sounds 0.39 0.18 0.18 Non-Word Reading 0.29 0.04 0.17 Oral Reading Fluency (ORF) 0.54 0.03 0.22 Reading Comprehension 0.08 0.04 0.12 English Listening Comprehension 0.27 0.14 0.16 Letter Sounds 0.36 0.15 0.18 Non-Word Reading 0.16 0.08 0.14 ORF 0.35 0.05 0.18 Reading Comprehension 0.09 0.06 0.12 The overall mean MDES for all subtests, grades, and languages is 0.17, which is lower than the MDES reported in the evaluation design report. This lower figure is due principally to the fact that the ICCs in the actual sample are generally lower than those in the 2015 National EGRA, which was used to inform the original sample size requirements. This improvement in the study’s power enables identification of even smaller impacts than expected at the design stage. Selection of School Sample The sampling frame for this evaluation, established during the 2017 baseline, consisted of all basic schools in 165 of the 216 districts of Ghana. These 165 districts form the original target districts of the EGR program, which were later reduced to 100. Schools in the 100 districts receiving the EGR program formed the sampling frame of treatment schools, whereas the other 65 districts formed the sampling frame of comparison schools. The sampling frame of comparison schools was limited to these 65 districts to minimize selection bias, as the first stage of district selection used complex criteria to distinguish the 165 districts from the remaining 51.17 Schools were sampled from both treatment and comparison districts using CEM, a robust and relatively new statistical matching technique through which balance between the treatment and comparison groups is established ex ante using variable cut points or bins, rather than discovered through repeated propensity score re-estimation (which can inadvertently create imbalance and model dependence).18 CEM allows for matching of treatment and comparison units on many different variables (rather than through a single composite propensity score), which is especially useful in cases where a large number of important predictors are available. Variables used in the CEM model include those that predict treatment status and/or the primary outcome of interest—performance on EGRA. Using data from Ghana’s Education Management Information System (EMIS), Basic Education Certificate Examination (BECE), and the 2015 National EGRA, the team ran regression models using purposively selected variables to identify those that predict treatment and EGRA scores. In all, 11 variables were robust and significant predictors of treatment status and/or EGRA performance and were therefore used in the CEM algorithm. These included urban/rural status, multilingual school status, vehicle access, school in top 20 percent of school-reported income, ratio of writing places to pupils, ratio of female primary teachers, pupil absenteeism rate, proportion of female pupils, primary pupil-teacher ratio, and circuit-level BECE scores from four core subjects. After treatment and comparison schools were grouped into strata using CEM,19 the evaluation team randomly selected 260 treatment schools stratified by language so that the number of treatment schools 17 The first stage of the district selection procedure is described at length in Annex I. 18 King, G., & Nielsen, R. (2016). Why propensity scores should not be used for matching. Copy at http://j. mp/1sexgVw Download Citation BibTex Tagged XML Download Paper, 378. 19 It is important to note that CEM limits the sampling frame to those treatment and control units that have been matched according to the specified criteria. Schools that cannot be matched are therefore excluded from the sampling frame. 16 selected in each language group was proportionately represented in the final sample. The team then randomly dropped excess comparison schools in each of the CEM strata so that 260 comparison schools were matched with 260 treatment schools. The team concluded by verifying sample balance, as well as comparing the characteristics of the sample to those of the broader sampling frame as an external validity check. The school selection process is summarized in Figure 3 and described in detail in Annex I. The balance statistics, showing the statistical comparability of the two groups, are presented in Annex I. Figure 3: Summary of School Sample Selection Procedure Using CEM Selection of Teachers and Pupils within Schools A two-stage sampling process was used to sample teachers/classroom and pupils for the midline survey. The first stage comprised revisiting the same P1 teachers and classrooms sampled at baseline (panel design). In the majority of the sampled schools, the evaluation team sampled the same P1 teacher who was interviewed at baseline—however, there were important exceptions: If the teacher sampled at baseline was no longer teaching in the school or class, supervisors sampled a new P1 teacher. Sampling of a new teacher was done in the same manner as at baseline. In consultation with the HT, data collection supervisors recorded all P1 classes and streams in the school. If there was only one P1 class, it was automatically selected; where the school had more than one P1 stream, one P1 class was randomly selected from the list of P1 classes in the school. For a class to be eligible for selection, the teacher of the class had to be present and teaching a reading lesson that day, and a minimum of five girls and five boys had to be present. When all criteria could not be met, the team sampled the class that met the greatest number of criteria. At the majority of schools, there was only one class for P1. In such cases, the classes were automatically selected. In the second stage, pupils were resampled at each sampled school (cross-sectional design). Upon the P1 teacher’s selection, supervisors used that day’s pupil class attendance register to randomly sample five boys and five girls. An interval was first calculated by dividing the total number of boys or girls by five to achieve the desired sample. Then, using a die to select a random starting point, the supervisor counted off the interval and selected every nth pupil until the five-pupil sample was realized. The procedure was then repeated to select three alternate pupils. If there was no attendance register, the procedure was Step 1 Prepare Sampling Frame Secure 2015 Basic Schools Census Data from EMIS Restrict sampling frame of schools to 165 original Learning districts (100 treatment, 65 comparison) Impute missing EMIS data using multiple imputations and nearest neighbor predictive mean matching Step 2Coarsened Exact Matching Using data from EMIS and BECE, run regression models to predict EGRA performance (2015) as well as selection into treatment Using language-of￾instruction and significant predictor variables, group treatment and control schools into matched strata using Coarsened Exact Matching (CEM) Step 3Stratified Random Sampling Randomly drop any excess treatment or comparison schools within each CEM strata Stratifying on language-of￾instruction (proportional to population), randomly sample 260 treatment and 260 matched comparison schools Step 4External Validity and Balance Checks Conduct balance checks on matched sample Check balance on key variables between the random sample and the broader sampling frame to ensure external validity is maximized Starting sample consists of 260 treatment schools matched to 260 comparison schools distributed across 143 districts of Ghana After baseline, purposively prune 50 schools (25 treatment, 25 control) in order to maximize baseline balance 17 conducted by having the boys and girls form a line and count off using the same interval method. The names and sampling order of the selected pupils were recorded in tracking sheets, and the children were interviewed and assessed in the order in which they were sampled. If a sampled child could not be found or did not assent to participate, the first alternate pupil was used in his or her place. Sample Realized The sample realized at midline is summarized in Table 6, followed by a discussion of reasons for discrepancies between planned and actual figures. Table 6: Midline Sample Realized, by Treatment and Comparison Treatment Comparison Total (Actual) Planned Difference Schools enumerated: 235 235 470 470 0 HT interviews: 235 235 470 470 0 CL 230 - 230 235 -5 P1 teacher interviews: 235 234 469 470 -1 P1 teacher observations: 235 - 234 235 -1 P1 boy interviews: 1,128 1,128 2,256 2,350 -94 P1 boy assessments: 1,123 1,128 2,251 2,350 -99 P1 girl interviews: 1,119 1,129 2,248 2,350 -102 P1 girl assessments: 1,115 1,129 2,244 2,350 -106 FOI observations 235 - 232 235 -3 CS interviews: 183 - 183 183 0 DATA COLLECTION METHODS Primary data collection instruments were selected based on their ability to test and triangulate the causal links and assumptions identified in the program ToC. These instruments are presented in their entirety in Annex V. Structured survey tools were developed by a team of education experts and drew upon EGR program documentation, supplementary EGRA Ghana survey tools, and an extensive bank of survey tools used for evaluating phonics-based EGR programs. The Structured Observation Tool (SoT)—which measures the extent to which teachers are employing best practices in teaching reading— built upon EGR program observation tools, as well as those used for the Basa Pilipinas and Malawi EGR Activity impact evaluations supported by USAID/Philippines and USAID/Malawi. In addition, the team employed three instruments widely used in the evaluation of school-based interventions in the developing world to facilitate analysis of pupil reading performance, classroom time on task, and implementation fidelity: the EGRA, the Stallings Observation System, and the fidelity of implementation (FOI) observation tool. EGR Assessment (EGRA) The EGRA was developed by USAID in response to a growing demand for an open-source, easy-to-use tool that assesses learner abilities along a reading continuum, from prereading skills—such as listening comprehension and letter sound identification—to reading fluency and comprehension. EGRA has been Primary Data Collection Instruments • Early Grade Reading Assessment Tool • Student Interview • Teacher Interview • Curriculum Lead Interview • Head Teacher Interview • Circuit Supervisor Interview • Structured Classroom Observation • Fidelity of implementation observation • Stallings Snapshot 18 adapted and used in more than 70 countries including Ghana, which conducted national EGRA assessments of P2 pupils in 2013 and 2015. In Ghana, the EGRA tool was piloted, adapted, and validated in the 11 official LOIs plus English, and an item bank of five equated subtests was created for future annual reading assessments. The specific stages of reading and skills evaluated by these subtests are described in Table 7. Table 7: Reading Skills Evaluated by the EGRA Stage Subtask Skill Description Prereading Listening Comprehension Oral language comprehension and vocabulary This subtask evaluates oral language comprehension and vocabulary by asking pupils questions relating to a short text read aloud to them. The resulting measure is a percentage of correct responses. Initial Reading Letter-Sound Identification Alphabetic principle— letter-sound correspondence This subtask evaluates a pupil’s ability to read the sounds of letters. This subtask is timed and thus gives the number of letter sounds produced correctly per minute (clpm). Initial Reading Nonword Reading Alphabetic principle— letter-sound correspondence and fluency—automatic decoding This subtask evaluates a pupil’s ability to decode unfamiliar words. The subtask is timed, so the resulting measure is the number of nonwords decoded correctly per minute (cwpm). Fluency and Comprehension Oral Reading Fluency (ORF) Fluency—automatic word reading in context This subtask evaluates how well a child reads aloud a short passage of connected text. It is timed, and therefore produces a measure that is the number of words of text read (cwpm). Reading Comprehension Comprehension In this subtask, pupils are asked questions relating to the text that they read aloud for the ORF portion of the assessment. The resulting measure is the percentage of correct responses. The evaluation team secured these item banks, procured by USAID/Ghana, in both paper and Tangerine (electronic) format. Pupils were tested on each of the five subtasks in the school’s official LOI (as reported by the HT on the day of data collection), as well as English.20 To minimize risks related to leakage and/or learning of the test, the team used two equated versions of the test and randomly selected which version was administered. Inter-rater reliability (IRR)—i.e., ensuring that every enumerator is conducting the survey in the same way—for EGRA was tested before launch and continuously monitored throughout data collection. When a given enumerator fell below the 90 percent reliability threshold established by the EGRA tool 20 Although the program logic anticipates improvements in prereading skills in English only, the evaluation team (in consultation with USAID/Ghana) opted to test on all five subtasks in English since children in Ghana have historically performed better in English on higher-order reading skills. For the sake of transparency, it is therefore important to understand the extent to which gains in Ghanaian language performance come at the expense of English performance. 19 developers,21 supervisors were notified, and retraining was immediately carried out by the supervisor in the field. At the completion of the midline, the overall rate of agreement between supervisors and enumerators was 99.4 percent for English subtests and 99.5 percent for Ghanaian language subtests. Stallings Classroom Observation System The Stallings “Classroom Snapshot” Observation System is a subject-neutral observation tool that provides reliable quantitative measures of teacher use of instructional time, use of materials, core pedagogical practices, and pupil engagement.22 The system’s high reliability and minimal training requirements make it a popular tool for measuring teacher effectiveness in developing countries. The process involves dividing a lesson into 10 equal segments and then taking 15-second “snapshots” of what is happening in the classroom between each segment, including teacher activities, pupil activities, use of materials and learning aids, and pupil distraction and discipline. Although an explicit objective of the EGR program is to improve classroom time on task (which Stallings measures), the team carefully considered how to measure differences in teacher practices between treatment and comparison schools in light of the program’s emphasis on lesson scope and sequence (as lesson content can vary considerably depending on the observation date). Stallings’ emphasis on the use of materials and aids and its capture of classroom distribution time across types of learning activities allow the evaluation to consider differences at endline with respect to the types of activities that the EGR program emphasizes.23 The distribution of learning activities captured by Stallings thus complements the structured observation data on the extent to which teachers are employing best practices for teaching reading. As with the EGRA, IRR of Stallings was tested during training and continuously monitored during data collection. At the completion of midline, the overall rate of agreement between supervisors and enumerators was 97.1 percent. The interview and structured observation protocols were developed in close consultation with the EGR program to ensure they aligned with the updated program design and built upon the monitoring and evaluation (M&E) resources developed both as part of the program and as part of the program’s own FOI activities. Before launch, all data collection tools were reviewed and pretested in schools outside the evaluation sample. In addition, the tools and broader study protocol (including consent forms and procedures) were reviewed and approved by two Institutional Review Boards (IRBs)—the Radiological and Medical Sciences Research Institute and Ethical Review Committee (RAMSRI-ERC) of Ghana, as well as the internal IRB of Social Impact, Inc. (SI) in the United States. The complete set of data collection tools, as well as IRB￾approved consent forms, can be found in Annex V. Implementation Fidelity Observation The FOI observation tool was adapted from the EGR program monitoring tools and administered in treatment schools to assess the extent to which program components were unfolding in line with the program model and milestones. Additionally, the FOI tool was used to test the embedded assumptions of the EGR program’s ToC. Specifically, it assessed the extent to which teachers in the treatment schools adhered to the scripted lessons plans outlined in the program teacher guide. The tool collected data on delivery of reading lessons with attention to lesson order, sequence, pacing, and the use of teaching and 21 Early Grade Reading Assessment Toolkit (2009). RTI International. Avalable at https://globalreadingnetwork.net/resources/early￾grade-reading-assessment-egra-toolkit-second-edition. 22 World Bank Group (2015). User Guide - Conducting Classroom Observations: Analyzing Classroom Dynamics and Instructional Time Using the Stallings “Classroom Snapshot” Observational System. 23 For example, treatment schools might be expected to have a larger proportion of class time focused on activity book work, discussions, and drills relative to more traditional rote activities such as copying). 20 learning materials in the classroom. Enumerators sat quietly at the back of each P1 reading class with a copy of the teacher guide and recorded their observations throughout the lesson. Interviews Five interview instruments—pupil, teacher, CL, HT, and CS interviews—were developed and used to collect data at midline. The instruments were developed in close collaboration with the EGR program and USAID/Ghana to ensure they would solicit enough information to answer the evaluation’s outcome questions. The instruments gathered information on school administrative data such as pupil and teacher attendance, LOI, availability of teaching and learning materials, use of continuous assessment, mentorship and coaching, and teacher beliefs and practices. The interview items were carefully selected based on their ability to triangulate and test the key assumptions outlined in the program ToC. Pupil Interview The pupil interview, administered in both treatment and comparison schools, was intended to elicit information on a range of issues that impact learning outcomes, including pupil language skills (first language, languages spoken at home and on the school playground, and languages used in the classroom), methods and frequency of pupil assessment, availability of reading materials and books in school and at home, the extent to which pupils read at home, and pupil socioeconomic background. Teacher Interview The structured teacher interview was designed to elicit information on classroom teacher practices, availability of teaching and learning materials, and teacher knowledge and attitudes about the efficacy of phonics-based instruction relative to other instructional methods for reading, such as the whole-language method. In addition, this instrument gathered data on both teacher and pupil language skills to generate information on the degree of fidelity to the GES-approved school LOI. The teacher interview solicited information on teachers’ beliefs about current instruction models for reading and how continuous assessment is used for tracking pupil achievements and for adaptive teaching. It sought information on the content, frequency, and consistency of HT coaching and how coaching is perceived by teachers. At treatment schools, the teacher interview assessed a range of program-related issues such as the challenges teachers and pupils face using curriculum materials provided by the EGR program, the efficiency and utility of HT and CL coaching and SBI meetings, and how monitoring and coaching feedback was used to improve instructional practice in the classroom. The questions were intended to gauge program implementation fidelity. Curriculum Lead Interview The CL is a role specific to the program as a leader and supporter of its pedagogical approach. The interview was designed to elicit information from treatment schools on how program-related curriculum materials are used by teachers for teaching reading, challenges teachers and pupils face using the new teaching and learning materials, and the frequency and content of coaching and SBI meetings. In addition, the CL interview solicited information on how teachers perceive the coaching sessions and whether it helped to improve learning outcomes. The instrument explored how monitoring data is used by teachers and the school to improve instructional practice. Head Teacher Interview The HT interview was designed to elicit a wide range of school-related information. For instance, it gathered data on enrollment and staffing for lower primary (KG1–P3) and pupil and teacher attendance data (P1 only) in both comparison and treatment schools. In addition, the HT interview gathered data on 21 the language background of the HT, teacher, and pupils and the official school LOI. Other data obtained from the HT interview included school resources, reading resources, in-service training and professional development, the HT’s beliefs on the best method of reading instruction, and pupil performance and evaluation. At treatment schools, the instrument solicited information on implementation fidelity, including the HT’s perception of how teachers and pupils are using program-related materials and what challenges teachers encounter in use of the materials. Circuit Supervisor Interview The CS interview sought information on the extent to which CSs feel equipped with the necessary skills to engage in monitoring duties as specified by the EGR program. It sought information on how teachers perceive the CS monitoring and whether such monitoring has impacts on learning outcomes. In addition to information on school visits, the interviews sought information on the CSs’ personal beliefs about the best approach to teaching reading effectively. In addition, this instrument examined the extent to which teachers engage in continuous assessment and how assessment data are used to inform adaptive teaching. Additionally, this instrument sought information on why some pupils struggle to become fluent readers. At treatment schools, the CS interview gathered information on how teachers are using the program’s teaching and learning materials and the challenges they face in doing so. DATA COLLECTION Supervisor and Enumerator Training ES contracted ILC Africa (ILC) as the local data collection partner for the evaluation. ILC has extensive experience in school-based data collection in Ghana, having previously carried out data collection for education projects supported by USAID and the United Kingdom’s Department for International Development, among others. Given the diverse language skills required of enumerators, as well as the sheer scale of the exercise, ES and ILC conducted data collection training in two phases. The first phase, held from April 25 to May 4 2018, was a Training of Trainers (ToT) whereby the evaluation team— including Dr. Andrew Epstein, Erika Keaveney, and Mohammed Dawuda—extensively trained field supervisors on data collection procedures, including logistics and sampling, administration of the tools, and data quality assurance (QA). Immediately following the supervisor training, a 10-day enumerator training was held in Accra from May 7 to May 18, 2018, which consisted primarily of breakout sessions (per language group and instrument) led by the supervisors and the ILC training team. Following the classroom training, the enumerators conducted a second pilot test in schools outside the sampling area. Before the launch of data collection, enumerators were tested on IRR for key data collection tools. The IRR test ensures that all enumerators are administering tests in the same way. For EGRA and the Stallings snapshot, both IRRs were 99 percent in agreement among raters, and for structured classroom observations the IRR was 98 percent in agreement among raters.24 Data were collected over four weeks from May 23 to June 26, 2018. Data Quality Assurance To ensure the high quality of data that is the cornerstone of this evaluation, the evaluation team closely managed the data collection process, including logistics, supervisor training, survey programming, data security and backup, and data quality assurance (QA). All survey forms were programmed using Open Data Kit (ODK), which allows for electronic data capture and regular submission and review. ES worked closely with ILC to mainstream data quality systems into fieldwork plans, including supervisor accompaniments, weekly team reporting, back checks, and high-frequency data quality checks. 24 IRR target thresholds established by the tool developers are 90 for EGRA and 80 percent for Stallings. See Early Grade Reading Assessment Toolkit (2009). RTI International. and World Bank (2015). Conducting classroom observations: analyzing classrooms dynamics and instructional time - using the Stallings 'classroom snapshot' observation system: user guide (English). Washington, DC: World Bank Group. 22 Additionally, ES conducted site visits during the data collection to ensure it was carried out in compliance with the data collection protocols. Supervisor Accompaniments and Co-Enumerations Each field team had a master-trained supervisor who was onsite for the duration of data collection. The supervisor was responsible for overseeing fieldwork logistics, as well as ensuring adherence to data collection protocols (including sampling). In addition, supervisors observed 10 percent of all interviews, observations, and assessments, making sure to supervise each enumerator. Supervisors completed an electronic accompaniment form for each observation that assessed the enumerator’s performance and flagged areas for retraining. For Stallings and EGRA, supervisors were required to co-enumerate approximately 10 percent of all forms, which allowed for real-time IRR monitoring. Back Checks During data collection, approximately 10 percent of schools were revisited by an independent back checker who re-interviewed the HT to ensure that the field team followed all protocols and sampling procedures. In addition, the HT was re-asked a subset of questions from the survey to verify the stability of responses over time, as well as to ensure the forms were completed correctly by enumerators. Although the back-check analysis revealed that some of the agree/disagree questions appear to be unstable (i.e., are significantly different between the two time points), all other issues flagged during analysis were satisfactorily resolved through discussions with ILC. High-Frequency Checks High-frequency checks (HFCs) were conducted through customized Stata do-file(s) that were run on incoming, raw data at regular intervals throughout data collection. The purpose of HFCs is to proactively identify and remedy issues related to survey programming, question clarity, and enumerator error/performance. Specific checks covered included date/time consistency, survey completion, duplicates, routing/logic checks, variable distribution, “don’t know”/“refused” frequencies, “other” frequencies, outliers, and comment review. In addition, the team used ODK’s in-built data quality features, including speed violations, module time stamps, and soft-check suppression. Following each round of review of HFC, accompaniment, and back check data, the evaluation team flagged areas of concern in a cloud-based “Issues Log” that was regularly updated and monitored by both SI and ILC. The log allowed data quality issues originating from the field to be logged/tracked by SI and addressed by ILC in real time. Issues flagged as “most urgent” (e.g., poor reliability) were expected to be resolved in less than 24 hours, whereas issues with less urgency (e.g., basic cleaning tasks that do not require enumerator recall) could be resolved within one week. Overall, the log shows significant improvements in protocol adherence following the first week of data collection. DATA ANALYSIS Sample Weights Because the number of classrooms and pupils sampled at each school was fixed regardless of school size, it is necessary to weight the data to make it representative of the schools sampled and account for the fact that not all classes and pupils had an equal chance of selection. Using data from the HT and teacher surveys, the team constructed sample weights equal to the inverse of the probability of being selected. Specifically, the team created two weights: one at the classroom level corresponding to the number of streams in the pupil’s primary level at their school, which is applied to class- and teacher-level data; and one at the pupil level corresponding to the gender-specific primary-level enrollment at the pupil’s school. The pupil-level weight is then combined with the classroom weight and applied to pupil-level data. Weights are applied to the dataset as probability weights, or pweights, using Stata 14’s set of survey commands. As 23 the teacher survey was applied only to the treatment group at midline, we have elected to retain the sample weights calculated at baseline for all analyses presented in this report. Principal Component Analysis The midline dataset includes hundreds of variables. Although having a large number of variables allows for capturing complex concepts such as school resources and the quality of teaching practices, it is not practical to use all these variables in an unrestricted way during analysis for many reasons—including the problem of multicollinearity.25 In such cases, it is usually best to aggregate these variables into indices, which then convey the main information contained in a group of variables. One way to construct these indices is a method called principal component analysis (PCA). This method decomposes a set of correlated variables into another set of linearly unrelated components that can be used as independent variables in regression models. One advantage of using this method over other approaches to constructing an index (such as adding or averaging all variables in a group) is that it allows the data itself to guide the construction of the index, rather than an external determinant. This study used PCA to create indices for pupil household assets, teacher practices, and the school resources used in the regression models and presented in this report’s Findings section. Impact Analysis Whenever impact analysis is presented in this report, whether in total or for a subgroup, the analysis presented is a linear, difference-in-differences regression model. In practice, the estimates yielded by this model represent the difference between changes in outcomes of interest in the comparison group and changes in the outcomes of interest for the treatment group, holding all other potentially explanatory factors constant. To use a simplified example, although the treatment group may have improved by an average of 10 correct letters per minute (clpm) on the letter-sound subtask from baseline to midline, if the comparison group improved by an average of 4 clpm over the same period, the difference in those differences would equal 6 clpm. After accounting for other factors that may have explained the change using a multivariate regression model, the final estimate of program impact may vary in one direction or another from that raw, 6 clpm difference in differences, yielding an estimate attributable to the program. Although other models were tested, including Tobit models that adjust for floor effects from zero scores, we present primarily the linear models, as these have turned out to be the most conservative in estimating impact. In other words, there is reason to believe that the program impact is not less than the level of impact presented in this report. Both the Tobit and linear models are included in Annex II for reference. LIMITATIONS Selection Bias Because this is a quasi-experimental evaluation, there is a built-in risk that system actors, schools, teachers, and pupils in comparison schools will be different from those in treatment schools on a range of observable and/or unobservable characteristics. The evaluation was designed to minimize this risk by narrowing the sampling frame to schools that have been statistically matched on observables, as well as oversampling and purposively pruning schools that are driving imbalance post-baseline. However, as described in the baseline report and in the balance statistics included in Annex 1, there remained a slight imbalance between treatment and comparison at baseline, which may lead to biased midline and endline results. To further minimize this risk during midline analysis, the team corrected for baseline differences using difference-in￾25 Multicollinearity can cause large standard errors for the coefficients on the correlated variables, sometimes even resulting in a situation where two variables that are correlated and that should have the same signs actually end up with opposite signs. It can also cause two different but related independent variables that have been shown to have an effect on a dependent variable appear to have no significant effect whatsoever. This is because each one diminishes the effects of the other. These kinds of unanticipated results also contribute to a second problem, which is that regression models with large numbers of variables are difficult to interpret. 24 differences regression analysis. In addition, the gains in statistical power reported in the Sampling section of this report allow for censoring some of the schools that are driving this remaining imbalance without a corresponding loss in power. More specifically, for this report, the team conducted sensitivity analysis to ensure that the observed treatment effect is robust to the exclusion of certain baseline outliers. Given that the team conducted a cross-sectional pupil survey/assessment, there is risk of bias if the program influenced the likelihood of certain types of pupils remaining in the treatment sample—either by influencing attendance, dropouts, or repetitions.26 For this reason, the team closely monitored data on attendance, dropouts, and retention between treatment and comparison schools so that imbalance could be tracked from baseline to endline. If imbalance was observed on these dimensions, the team will conduct checks to see if the results were robust to the exclusion of pupils that are observably different on these dimensions. Response Bias Response bias encompasses the range of tendencies among interview respondents to answer in a way that is untruthful. For this evaluation, the risk of response bias comes primarily from recall bias (inability to recall facts or past events) and social desirability bias (tendency to answer in a way that will be seen as favorable instead of answering truthfully). Although it is difficult to overcome this risk in survey-based research, the team worked to minimize it through question framing, shortened recall periods, and preambles to sensitive questions reminding respondents of both the strict confidentiality of their responses and the importance of accuracy in research. Related to social desirability bias is the observer, or Hawthorne, effect, whereby respondents alter their ordinary behavior in response to the presence of an observer. For midline and endline data collection, the evaluation corrected for some of this effect on key behaviors such as attendance, which is captured during unannounced introductory visits at schools one to two days before commencing full data collection. However, the evaluation team believes that this bias could still be pervasive with respect to classroom observations, with teachers changing the way they conduct reading lessons and, in some cases, even developing special advanced lesson plans in anticipation of the classroom observation.27 Consequently, classroom observation data may not fully capture the true day-to-day practices of teachers. Contamination and Compliance Contamination occurs when there is crossover between treatment and comparison groups—specifically, if ideas or materials from the EGR program are brought to the treatment schools. When this crossover occurs, the internal validity of the evaluation would be threatened as children in the comparison group would no longer be isolated from the program. Furthermore, contamination may occur if similar interventions to the EGR program target the comparison sites selected for the study. The team believes the risk of contamination at the teacher level is moderate and has identified a few threats to contamination over the course of implementation: first, as teachers in Ghana regularly request transfers to other districts throughout the country, EGR program teachers may be transferred to comparison schools. Second, we have observed that in some cases, as common with program 26 For example, if low performers in P1 are less likely to repeat in EGR program schools, treatment schools may have lower average scores at endline in both P1 and P2, as those pupils may be lower performers in P2 but would have been relatively high performers in their class had they remained in P1 for a second year. 27 Advanced teacher preparation was reported by field teams during the first few days of data collection. However, based on ILC’s prior experience in Ghana, it is difficult to obtain consent if arriving at a school unannounced. To address the problem of advanced preparation, the team therefore developed a script that was reviewed with the HT during the scheduling phase stating, “This data collection effort involves classroom observations. Such observations will be the most valuable if teachers teach their reading class as they do on a NORMAL SCHOOL DAY and as if nobody is observing them. Again, the team is not assessing the performance of teachers but rather trying to understand what happens in Ghanaian classrooms on an ordinary school day. The teachers should not prepare special lessons or change their behavior in any way – doing so will undermine the quality and the accuracy of the research and create unnecessary pressure for the teachers.” 25 implementation, EGR program materials are regularly shared with non-EGR program actors (i.e., lesson plans, books, and training guides may appear in comparison schools). Finally, there are other large EGR initiatives throughout the county. To account for this compliance threat, the evaluation team implemented strong anticontamination measures. First, the team regularly measured and identified contamination threats. In cases where there are other programs or information has been spread, ES identified the scale and scope of that spread within our comparison group. Second, interview questions with the various actors during the data collection process were posed to identify contamination issues. One specific contamination threat noted at the midline evaluation stage was the rollout of the Jolly Phonics program—an English-based phonics approach—in 63 of the 235 (26.8 percent) comparison schools. The influence of the Jolly Phonics program on the comparison group presented a risk of increasing its average score, thereby causing the EGR program’s impact to be underestimated against this potentially improved comparison group. Accordingly, the evaluation conducted extensive robustness checks and sensitivity analyses to ensure that reported results were not contaminated by participation in the Jolly Phonics program. The team reran all analyses in multiple ways, including controlling for Jolly Phonics participation, excluding Jolly Phonics schools, and comparing Jolly Phonics schools with other comparison schools. However, across almost all subtasks and both English and Ghanaian languages, the results were remarkably consistent regardless of whether Jolly Phonics schools were included, excluded, or controlled for. The only difference was in listening comprehension in Ghanaian languages, where the team found that the EGR’s program’s estimated effect is slightly lower when Jolly Phonics schools are excluded. Furthermore, the team found few differences between Jolly Phonics schools and other comparison schools. Except for listening comprehension in Ghanaian languages, Jolly Phonics–treated schools were associated with a small but significant decrease in scores relative to other comparison schools across almost all tests. Based on this analysis, for the midline results, Jolly Phonics does not represent a significant threat to the validity of results. However, for the endline, the evaluation team will continue to track the implementation of the Jolly Phonics program and conduct additional testing to ensure the robustness of the EGR program’s endline results. Spillovers Spillovers occur when members of the comparison group receive indirect or secondary benefits from the treatment. For example, if children in treatment schools share lessons and reading materials with their neighbors from comparison schools, it may lead to an underestimation of program impacts, since comparison group children are receiving a spillover form of the intervention. To gauge the risk of spillovers and contamination at the pupil level, the team used baseline geocoordinates to calculate the distance from each comparison school to the nearest treatment school and found that 16 comparison schools were within 0–5 km of a treatment school, and 30 were within 5–10 km. Although the team does not anticipate this proximity to be a major threat to the study’s internal validity, the team will conduct sensitivity analyses to ensure that observed endline results are robust to the exclusion of treatment and comparison sites that are within 5 km of each other. External Validity The use of statistical matching to narrow the sampling frame will limit the extent to which results can be generalized to other schools in Ghana. Relatedly, the focus of the EGR program on only those districts where implementation fidelity is expected to be high may limit the usefulness of the evaluation results in informing program replication or scale-up in low-performing districts. Although the evaluation explored mediating factors of program impact, it was not designed to disentangle the various program components. Therefore, the evaluation results will be valid only for the entire “package” of EGR program inputs. Finally, 26 because the survey is school-based and pupil absenteeism is high in Ghana, the learners tested may not be representative of the broader school population. Finally, midline data were collected only on P1 pupils. Therefore, the findings and conclusions presented in the following sections apply only to summary statistics, program effects, and implementation fidelity predictors of program effects at the P1 level. 27 FINDINGS AND CONCLUSIONS This evaluation examined the EGR program’s effectiveness through five analyses: an analysis of impact on EGRA scores; an assessment of pupil and teacher outcomes; an assessment of implementation fidelity; an analysis of contextual factors, risks, and assumptions; and a regression analysis of predictors of program success. IMPACT ANALYSIS Question 1: To what extent did the EGR program improve reading skills in the Ghanaian language of instruction and prereading skills in English? Findings Program and control pupils had similar baseline assessment scores in four out of five tests (Figures 4–8). At midline, program pupils scored higher across all subtasks in the Ghanaian language assessment and across the letter-sound and nonword reading subtasks in the English language assessment. In the letter￾sound subtask, where there was a significant difference in the baseline balance statistics, the magnitude of the difference at midline was nearly 16.5 clpm larger on the Ghanaian assessment and 13.0 clpm larger on the English assessment. Zero score percentages were lower for all subtasks in the Ghanaian LOI (Figure 9) and for letter-sound and nonword reading subtasks in English. Figure 4: Listening Comprehension by Treatment and Time (%) 28 Figure 5: Letter Sounds by Treatment and Time (CLPM) Figure 6: Nonword Reading by Treatment and Time (CWPM) 29 Figure 7: Oral Reading Fluency by Treatment and Time (CWPM) Figure 8: Reading Comprehension by Treatment and Time (%) 30 Figure 9: Percentage of EGRA Zero Scores at Midline, Ghanaian LOI by Subtask and Treatment Status To put pupil overall achievement on the assessments at midline into context, the team applied the national reading scale established in Ghana’s national EGRA assessment, which combines the scores of the subtests to fit pupils into one of four reading categories: cannot read, can read some words but not understand, reads with some understanding, and reads with understanding and fluency. By this metric, a larger proportion of program pupils than control pupils could read some words in Ghanaian language at midline (the gain amounted to a 32 percentage point increase) and some program pupils were beginning to read with some understanding (Figure 10). No pupils in either the program or control schools were categorized as pupils who could read fluently and understand. Figure 10: Midline Level on National Reading Scale,28 by Treatment Status and Language 28 GHANA 2015 Early Grade Reading Assessment and Early Grade Mathematics Assessment: Report of Findings. 85.43 57.08 63.46 9.55 27.00 98.40 90.44 89.71 41.57 36.94 0 20 40 60 80 100 Control Treatment Listening Comprehension Letter Sounds Non-Word Reading Oral Reading Fluency Reading Comprehension 31 Comparing mean average subtask scores with mean subtask scores for the top 25 percent and top 10 percent of pupils in each group at midline shows that although average scores saw statistically significant improvement overall, the size of that improvement was larger for the top tier of pupils (Table 8). Table 8: Mean Scores of Top Performers on Ghanaian LOI Assessments, Midline Only EGRA Subtest Average, C Average, T Top 25%, C Top 25%, T Top 10%, C Top 10%, T Listening Comprehension (%) 34.30 43.78 74.52 77.88 74.52 100 Letter Sounds (clpm) 23.94 25.54 20.55 51.94 31.05 61.25 Nonword Reading (cwpm) 4.08 5.00 0.85 18.24 8.20 25.93 ORF (cwpm) 1.54 6.90 0.90 22.72 8.64 33.76 Reading Comprehension (%) 0.53 4.96 0.42 4.95 0.42 32.89 These changes in scores cannot be attributed to the program alone, however. To isolate the causal effect of the EGR program on Ghanaian and English EGRA scores, the team examined program impact in terms of the units of each score, estimating the program effect using a Tobit model (correcting for censoring based on zero scores) and an ordinary least squares (OLS) model (Table 9). Whereas the OLS regression treats all zero scores as the same, the Tobit regression assumes variability in pupil aptitude across zero scores and attempts to account for “flooring effects” induced by the assessment’s difficulty. Because the majority of pupils in the sample had zero scores on the majority of subtasks in either language, the Tobit model estimates much larger impacts than the OLS model. The models are consistent, however, on direction and significance of effects, and although OLS will be used for much of the analysis of effect size, both models are valid ways of estimating EGR program effects. 59% 64% 90% 57% 40% 36% 10% 40% 3% 0% 20% 40% 60% 80% 100% Percentage of students Cannot read Read some words, but not understand Read with some understanding Read fluently and understand Ghanaian LOI, Treatment Ghanaian LOI, Comparison English, Comparison English, Treatment 32 Table 9: EGR Program Impact on EGRA Scores EGRA Subtest, Ghanaian LOI Tobit Estimate p-value OLS Estimate p-value Listening Comprehension (percentage pts) 11.60*** 0.001 4.72*** 0.000 Letter Sounds (clpm) 21.67*** 0.000 18.54*** 0.000 Nonword Reading (cwpm) 18.00*** 0.000 4.04*** 0.000 ORF (cwpm) 25.80*** 0.000 5.17*** 0.000 Reading Comprehension (percentage pts ) 67.10*** 0.000 6.17*** 0.002 EGRA Subtest, English Tobit Estimate p-value OLS Estimate p-value Listening Comprehension (percentage pts) -14.00 0.105 0.411 0.278 Letter Sounds (clpm) 13.02*** 0.000 13.09*** 0.000 Non-Word Reading (cwpm) 10.82*** 0.000 2.94*** 0.000 ORF (cwpm) 1.52 0.287 0.90 0.239 Reading Comprehension (percentage pts ) 8.74 0.368 -2.47 0.101 Note: The p-value represents the percent likelihood that the impact is statistically equivalent to zero. P-values below 0.10 are significant at a 90 percent confidence level (i.e., one can be at least 90 percent certain that the effect observed is non-zero), p-values below 0.05 are significant at a 95 percent confidence level, and so forth. The (more conservative) OLS estimates show that a program pupil would correctly identify 18.54 clpm more on the Ghanaian assessment and 13.09 more clpm on the English assessment than he or she would have in the absence of the program. The data show a difference between program and control pupils on these subtests with greater than 99.9 percent confidence.29 Significant positive impacts were also observed across all subtasks of the Ghanaian LOI assessment and the nonword reading subtask of the English language assessment (measuring prereading skills). However, the program did not have a large impact on higher-level skills in English language—significant effects were not observed on English reading comprehension, English listening comprehension, and oral reading fluency. (Full regression outputs, including alternative model specifications, are accessible in Annex II of this report.) The team compared program effects across subtasks—overall and by gender—by normalizing the effects into a standardized “effect size,” which measures the changes in standard deviation units (Figures 11 and 12).30 The magnitude of program effects was larger on the Ghanaian LOI assessment than on the English language assessment. On both assessments, the effect on letter-sound identification was largest, and the effect on comprehension subtasks relatively small (except for reading comprehension on the Ghanaian subtask). There were no statistically significant differences between program effect for boys and for girls, except for Ghanaian oral reading fluency scores, where girls experienced a substantially larger program effect. 29 In other words, the population difference between treatment and comparison groups is not zero. 30 Specific effects for other subgroups, such as rural/urban subgroups and language match subgroups, are presented later in the Program Predictors of Success section. 33 Figure 11: EGR Program Effect Size on Ghanaian LOI Assessment, by Subtask and Gender Figure 12: EGR Program Effect Size on English Assessment, by Subtask and Gender The Ghana EGR program’s impact on mother-tongue letter-sound identification was larger than that of any other program included in the World Bank review of EGR interventions worldwide (by either Tobit or OLS estimates), and impact on oral reading fluency was larger than that of all but four other programs (Figure 13).31 31 Graham, Jimmy and Sean Kelly. January 2018. “How Effective Are EGR Interventions? A Review of the Evidence.” http://documents.worldbank.org/curated/en/289341514995676575/pdf/WPS8292.pdf -0.4 0.1 0.6 1.1 1.6 Oral Reading Fluency Listening Sounds Invented Words Reading Comprehension Listening Comprehension Effect Size Male Female Overall -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Oral Reading Fluency LIstening Sounds Invented Words Reacing Comprehension Listening Comprehension Effect Size Male Female Overall 34 Figure 13: Benchmarking Ghana EGR Program Effect Size Relative to Other Interventions Conclusions The EGR program substantially improved pupil EGRA scores. Despite these improvements, however, average pupil reading levels were not as high as expected by USAID and the Government of Ghana.  EGRA scores among program pupils rose across all subtasks in the Ghanaian language assessment and in the letter-sound and nonword reading subtasks in the English language assessment.  The percentage of zero scores declined significantly, but the remaining zero scores are driving low average scores.  Program pupils improved in higher-level subtasks (oral reading fluency, reading comprehension).  The gains observed in letter-sound identification and nonword reading in the English language assessment indicate that pupils are applying the skills learned in the Ghanaian LOI to their reading of English. However, as predicted, the program did not have a significant impact on higher-level skills in English language in the short term.32  Program pupils who shifted categories from “nonreaders” to pupils “who can read some words but not understand,” still are not categorized as readers. 32 See Cummins, Linguistic Interdependence and the Educational Development of Bilingual Children, 1979. Brisbois J. (1995). Connections between first and second language reading. Journal of Literacy Research, 27 (4), 565–584. Bernhardt E.B., & Kamil M.L. (1995). Interpreting relationships between L1 and L2 reading: Consolidating the linguistic threshold and the linguistic interdependence hypotheses. Applied Linguistics, 16, 15–34. ELIZABETH B BERNHARDT, MICHAEL L KAMIL; Interpreting Relationships between L1 and L2 Reading: Consolidating the Linguistic Threshold and the Linguistic Interdependence Hypotheses, Applied Linguistics, Volume 16, Issue 1, 1 March 1995, Pages 15–34 0 0.5 1 1.5 2 2.5 3 Oral Reading Fluency Letter Sounds 35  The top 10 percent and top 25 percent of program pupils in the program achieved the largest gains and are meeting or exceeding program expectations.  Among EGR programs worldwide, the Ghana EGR program demonstrated one of the largest effect sizes on key early grade reading skills. OUTCOME ASSESSMENT The evaluation examined several teacher and pupil outcomes, including use of classroom time, use of teaching materials, application of program-aligned teaching practices, use of adaptive teaching, teacher incorporation of coach and mentor feedback, and pupil engagement with books both inside and outside the classroom. Question 2: To what extent did the EGR program improve teacher and pupil classroom time on task? Findings Use of Classroom Time Using the Stallings “Classroom Snapshot” Observation System, the evaluation team examined changes in program teachers’ use of time and materials between baseline and midline. Ten “snapshots” equally divided between the start and end of class were taken, and data were recorded on the activity in which the teacher was engaged. Activities were divided into three types: learning, classroom management, and off-task activities (Figure 14). Figure 14: Teacher Use of Reading Class Time (Baseline Left, Midline Right) Teacher use of classroom time improved, with significant reductions in the proportion of class time spent off task, on general classroom management, and reading aloud. More time was spent on assignments, 36 practice, and demonstration. Although time spent on classroom management activities rose by 5 percent at the beginning of class from baseline to midline, at midline it tapered over the middle intervals, taking up nearly 20 percent less class time over the last two intervals than at baseline. Perhaps one of the biggest improvements from baseline to midline was the greater proportion of class time at the early intervals devoted to practice. Figures 15 and 16, showing the overall proportion of class time distributed among the three categories of activity, reveal the same trends. More time was spent on learning activities with less time spent off task or on classroom management. Figure 15: Comparison of Classroom Activities by Survey Wave Similarly, the evaluation team examined the relationship between the proportion of class time each activity took at baseline compared with midline (Figure 16). Time off task and on some classroom management tasks such as preparing materials for lessons or transitioning between activities declined and some learning activities increased. Teaching practices changed, with reductions in reading aloud, verbal instruction, and copying and increases in assignments, pupil practice, and demonstration. 78% 18% 4% 85% 13% 2% 0% 20% 40% 60% 80% 100% Learning Activities Classroom Management Off-Task Midline Baseline 37 Figure 16: Use of Class Time by Activity and Survey Wave Use of Teaching Materials Tracking use of teaching materials is important to determine implementation fidelity, as well as the project design aspects that influence pupil outcomes. Teacher use of materials was tracked by the Stallings tool, which records the material that is being used by the teacher and notes if the teacher is working with an individual, small group (2–5 pupils), large group (from 6 pupils to all pupils minus one), or everyone in the class.33 One of the most important findings concerning teacher material use is the reduction in time pupils spent without any learning materials. Although teachers were observed using the blackboard with roughly the same frequency at midline as at baseline, teachers at midline were more likely to use materials such as textbooks for a larger proportion of class time, especially when working with small groups (Figures 17– 19). The improvements in use of class time and teaching materials were likely influenced by the structure of the EGR program lesson plans, which provided contents and pacing of teaching and learning activities, and by the availability of teaching materials, which enabled teachers to begin lessons earlier. 33 Because the Stallings tool is designed as a standardized measure for both treatment and comparison schools, it was not customized to detect project-specific practices or materials, which are captured instead through the Stallings Observation Tool. The Stallings tool uses preset categories of materials. 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% Read Aloud Demonstration Discussion Practice Assignment Copying Verbal Instruction Discipline Classroom Management Social Interaction Teacher Uninvolved Teacher Out of the Room Learning Activities Classroom Management Off-Task Midline Baseline 38 Figure 17: Materials Use by Pupil Group Size 27% 28% 1% 39% 5% 0% 0% 13% 68% 0% 18% 0% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% No material Textbook Notebook Blackboard Learning Aides ICT Cooperative Materials Use-One Pupil Baseline Midline 39 Figure 18: Materials Use Small Group Figure 19: Materials Use by Large Group/Everyone 33% 25% 0% 39% 3% 0% 0% 2% 51% 0% 45% 2% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% No material Textbook Notebook Blackboard Learning Aides ICT Cooperative Materials Use-Small Group Baseline Midline 20% 28% 1% 45% 6% 0% 0% 10% 44% 1% 40% 4% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% No material Textbook Notebook Blackboard Learning Aides ICT Cooperative Materials Use-Large Group/Everyone Baseline Midline 40 Conclusions Teacher use of classroom time and teaching materials improved. Overall more time was spent on learning with less time spent off task and with classroom management. Furthermore, the distribution of learning activities was more in line with instructional best practices.  Less class time was spent off task, on general classroom management, and reading aloud.  More class time was spent on assignments, practice, and demonstration.  More class time at the early intervals was devoted to practice.  Pupils spent less time without learning materials. Question 3: To what extent did the EGR program increase the likelihood that teachers apply program-aligned best practices in teaching early grade reading? Findings To assess this outcome, program teachers were asked to report the frequency with which they employed various program-aligned best practices and were observed by independent enumerators to assess the frequency with which they demonstrated these practices in the classroom at baseline and midline. By the teachers’ own estimations, they increased their use of nearly all the reading instructional practices included in the survey. The only practices that were reported less frequently at midline were teaching letters through songs, teaching vocabulary through writing words on the board, and using choral reading or echo reading to improve reading speed and accuracy. These practices are not necessarily aligned with program recommendations for P1. Figure 20 displays changes from baseline to midline in self-reported employment of phonics methods. Overall, teachers reported the use of different phonics methods at consistently higher levels than at baseline. Figure 20: Extent to Which Teachers Employ Phonics Methods (Self-Reported) Figure 21 presents the degree to which program-aligned practices were observed during structured observations at midline. The proportion of treatment teachers who reported employing various program￾aligned phonics methods, such as blending letter sounds and identifying letter names or sounds, was 92% 88% 70% 86% 70% 88% 85% 43% 59% 62% 68% 68% 77% 77% 0% 20% 40% 60% 80% 100% Clap words or syllables Blend letter sounds and syllables together Explain vowels and consonants Segment words into syllables Produce words that rhyme Identify the first sounds in a word Identify same or different words or sounds Baseline Midline 41 frequently similar to the proportion that displayed them in practice. The major exception, however, was the practice of asking pupils to clap syllables of words, which 93 percent of teachers reported employing, compared with only 43 percent who were observed employing this method in practice. Figure 21: Extent to Which Teachers Employ Phonics Methods (Observed, Midline Only) Aside from phonics methods, the proportion of teachers who reported employing other specific methods for alphabet, vocabulary, reading fluency, and reading comprehension instruction rose. Noteworthy examples of these methods include asking learners prereading and post-reading questions; using letter games; using alphabet cards; posting letters on the class wall; giving synonyms; guided oral reading; silent reading (especially large shift); having pupils follow words with a finger; using visual representations of text; and applying multiple methods to support comprehension, including games and group work. However, unlike the use of phonics methods for reading instruction—where teachers not only reported improvements in practices but were observed to display these practices more often as well—there was no significant change in the proportion of teachers who displayed most alphabet, vocabulary, reading fluency, and reading comprehension instruction methods at least some of the time during structured observations compared with baseline. There were even reductions in the extent to which some of these were observed, including engagement of learners in reading activities or games, asking pupils to sound out a word, asking prereading questions, asking learners to read aloud, and explaining the meaning of new words. This result may be reasonable, though, given the appropriateness of some of these activities to P1 learners and the frequency with which they may have been included in scripted lesson plans compared with previous lesson plans. Predictably, there were increases in the proportion of teachers using a lesson plan at midline—nearly all according to self-reporting and structured observation. Likewise, the proportion of teachers observed to be using materials in a language that their pupils understand rose from 80 percent at baseline to 97 percent at midline. Aside from instructional practices, there was evidence of changes in program-aligned beliefs from baseline to midline among treatment teachers. Specifically, there was a reported increase in the proportion of teachers who “strongly agreed” that children should learn to read syllables before learning the whole 13% 6% 35% 12% 40% 9% 13% 57% 28% 22% 20% 16% 24% 21% 21% 10% 58% 71% 43% 72% 34% 70% 66% 33% 1% 1% 2% 0% 2% 0% 0% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Midline Baseline Midline Baseline Midline Baseline Midline Not Done Done Sometimes Done Very Well Not Applicable Encourages learners to sound it out when they don't know a word Asks learners to recognize letters and say letter names and/or sounds Provides instructions on how to blend sounds and/or decode syllables Pupils clap syllable of words 42 word, from 52 percent at baseline to 74 percent at midline. There was similarly a significant increase in the teachers who “disagreed” or “strongly disagreed” that children in Ghana would benefit more if they were taught how to read initially using English, from 51 percent at baseline to 82 percent at midline. Although this figure reflects greater knowledge about mother-tongue instruction, it also reflects the perceived effectiveness of the intervention by teachers, as they had been using it for many months at the time of the interviews. Conclusions: Although teachers generally believed that they were demonstrating nearly all the program-aligned best practices, observation showed more nuance to their progress.  Improvements in the use of phonics methods for reading instruction were both reported and observed.  The proportion of teachers using a lesson plan at midline grew.  There were increases in program-aligned beliefs in the importance of learning syllables before words and learning in mother tongue initially.  There was no significant change in the proportion of teachers who displayed most alphabet, vocabulary, reading fluency, and reading comprehension instruction methods at least some of the time during structured observations compared with baseline.  Certain best practices declined between baseline and midline, including engagement of learners in reading activities or games, asking prereading questions, asking learners to read aloud, and explaining the meaning of new words. Question 4: To what extent did the EGR program increase the likelihood that teachers engage in adaptive teaching? Findings To assess adaptation, this evaluation relied on both self-reported and observed data on teacher behavior. However, there was not much quantitative evidence for increases in adaptive teaching based on pupil needs from P1 teachers in treatment schools from baseline to midline. There were significant decreases in the frequency of written and oral pupil assessments at midline (Figure 22). Similarly, the proportion of teachers who were observed to assess pupils at least sometimes dropped from 96 to 89 percent. The frequency with which teachers checked pupil workbooks and homework remained practically stagnant, although these both occurred relatively frequently at midline, with 84 and 95 percent of teachers engaging in this assessment at least weekly. Figure 22: Teacher Frequency of Assessing Pupil Reading Performance (Self-Reported) 43 Although 96 percent of teachers reported adjusting their teaching based on individual assessment at baseline, only 91 percent did so at midline. There was a 10 percent increase in teachers who adapted by providing struggling learners with tutoring outside of class (Figure 23). The team hypothesizes that this increase correlates with the decrease in the number of teachers who chose to repeat previous lessons or make new lessons to teach difficult content in a different way, pay more attention to high-performing learners, and pair struggling learners with strong ones during class. In other words, the increase in providing individual help outside of class displaced these other practices. Figure 23: Methods for Adjusting Teaching Based on Individual Assessment (Self-Reported) 23% 74% 64% 80% 72% 20% 33% 15% 3% 3% 1% 0% 20% 40% 60% 80% 100% Daily Weekly Monthly Quarterly Annually Not at All Written Assessments, Baseline Written Assessments, Midline Individual Oral Assessments, Baseline Individual Oral Assessments, Midline 71% 62% 92% 35% 39% 49% 73% 8% 24% 95% 75% 88% 45% 29% 41% 82% 9% 33% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Repeat previous lesson Make new lessons for hard topics Give struggling students more attention Give high-performing students more attention Tutor struggling students outside class Find tutors to help struggling students outside class Pair struggling and proficient students during class Skip lessons that students do well in Quickly go through lessons that students do well in Baseline Midline 44 Conclusions Although program teachers assessed pupils less frequently than before, they were still more likely to engage in adaptive teaching practices. The frequency of oral and written assessment mostly declined from daily to weekly while adaptive practices increased. This finding indicates an unnecessarily high frequency of assessment at baseline. The changes still fit within the structured lesson plan practices. Question 5: To what extent did the EGR program increase the likelihood that teachers incorporate coach and mentor feedback? Findings To examine the extent to which teachers adapted instructional approaches in response to coaching, the team first verified whether there was an increase in the amount of coaching, mentoring, and SBI received by teachers at midline and then examined the response to this mentoring according to teachers and HTs. Ninety-nine percent of program teachers had attended formal in-service training or professional development over the past school year at midline, compared with only 40 percent at baseline. The average teacher had attended seven days of training in the past school year on Ghanaian language reading instruction, compared with only 1.3 days at baseline. Meanwhile, 84 percent of teachers surveyed at midline had attended SBI on Ghanaian language reading instruction. HTs at around 91.9 percent of program schools reported hosting SBI sessions in the current school year, compared with only 23.8 percent of comparison schools. Based on recorded attendance of SBI meetings, treatment schools held an average of 4.1 SBI sessions in the past school year attended by an average of 6.7 teachers per session, whereas comparison schools held an average of 2.4 SBI sessions attended by an average of 9.6 teachers per session.34 The team observed increases in the proportion of teachers receiving coaching and mentoring in program schools at midline. Namely, 97.5 percent of HTs surveyed at midline in program schools reported having teachers who received coaching or mentoring in the past school year, compared with 69 percent of HTs at the same schools at baseline, 55.7 percent of HTs at comparison schools at midline, and 60.3 percent of HTs at comparison schools at baseline. Program schools were more likely than comparison schools to have coaching or mentoring given by HTs, CLs, or CSs. DTSTs are a specific part of the EGR program; they were not present at baseline but were available for mentorship at midline. Within program schools, the proportion of teachers who received coaching or mentoring from a CL increased from baseline to midline from 26.6 to 82.1 percent of schools. According to teacher interviews, coaching and mentoring received in program schools increased, although this coaching and mentoring was not received by all teachers in the school. Specifically, 64 percent of program teachers received coaching or mentoring in the past school year at midline, compared with only 36 percent at baseline. Program teachers received more coaching and mentoring, especially from HTs, CLs, and LSs, while receiving significantly less coaching and mentoring from other teachers in their schools (Figure 24). 34 Only 26.8 percent of comparison schools where SBI sessions were held, or 6.4 percent of the total surveyed, could present recorded attendance of these sessions. In turn, 71.2 percent of treatment schools where records were kept or 55.7 percent of the total survey could present their records. So it is possible that the actual figures for all comparison schools are significantly lower than what is presented here. 45 Figure 24: Source of Coaching/Mentoring Given in Last School Year The types of coaching that program teachers received differed significantly from coaching at comparison schools. Specifically, 76.4 percent of HTs at program schools reported coaching being given based on an observed lesson, compared with 56.5 percent of HTs at comparison schools. At baseline, the types of coaching received by the two groups were statistically equivalent. At midline, program school coaches were more likely to use classroom observation–based coaching methods, rather than training- or exam￾based methods. This finding appears to align with EGR program practices, which emphasize feedback on classroom observation as a coaching method. To understand if teachers were effectively applying lessons learned from coaching, the evaluation team spoke with both teachers themselves and their HTs. Overall, teachers responded favorably to coaching and mentoring regardless of treatment status or time period. At midline, between 85 and 90 percent of teachers who received coaching and mentoring from an HT, CL, CS, or learning specialist found it to be somewhat or very helpful. Virtually all HTs whose schools included teachers receiving coaching based on an observed lesson believed that this was received positively and that teachers adapted their approach to reading in response. Similarly, 99 percent of respondents in treatment schools felt that formal SBI sessions positively impacted their ability to teach reading. Table 1010 lists the most commonly cited types of teaching adaptations to SBI and coaching, according to treatment teachers and HTs. Table 10: Positive Teaching Adaptations in Response to SBI, Coaching, and Mentoring SBI Coaching Based on Observed Lessons  General teaching ability (39%)  Collaboration with other teachers (15%)  Understanding of non-GES language (9%)  Classroom management (7%)  Pupil assessment (2%)  Teaching vocabulary (1%)  Better lesson planning (81%)  More interactive class (64%)  Providing equal opportunities for pupils (34%)  Individual assessment (27%)  Group assessment (20%) 58% 15% 39% 17% 17% 68% 79% 81% 0% 11% 36% 9% 35% 54% 17% 69% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% DTST GES/MOE An NGO Literacy Specialist Other teachers at school Circuit Supervisor Curriculum Lead Head Teacher Baseline Midline 46 Although all HTs, regardless of treatment status, felt that teachers in their school applied what they learned from SBI sessions at midline, 64 percent from program schools, compared with only 46 percent from comparison schools, reported that teachers were applying it “very well” compared with “with room for improvement” (33 percent for treatment and 45 percent for comparison) or “not very well” (0.5 percent for treatment and 9 percent for comparison). Conclusions Program teachers were significantly more likely to adapt instructional approaches in response to coach and mentor feedback. Interview findings about this adaptation give us a stronger picture of why these adaptations are taking place:  Program teachers were more likely to receive both SBI training as well as coaching and mentoring from HTs, CLs, DTSTs, and CSs.  Program teachers were more likely to get coaching and mentoring based on observed lessons than generalized training and coaching. The adaptation guidance based on observation was reported to be more comprehensive, specific, and actionable than generalized guidance and coaching. Based on these findings, the team concluded not only that teachers can adapt to feedback but also that the structure of that feedback, based on specific observations, likely provides more helpful guidance than more generalized approaches. Question 6: To what extent did the EGR program increase the likelihood that pupils engage with books both inside and outside the classroom? Findings Compared with baseline, where around 33 percent of program pupils had read a book on the most recent school day, 51 percent of midline program pupils reported having done so. Program pupils were more likely to have read at school than their peers at comparison schools (47 and 35 percent, respectively), whereas comparison readers were more likely to have done so at home (83 versus 78 percent). These differences may be explained in part by the fact that more comparison readers read a book that belonged to them or their family (21 versus 12 percent of program pupils), whereas program pupils were more likely to have read a book from a classroom library (35 versus 6 percent of comparison pupils). Similarly, at baseline, program and comparison pupils were read to at home at about the same frequency, with only about 17 percent of pupils being read to at least twice per week. At midline, only 15 percent of comparison pupils were read to at this frequency, compared with 24 percent of program pupils. Significantly more program pupils than comparison pupils were read to at home every day at midline, and significantly fewer were hardly ever read to (Figure 25). 47 Figure 25: Pupils Who Reported Reading on Most Recent School Day The provision of “take-home” books by the EGR program could explain the improvement in engagement with books among program pupils. Although pupils tended to report liking reading regardless of treatment status, program pupils tended to be more enthusiastic at midline than those at comparison schools—92 percent of program pupils liked reading and 3 percent disliked reading, compared with 85 and 8 percent of comparison pupils, respectively. Program and comparison pupils were equally enthusiastic about reading at baseline. As at baseline, program and comparison schools were equally likely to have had a library at midline, and libraries at program and comparison schools were equally likely to have books in the GES-approved Ghanaian LOI. However, pupils in comparison schools were significantly more likely to report not taking books home from school at midline because there were not enough books in the library to take home (Figure 26). Finally, fewer children in the treatment program reported “There are not enough books in the library” or “They don’t understand the language of the books” as reasons they could not take books home (Figure 27). Fear that they will lose or damage the books was still a prevalent reason for not taking books home. Some children reported that “They are not allowed to [take books home] based on the rules of the sponsor.” This is in line with reports from HTs that some schools restrict access even to EGR program materials for fear of losing or damaging them—causing some limitations in implementation—as none of the learning materials are to be restocked or replaced in the second year of the intervention. 30.96% 27.70% 29.32% 33.02% 51.31% 42.19% 0% 20% 40% 60% 80% 100% Baseline Midline Overall Treatment Control 48 Figure 26: Pupil-Reported Frequency of Being Read To at Home, Midline Figure 27: Reasons Pupils Do Not Take Library Books Home, Midline Every day 7% 2-3 times per week 8% Only sometimes 40% Hardly ever 45% Comparison Every day 2-3 times per week Only sometimes Every day 15% 2-3 times per week 9% Only sometimes 41% Hardly ever 35% Treatment 0% 0% 5% 0% 5% 95% 5% 9% 5% 5% 50% 82% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% They don’t understand the language of the books They are not allowed to based on the rules of the sponsor They don’t want to take books home No one at home can read with them There are not enough books in the library They will lose or damage them Comparison Treatment 49 Conclusions The EGR program improved pupil engagement with books both inside and outside the classroom.  Program pupils were more likely to engage with books than were their peers at comparison schools at midline. They were also read to more often at home and more enthusiastic about reading than their peers.  Books were more accessible to pupils in program schools both generally and in a language that pupils understood.  Comparison pupils were more likely than program pupils to have read at home, likely because of how books provided to schools were used. The findings indicate that when books were made available (as they were in the EGR program), teachers were more likely to disallow pupils from taking the books home because they may be lost or damaged, suggesting that they are deciding to keep the books in the classroom for what they consider to be the pupils’ long-term good. 50 FIDELITY ASSESSMENT Question 7: To what extent was the program implemented according to plan? The evaluation team examined the three program areas—curriculum implementation; materials provision; and teacher training, coaching, and support—and their components to assess implementation fidelity (Table 11). The evaluation team first examined each of the three program areas individually and then examined overall fidelity compared with the relative fidelity of components using an “implementation fidelity index” (explained below). Findings and conclusions are presented together. Table 11: Dimensions of Implementation Fidelity Program Area Component Definition Curriculum Implementation Pace Progress on scripted lesson plans in relation to program-prescribed pace, according to weeks ahead or behind pace when class was observed Lesson performance Teacher fidelity to steps in the lesson plan and pupil engagement with the lesson plan, as observed by independent observer Materials Provision Teacher textbook Use of teacher textbook by teacher, as observed by independent observer Pupil workbook Use of pupil workbook by teacher and pupils, as observed by independent observer Supplemental materials Presence of supplemental materials such as teacher guide, alphabet chart, flash cards, Big Books, and conversation posters, as observed by independent observer Teacher Training, Coaching, and Support Reading instruction training Compliance with expectation for eight or more days of in-service training on reading instruction in Ghanaian LOI, as reported in teacher interview Observation and coaching Compliance with expectation for biweekly observation and coaching by HT and/or CL, as reported in HT and CL interview School-based in￾service (SBI) Compliance with expectation for one hour-long SBI session per week, as reported in HT interview CS monitoring Compliance with expectation for monthly visits from CS to monitor program implementation, as reported in CS interview Findings Implementation Fidelity by Component Program Area 1: Teacher Training, Coaching, and Support The data used to determine the implementation fidelity of teacher training, coaching, and other support were taken from teacher, HT, CL, and CS surveys collected through this evaluation (Annex V) at 241 program schools. The survey questions asked about the length (in hours or days) and frequency (weekly, monthly, etc.) for various inputs compared with the targets established by the EGR program and USAID (Table 12). 51 Table 12: Teacher Training, Coaching, and Support Input Targets Input Target Teacher In-Service & Follow-Up Training 11 days (5 initial/6 follow-up) Coaching by HT or CL Once every two weeks School-Based In-Service Meetings Once every two weeks CS Monitoring Visits Twice per term Teacher In-Service and Follow-Up Training This training regime was intended to be delivered by district and circuit officials who were trained directly by the EGR program. The first five-day training was designed to instruct KG2 and P1 program school teachers to use the Learning Activity Curriculum35 in their reading classes, and three two-day follow-up trainings (one each term) were to be delivered by district and circuit staff to provide further support in using the curriculum and materials. According to teacher and HT surveys, half of the program schools reported receiving nine or more days of teacher in-service and follow-up training combined,36 a quarter reported receiving between five and nine days of training, and a final quarter reported receiving four days or fewer. Coaching Coaching of teachers was intended to be conducted by HTs and/or CLs once every two weeks in coaching “sessions” to include any combination of individual observations and subsequent feedback. According to teacher and HT surveys, more than 80 percent of treatment schools received the intended inputs for coaching (at least twice a month) from HTs and CLs. Approximately 10 percent of teachers surveyed reported receiving coaching support less than once a month. School-Based In-Service Meetings According to teacher and HT surveys, 90 percent of teachers reported attending SBI meetings once every six weeks or more; however, once every two weeks was the target, indicating either that this measure has not been implemented well across the sample or that the expectations of the intervention in this area were unrealistic. Circuit Supervisor Visits In line with the EGR program model, CSs were required to conduct follow-up visits to schools to assess the program’s implementation fidelity. Among their responsibilities were to conduct regular visits to schools to collect data on teacher attendance, absenteeism, use of teaching and learning materials, and pedagogical approaches used for teaching reading in the classroom. CSs held regular interviews with HTs, CLs, and teachers on use of teaching and learning materials such as the scripted lesson plans, program￾related textbooks and learning materials, and the degree to which teachers were on track with their weekly lesson implementation targets. In addition, the CS observed lessons and SBI in the school. For midline treatment schools, circuit supervisors reported that they visited schools 1.16 times per month on average, on track with the target of six visits annually. Furthermore, 96.77 percent used an electronic tablet during their visit. These outcomes are within the team’s expectations and demonstrate a high implementation fidelity for circuit supervisions. 35 This includes materials in both English and the official school LOI (11 in all). 36 Since the third teacher in-service follow-up session was scheduled in the spring term around the same time as data collection for this report, a total of nine days is considered “full” implementation. 52 Program Area 2: Materials Provision The data for materials provision were taken from a classroom observation implementation fidelity tool (Annex V), where pupil, teacher, and classroom supplemental materials were the components for the sub￾index (Table 13). This index does not account for library or “take-home” books provided by the EGR program as it was not possible to verify this component at the school level. Table 13: Materials Input Targets Input Target Teacher Materials All teachers have a teacher textbook Pupil Materials All pupils have a pupil textbook Classroom Supplemental Materials All classrooms have the following materials: • Conversation book and picture poster • Big Books • Flashcards and storage box • Alphabet chart • Pupil textbook for the teacher Teacher and Pupil Materials All 241 teachers observed had a copy of the EGR program teacher textbook, and in 80 percent of classrooms observed, “all” or “most” of the pupils had their own pupil book. Alternatively, roughly one in five classrooms were observed where only “some” (<50 percent) or “none” of the pupils had their own pupil book, and in only 10 percent of classrooms did “all” pupils have their own pupil book (Figure 28). Figure 28: Proportion of Pupils in Observed Classrooms with Their Own Workbooks Classroom Supplementary Materials There was considerably more variability in the distribution of supplementary classroom materials such as alphabet charts and flash cards (Figure 29). Alphabet charts and Big Books were observed in only one￾third to one-half of classrooms, and conversation/picture books and flashcards were observed in roughly two-thirds of classrooms. Most teachers had their own copy of a pupil book. In all, 80 percent of observed classrooms were missing at least two of the six supplemental materials listed in Figure 29. Materials degradation and attrition, the “management” of access to limited materials, and fluctuating enrollment are common to many parts of Ghana and present ongoing challenges to resourcing every pupil. These factors have a negative effect on reading achievement. However, “too much” access is not a factor. In fact, mere access to teaching and learning materials is correlated with improved use of classroom time and positive pupil outcomes. Figure 29: Supplemental Classroom Materials Observed 6% 15% 71% 9% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% None Some (<50%) Most (>50%) All 53 Program Area 3: Teacher Curriculum Implementation The data for this sub-index were taken from a classroom observation implementation fidelity tool (Annex V). Teacher instructional implementation fidelity is measured by examining two output components: pace and performance. Pace refers to how closely a teacher is adhering to the prescribed lesson schedule, which indicates the lessons that are to be completed each day. Performance refers to how well the teacher followed the scripted lesson plans and the extent of pupil participation in the observed lessons (Table 14). Table 14: Curriculum Implementation Output Targets Output Target Pace All teachers adhere to the lesson schedule. Performance All teachers follow the lesson plan steps “all” of the time. All pupils are observed as participating in the lesson. Pace Of 241 total treatment classrooms observed, only one was observed to be on or ahead of the prescribed curriculum schedule. Roughly one in five classrooms were 10 or fewer days behind schedule. The vast majority of classrooms, about 80 percent, were two-and-a-half weeks or more behind the recommended lesson schedule (Figure 30). Figure 30: Curriculum Pacing by Observed Classroom (Number of Days Behind Schedule) In interviews, 60 percent of program teachers indicated that the EGR program curriculum pace was “much too fast” or “a little too fast.” The same proportion reported that they felt pressure to keep pace with the lesson schedule prescribed by the EGR program. No teachers described it as being too slow. 90.94% 34.65% 63.39% 88.82% 52.57% 66.14% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Teacher has Pupil Book Alphabet Chart in Class Teacher has Flash Cards Teacher has Flash Card Storage Box Teacher has Big Books Teacher has Book of Conversation/Picture Poster Observed Not Observed 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 1-10 Days 11-20 Days 21+ Days 54 Additionally, half the teachers interviewed reported that they “sometimes” go off script when using the EGR program lesson plans. The most common reason cited is to repeat a section or adapt their teaching methods. Other reasons included adjusting the pacing of the curriculum and making “improvements.” Finally, variability in the implementation of coaching and support and pupil and classroom materials, as well as time spent on reading instruction, reported in more detail below, are important factors likely contributing to pacing. Performance Observers indicated that 95 percent of program teachers properly followed the directions and steps outlined in the teacher guides “all” or “most” of the time, with nearly all pupils observed as participating. Indeed, 70 percent of teachers interviewed at program schools found the step-by-step lesson plans in the program curriculum “very easy” to use, and another quarter described it as “somewhat easy.” Most tellingly, 99 percent of the teachers said they liked using the EGR program lesson plans. Overall and Relative FOI To establish a simple snapshot of implementation fidelity, the evaluation team established an implementation fidelity index with three sub-indices characterizing each surveyed school at midline along the nine program components within the three program areas. These scores were then bundled and categorized into low fidelity, partial fidelity, near-full fidelity, and full fidelity bins (Table 15). Figure 31 shows how each component feeds into a sub￾index for each program area, which in turn feeds upward into the overall index. Data for these measures were collected through 241 classroom observations, 235 teacher and HT surveys, 230 CL surveys, and 183 CS surveys. Fidelity Scale by Component This index is constructed by taking the findings of each of the components described above and creating a scale for each—using the planned targets as maximums or “full” fidelity—on which to score each of them. The scales used to give observed treatment schools a fidelity score for each component in this sub-index are detailed in Table 15. 1.Pace 2.Lesson performance Sub-Index 1: Curriculum Implementation •Teacher Textbook •Pupil workbook •Supplemental materials Sub-Index 2: Materials Provision 1.Reading instruction training 2.Observation and coaching 3.SBI 4.CS monitoring Sub-Index 3: Teacher Training, Coaching & Support Figure 31: Implementation Fidelity Index Structure 55 Table 15: Fidelity Sub-Index and Component Scoring Table Sub-Index Fidelity Scale Component Component Scale Curriculum Implementation Full = 5.5 to 6 Near-Full = 4.5 to 5 Partial = 3.5 to 4 Low = <3 Pace 3 = five or less days behind 2.5 = 6 to 10 days behind 2 = 11 to 15 days behind 1.5 = 16 to 20 days behind 1 = 21 to 25 days behind 0.5 = 26 to 30 days behind 0 = 31+ days behind Performance 3 = Teachers follow lesson plans ALL the time/ALL pupils participating 2.5 = All/Most or Most/All 2 = Most/Most or Some/All or All/Some 1.5 = Most/Some or Some/Most 1 = Some/Some 0.5 = None/Some or Some/None 0 = None/None Teacher Training, Coaching, and Support Full = 10 to 12 Near Full = 8 to 9.5 Partial = 6 to 7.5 Low = <6 Full = 8 to 9 Near Full = 6 to 7.5 Partial = 4.5 to 5.5 Low = <4.5 Teacher In-Training 3 = 9+ days 2.5 = 7 to 8 days 2 = five to 6 days 1.5 = 3 to 4 days 1 = 1 to 2 days 0 = 0 days Observation and Coaching of Teachers 3pts = Twice a month 2pts = Monthly 1pt = Less than monthly 0pt = none SBI Meetings At least once every 2 weeks (3) Once every three weeks (2.5) Once a month (2) Once every six weeks (1.5) Once every two months (1) Less than once every two months (0.5) None (0) CS Monitoring 3 = 6 or more annual visits 2.5 = 5 annual visits 2 = 4 annual visits 1.5 = 3 annual visits 1 = 2 annual visits 0.5 = 1 annual visit 0 = No visits Materials Provisions Full = 8 to 9 Near Full = 6 to 7.5 Partial = 4.5 to 5.5 Low = <4.5 Teacher Materials Teacher has teacher book/Teacher has copy of pupil book 3 = Yes/Yes 2 = Yes/No 1 = No/Yes 0 = No/No Supplemental Materials 0.5 points for each of 6 materials observed for a total of 3 points 3 = 100% have pupil textbooks 2.5 = 85% to <100% 2 = 65% to <85% 56 Sub-Index Fidelity Scale Component Component Scale Pupil Materials 1.5 = 35% to <65% 1 = 15% to <35% 0.5 = >0% to <15% 0 = 0% SBI coaching and supplemental materials scored the lowest of the nine components, whereas the teacher materials component has been implemented relatively according to plan (Figure 32). Figure 32: Average Implementation Fidelity Score by Component Fidelity Scale by Program Area Sub-Index The components of each program area sub-index were equally weighted, each earning a maximum of three points, and taken together produce the Sub-Index Fidelity Score. The scale for each component was constructed in the same way: anything less than 50 percent of target is considered low fidelity. The top half was then divided into equal thirds: full, near-full, and partial fidelity. Using this index, Figure 33 shows the percentage of program schools that were scored as full fidelity, near-full fidelity, partial fidelity, or low fidelity for each of the three program areas. 2 2.8 2 2.9 1.5 2.4 2.7 1 2.7 On-Track Component Lesson Performance Component Pupil Materials Component Teacher Materials Component Supplemental Materials Component Teacher In￾Training Component HT/CL Coaching Component SBI Coaching Component CS Monitoring Component 57 Figure 33: Sub-Index Distribution In implementation of the teacher training, coaching, and support components, just over one-third of the 271 schools observed were at full fidelity, and close to half were at near-full fidelity, together constituting over 80 percent of the sample. Close to one in five schools in the sample, however, were at partial or low fidelity. In materials provision, about 70 percent of program schools were at full or near-full fidelity. One in four program schools were at partial or low fidelity. In implementation of curriculum outputs, over 80 percent of program classrooms were at full or near-full fidelity. Almost one in five classrooms, however, were at partial or low fidelity. Overall Fidelity Index The overall fidelity index was constructed by combining the three sub-indices above into a single 27- point scoring scale (Table 16). Table 16: Fidelity Index Distribution Fidelity Category Point Value (Overall) Point value (Average per Component) Low Fidelity 13 points or less Less than 1.5 points per component Partial Fidelity 13.5 to 17.5 points 1.5-2 points per component Near-Full Fidelity 18 to 22 points 2-2.5 points per component Full Fidelity 22.5 points or more 2.5 points or more per component The scale was constructed in the same way as the sub-indices and components: anything less than 50 percent of target was considered low fidelity. The top half was then divided into equal thirds: full, near￾full, and partial fidelity. Figure 34 illustrates the distribution of observed treatment classrooms using the overall fidelity index. 15.50% 66.67% 14.73% 3.11% Curriculum Implementation Sub-Index 34.11% 48.06% 13.57% 4.27% Teacher Training, Coaching & Support Fidelity Sub-Index Full Fidelity Near-Full Fidelity Partial Fidelity Low Fidelity 15.12% 54.65% 23.26% 6.99% Materials Provision Sub-Index 58 Figure 34: Overall Implementation Fidelity Overall, over 80 percent of program schools were at full or near-full fidelity, with those in the full category constituting 17 percent of the sample. Fewer than 1 percent of schools fell into the low category. Conclusions The EGR program has not been fully implemented, which is not uncommon for a program only 10 months into implementation. In this light, the activity has made very good progress, with 80 percent of program schools in a state of full or near-full implementation fidelity on multiple components. Some of the challenges to implementation fidelity are part of larger-scale issues that the MOE is already working to address. Issues such as data quality and shifting enrollment figures affect materials distribution, and poor or undersupported district- and circuit-level management and supervision structures can make it difficult to ensure that SBI sessions are moving according to schedule. However, there is considerable room for the activity to improve implementation fidelity of all components, but particularly of SBI, curriculum pacing, pupil textbook distribution, and supplemental classroom materials. And even though these components have not been fully implemented, teacher performance and pupil outcomes are still quite strong. CONTEXT, RISKS, AND ASSUMPTIONS ANALYSIS Question 8: How have contextual factors affected program success? Program impacts can be bolstered, attenuated, or cancelled out by a program’s operating context or the veracity of assumptions underlying the ToC. The team examined the following contextual factors in program schools (citing data from comparison schools where relevant) and their change from baseline to midline: language match, use of the Ghanaian LOI, pupil and teacher absenteeism, and amount of reading instruction time. Aspects of the operating environment that are not likely to have changed since baseline, such as school assets, have been omitted. 16.67% 65.50% 17.05% 0.78% Full Fidelity Near-Full Fidelity Partial Fidelity Low Fidelity 59 Findings Alignment between Ghanaian LOI and Teacher and Pupil Language Abilities Both pupils and teachers in EGR program schools became more aligned with the Ghanaian LOI from baseline to midline, mostly shifting from the low-match category to the medium-match category (Figure 35)—a shift from being unable to speak the LOI to being able to speak it, even if it is not the language that is spoken at home or during play. Teachers’ access to scripted materials in the school LOI contributed to an increase in teacher confidence to teach in the local languages. 60 Figure 35: Pupil and Teacher Language Match with Ghanaian LOI, by Category The resulting figure from the baseline pupil and teacher language match scenario was an overall 55 percent high language match (both pupil and teacher have high language match), 22 percent medium language match (neither pupil nor teacher has low language match, and at least one has medium language match), and 23 percent low language match (either pupil or teacher has low language match). As might be expected given the individual language match figures, the status of overall language match in program schools at midline demonstrates an increased proportion of pupils with medium language match with their teachers (Figure 36). Thus, whether or not the change was caused by the program, the midline context was more conducive to fruitful Ghanaian language early reading instruction than was the case at baseline. 67% 19% 14% Pupil Language Match, BL 69% 20% 10% Pupil Language Match, ML High Match Medium Match Low Match 83% 9% 8% Teacher Language Match, BL 84% 14% 3% Teacher Language Match, ML High Match Medium Match Low Match 61 Figure 36: Overall Language Match Between Pupils, Teachers, and the Official Ghanaian LOI (Based on Proportion of Pupils) Baseline Endline 62 Materials and Practices Improve upon Existing Materials and Practices The ToC implicitly assumes that the program materials and practices will be adopted because they improve upon the baseline materials and practices. The data largely support this notion (Figure 37). Although program teachers mostly reported and were observed to use the official Ghanaian LOI in class at baseline, nearly all were observed to do so at midline. Figure 37: Percentage of Teachers Using Official Ghanaian Language of Instruction The availability and use of appropriate materials in the Ghanaian LOI rose from baseline to midline in program schools (Figure 38). Nearly all classrooms sampled at midline had textbooks/readers, teacher guides, and scripted lesson plans, and more than half had pupil workbooks in the Ghanaian LOI. Teachers nearly universally reported using the textbooks, teacher guides, and lesson plans daily at midline, compared with only 77, 53 and 22 percent respectively at baseline. The frequency of pupil workbook use remained unchanged relative to baseline, with around 70 percent of teachers using these daily. Figure 38: Percentage of Classrooms with Materials in the Ghanaian Language of Instruction Pupils Are Punctual and Attend Class Often Enough to Benefit from Structured Pedagogy The number of program pupils who at least sometimes missed school fell from 57.0 to 44.6 percent from baseline to midline, and the number who were tardy to school fell from 59.4 to 51.4 percent. Nevertheless, pupils in these schools had missed a statistically equivalent number of days on average in the past school week at midline compared with baseline, with an average of 0.56 days missed per week per pupil, signifying that the pupils who did miss school at least sometimes missed it for longer. The most common reasons for being sometimes or regularly absent or late to school stayed much the same from baseline to midline: illness, injury, or the need to do work or chores at home (Figures 39 and 40). 86% 91% 98% 95% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Observed During Lesson Teacher Reported Midline Baseline 53% 27% 77% 21% 100% 98% 99% 56% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Teacher's guide Scripted lesson plan Textbooks/readers Pupil workbooks Midline Baseline 63 Figure 39: Pupil Reasons for Being Sometimes or Regularly Absent (Self-Reported) Figure 40: Pupil Reasons for Being Tardy (Self-Reported) Teachers Teach Reading Lessons with Prescribed Frequency and Duration HTs reported at baseline that P1 pupils were supposed to receive 3.7 hours of reading instruction each week with an average lesson duration of about 55 minutes. Factoring in observed lesson durations, observed time on task, and reported frequency of absence by teachers and pupils at baseline, actual time spent on reading instruction was closer to 1.95 hours per week across P1 and P2 at baseline. Average observed lesson duration among program teachers fell from 43 minutes at baseline to 32 minutes at midline. As discussed in earlier sections, time spent on learning activities increased from an average 78 percent of the observed reading lesson at baseline to 85 percent at midline. Hence, the amount of reading instruction received per week declined from 2.24 hours to around 1.81 hours at midline, without factoring in pupil and teacher absence. On average, there was an increase in teacher absence from 0.23 days per week at baseline to 0.48 days per week at midline. As discussed earlier, pupil absence remained relatively stable at 0.56 hours per week at midline, compared with 0.61 hours per week at baseline. Hence, factoring the joint probability that either a given pupil or his or her teacher is absent at any point during the school week, the actual amount 1% 2% 3% 6% 13% 18% 67% 0.74% 2.55% 2.44% 2.78% 14.75% 18.80% 60.82% 0% 20% 40% 60% 80% 100% I want to play instead I am too tired to come It is too far away No transportation or money for transportation I have to help with household chores I have to do work for the family I was sick or hurt Midline Baseline 2% 4% 4% 8% 13% 24% 64% 0.67% 9.28% 3.76% 5.64% 12.40% 22.68% 65.23% 0% 20% 40% 60% 80% 100% The weather is bad The person who takes me to school is late in taking me I have to babysit younger siblings I have to do work for the family I have to walk a long time I wake up late I have to help with household chores Midline Baseline 64 of reading instruction received per week declined from 1.90 hours at baseline to around 1.47 hours at midline. The evaluation interviews did not provide an explanation for the greater frequency of teacher absences. The increase was likely the result of a generalized escalation in teacher absences from the beginning of the year to the end of the year. Conclusions Understanding the program context is essential for effective implementation at scale. The EGR program was implemented in the context of the following conditions:  Language match was not achieved in all schools.  Adherence to the Ghanaian LOI policy grew.  Pupil absence remained essentially the same.  Teacher absenteeism rose.  Overall reading instruction time was not at the expected level and even declined. The program achieved significant and sizeable impacts on P1 pupil performance despite the challenges in language match and the reduction in time spent on reading instruction, making the improvement in instruction quality even more evident. FACTORS THAT PREDICT SUCCESS Question 9: What program components and pupil- and school-level characteristics were most strongly associated with program success? Findings The evaluation team used regression analyses to uncover explanatory factors underlying pupil outcomes and observed treatment effects. Factors that Generally Predict Assessment Scores To determine which pupil- and school-level factors predicted assessment scores regardless of treatment status, the evaluation team combined all sampled pupils into a single regression model. This model additionally helped the team identify covariates to use in other regressions. Commonly predictive pupil-level variables included hunger, enthusiasm for reading, punishment for poor performance, gender, and pupil language match, and predictive school-level factors included official LOI, the percentage of female teachers, the adequacy of materials for teaching reading, and locality (rural versus urban). Although the magnitude of these effects and the extent to which they predicted outcomes varied across subtasks, the direction of their effects was mostly consistent. Some examples of regression coefficients on Ghanaian LOI subtasks are provided in Table 17. (The full regression output for this model—which was run on each of the five subtasks in both the English language and the Ghanaian LOI assessments, for 10 total iterations—can be viewed in Annex III.) 65 Table 17: Direction of Effect, Commonly Predictive Covariates on Assessment Scores Negative Positive  Pupil hunger (Being hungry “every day” associated with 1.3 word per minute decrease in ORF)  Punishment for poor performance (Associated with 1.2 word per minute decrease in ORF)  Low language match (Associated with 1.4 word per minute decrease in ORF)  Rural school (Associated with 1.2 letter per minute decrease in letter-sound identification)  Enthusiasm for reading (“Liking reading” associated with 1.2 word per minute increase in ORF)  Female pupil (Associated with 1.3 letter per minute increase in letter-sound identification)  Adequate materials for teaching reading (Associated with 3.9 word per minute increase in ORF)  Higher percentage of female teachers (School with 100% female teachers associated with 0.6 word per minute increase in nonword reading relative to 0% female teachers) Assessing Treatment Effects for Subgroups of Interest To distinguish the program’s effect on various subgroups, the team then added a series of interaction terms to difference-in-difference regression models. Program effects varied markedly depending on subtask and subgroup (Figures 41 and 42). For example, the program had a larger effect on pupils in urban schools than on pupils in rural schools. Except for oral reading fluency in the Ghanaian LOI, on which the program had a larger impact on girls than on boys, the program’s effect did not vary substantially according to pupil gender. In many cases, the program had a larger effect on pupils with medium language match (do not speak LOI at home but can speak it) than on pupils whose mother tongue was the LOI (high language match) or who do not speak the LOI. However, in English listening comprehension, Ghanaian oral reading fluency, and Ghanaian nonword reading, the low language match group had the largest effect size among the three—although the differences were mostly small. 66 Figure 41: Differential Program Effects on Ghanaian Subtask Scores, Demographics37 Figure 42: Differential Program Effects on Ghanaian Subtask Scores, Language Match 37 To enable comparison across subtasks scored in different units, normalized effect sizes are displayed in this and the following chart. The effect size is calculated by dividing the program effect for a subgroup by the standard deviation of the outcome variable for that same subgroup. For general reference, an effect size of 0.2 might be considered small, 0.4 might be considered medium, and 0.8 might be considered large. -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Oral Reading Fluency Listening Sounds Invented Words Reading Comprehension LIstening Comprehension Effect Size Rural Urban Male Female -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Oral Reading Fluency Listening Sounds Invented Words Reading Comprehension LIstening Comprehension Effect Size Rural Urban Male Female 67 Association Between Implementation Fidelity Components and Treatment Effects Using the same difference-in-differences approach, the evaluation team then tested the differential effect of the program on pupils in schools with varying implementation fidelity to the program design. This analysis can roughly be thought of as a dose-response to the program (the added value of implementing the full program, versus only some part), although there may be unobserved factors that explain both assessment scores and implementation fidelity that prevent these findings from being causal. Across nearly all subtasks and regardless of language of assessment, the team found increasing fidelity to the full program to be associated with a higher effect size (Figure 43). Although the program effects on English language assessment scores did not vary extremely between the near-full and full fidelity groups, the program effects on Ghanaian language assessment scores were higher in schools with full fidelity. On all Ghanaian language assessment scores aside from letter-sound identification, the program effect on low- and partial-fidelity groups was similar and frequently close to zero. Figure 43: Differential Program Effects on Ghanaian Subtask Scores, Implementation Fidelity To identify the program components most associated with high treatment effects, the team examined how treatment effects varied with near-full implementation or better (two points or more out of three) of each of the nine dimensions of the fidelity index (Figure 44). The teacher textbook dimension needed to be excluded, since all schools got at least two points out of three on that component. Otherwise, the only fidelity dimensions where near-full implementation or better was associated with higher outcomes on the letter-sound identification or oral reading fluency subtask were lesson performance, pupil workbook, and supplemental materials. Although the effect sizes for CS monitoring and SBI were similarly large, they were not statistically different from zero. However, it is not entirely helpful to determine which program component is most associated with treatment effects, as this dimension is more of an outcome than a program output. In other words, although the dimensions within the training, coaching, and monitoring sub-index all had a statistically insignificant association with the treatment effect, they could help determine how well teachers perform in their lesson. For this reason, this analysis is limited in its ability to state a causal relationship between each program component and program success. -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Oral Reading Fluency Listening Sounds Invented Words Reading Comprehension Listening Comprehension Effect Size Low FOI Partial FOI Near-Full FOI Full FOI 68 Figure 44: Program Component Association with Treatment Effects Factors that Predict High Program Fidelity The relationship between implementation fidelity and assessment scores cannot be interpreted causally with the data available. Schools effectively self-selected into their level of fidelity. Schools with full implementation fidelity may have had factors driving their fidelity, such as effective school management, teacher or administrator motivation, and school resource level, that also drove their higher assessment scores. Therefore, the team sought to uncover factors that can predict level of implementation fidelity and for which data were available. Given the positive relationship of implementation fidelity with assessment scores, these explanatory factors may help illuminate which kinds of schools are likely to implement a similar program faithfully. Outcome Index Positive Relationship with Overall Fidelity Index  Average level of pupil assets  Level of school resources  Number of dropouts Lesson Performance Index  Number of teachers at school  Average hours P1 pupils spend at school per day Pupil and Teacher Materials Index  Average level of pupil assets  Average hours P1 pupils spend at school per day Training and Monitoring Index  Proportion of pupils rewarded by teacher  Whether the teacher received a college education or higher  Number of dropouts -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 ORF Letter Sounds Pupil Workbook Sup. Materials Lesson Performance Reading Instructor Training SBI Obs. & Coaching CS Monitoring Lesson Pace 69 The evaluation team examined the overall fidelity score and its three subcomponents: how well schools adhere to and perform the designed curriculum, whether pupils and teachers possess the necessary school supplies and lesson manuals, and the level at which schools received the expected training sessions and monitoring visits. The model’s comparison for standard covariates were the average school-level frequency that pupils read per week, the percentage of pupils that report that their teacher rewards and punishes them, whether the HT reports that their school has enough materials to implement the curriculum, teacher gender, and the school-level ratio of female pupils. The analysis identified the level of pupil assets, which is measured using an aggregated index of various household assets, as the most robust predictor for implementation fidelity. A greater average level of pupil assets is positively correlated with overall program implementation and materials level. School resources have a positive association with overall implementation fidelity, even after controlling for average pupil assets. This finding suggests that pupil wealth and school wealth have distinct consequences for program implementation. The only consistent variable found to negatively correlate with implementation fidelity is the number of pupils who dropped out of school. Pupil dropouts were negatively related with the level of training and monitoring a school receives. Implementation fidelity did not vary with respect to gender, as neither the proportion of female pupils in a school nor the teacher’s gender is correlated with overall FOI and its subcomponents. An additional noteworthy finding is that classes with teachers that possess undergraduate degrees or higher have greater levels of training and monitoring adherence. Conclusions Implementing the EGR program at scale requires decision makers to understand what factors are most associated with program success. This analysis identified the following associations:  The factor with the strongest association, positive or negative, was the positive effect of providing adequate reading materials to pupils. This finding demonstrates that effective provision of this component can outweigh other mitigating factors that may not be within the program’s scope.  Some factors that predict success are out of reach of the current program scope. Pupil characteristics such as hunger, coming from a rural school, and not matching the school LOI relate to wider issues that need to be considered at the government level.  Full program implementation fidelity was the most significant predictor of overall success and effect size.  Adherence to prescribed lesson plans is associated with the most significant changes in effect size. This result is intuitive—schools where teachers followed the prescribed lesson plans most closely with the most pupils engaged had the highest-performing pupils.  Schools with certain characteristics—more resources, fewer school dropouts, more teachers, longer hours, and teachers with a college education or higher—were more successful in implementing the program fully. 70 SUMMARY CONCLUSIONS The EGR program yielded substantial gains in early reading skills among P1 pupils despite less than full implementation. The following tables chart the program’s progress toward its goals according to the program ToC. Table 18 shows performance across outputs, outcomes, and impact, and table 19 shows program risk assessments. Table 18: Program ToC/Logic Model Performance Summary Outputs Outcomes Goal Impact Pace: Progress on scripted lesson plans in relation to program-prescribed pace Lesson Performance: Teacher fidelity to steps in the lesson plan and pupil engagement with the lesson plan Teacher Textbook: Use of teacher textbook by teacher Pupil Workbook: Use of pupil workbook by teacher and pupils Supplemental Materials: Presence of materials such as teacher guide, alphabet chart, flash cards, Big Books, and conversation posters Reading Instruction and Training: Compliance with expectation for eight or more days of in-service training on reading instruction Observation and Coaching: Compliance with expectation for biweekly observation and coaching by HT and/or CL (Reported) SBI: Compliance with expectation for one hour-long SBI session per week (Reported) CS Monitoring: Compliance with expectation for monthly visits from CS to monitor implementation (Reported) Pupils are more likely to engage with books and learning materials, inside and outside classroom Teachers demonstrate mastery of phonics￾based approach to teaching reading in Ghanaian LOI Classroom time on task improves for pupils and teachers Teachers adapt instructional approach to pupils’ needs through continuous assessment Teachers adapt instructional approach based on SBI learnings, coaching, and monitoring Improved reading outcomes in Ghanaian language for P1 Improved pre￾reading outcomes in English for P1 ∅ Not achieved as planned √ Somewhat achieved + Achieved well Some of the biggest challenges to the ToC are in the risks and assumptions underlying the program. The evaluation found that many assumptions stated in the ToC were very prevalent risks (Table 19). The team assessed these risks’ likelihood of occurring and impact if they do occur. The categorization of likelihood was based on survey responses, and the categorization of impact was based on the data on associations. The team also added risks that were not part of the original model but need to be considered. 71 Table 19: Program ToC Risk Assessments Risks Likelihood Impact on Reading Scores Pupils miss class or are tardy to class Teachers miss class Pupils do not understand the LOI Teachers are not able to teach in the LOI Pupils have a low level of assets Schools have a low level of resources*(added) Teachers use punishment for poor performance*(added) High High Med Low High High High Low Low High Med High High High 72 RECOMMENDATIONS The following short- and medium-term recommendations are divided into two categories: recommendations that can support program improvement in its final year and recommendations for policymakers to consider while using these data for decision making. RECOMMENDATIONS TO SUPPORT PROGRAM IMPROVEMENT Review and improve the approach to low implementation fidelity areas—SBI trainings, assessment for adaptation, pacing, and materials provision: The EGR program should review its approach to areas with low fidelity, including SBI trainings and sessions, assessment for adaptation, pacing, and materials provision. It should investigate the management of these components to identify ways to better hold teachers accountable for meeting the program targets. The EGR program should review the current targets by, for example, discussing the lack of fidelity with teachers and EGR program management to understand if these components need a revised approach. Provide more targeted support to nonreaders: The EGR program is working primarily with pupils who are entering the program with zero scores in P1. At midline, the vast majority of these pupils are still zero scorers. The EGR program should further consider the reality that these zero score pupils are the norm and adjust their approach accordingly. The program should develop and use assessments and teaching strategies targeting pupils unable to cross the zero score threshold. Address school resource levels, pupil assets, and teacher use of punishment: The EGR program was designed assuming that school resource levels, pupil assets, and teacher use of punishment would not affect the program. However, the data show that these factors are important predictors of program outcomes. Although the EGR program will likely not be able to expand its scope, it should adjust its training to inform teachers of the associations between learning outcomes and school and pupil resource levels. This training could lead to a dialogue on potential mitigation strategies that are within the program’s scope. In addition, teacher training should promote positive methods of classroom discipline rather than punishment. Improve data quality for strong implementation: The MOE should continue its efforts to improve data quality for data that support program implementation, including data on school LOI, teacher language skills, enrollment, and school resources. Currently, the EMIS does not collect data on language background of teachers and pupils, which are necessary for the effective implementation of any LOI policy. To this end, the MOE should include teacher language and pupil language variables in the annual school census questionnaire. Additionally, the MOE should review the LOI for each school to ensure that the language reflects that of the school’s community. RECOMMENDATIONS FOR FUTURE EGR PROGRAMMING Review reading outcomes holistically to understand progress: Policymakers should review reading outcomes for low-, middle-, and high-performing pupils separately. Average scores on tests such as the EGRA are useful to track program progress. However, disaggregated data should be considered in equal measure to inform policy-level decision making. For example, data on the top 25 percent of pupils show average scores more in line with program targets and expectations, providing a window into what is possible at a mass scale. At the same time, looking into the reductions in zero scores allows policymakers to differentiate within a group that is often considered a single pool of nonreaders. Especially in P1, where pupils are just beginning their formal reading instruction, and coming from varied , where most pupils are just beginning their reading instruction, looking at this variability and the impact that a program has among different sets of pupils is essential. Integrate economic and health considerations into EGR policy design: Given the impact that a pupil’s household conditions can have on reading performance, MOE policymakers should continue to 73 work with implementers and partner government agencies to design programs that integrate economic or heath programming with education programming. This collaboration could entail working with implementers to integrate economic support into EGR programming or coordinating EGR programming with economic programming and support from other ministries. The school feeding program could improve many literacy outcomes, as the evaluation found a very high association between hunger and low pupil performance. Ensure that accountability and management systems are in place for program adoption: The EGR program succeeds most in a context where implementation fidelity is high. To bring this program to national scale and integrate it into government programming, the MOE and GES must have accountability and management systems in place to ensure that the program is on course. This step includes building on the EGR program’s fidelity tracking mechanisms as an essential component of program success. 74 ANNEXES 75 ANNEX 1. THEORY OF CHANGE Table 20: Program Theory of Change/Logic Model and Instrument Mapping – Training and Materials Inputs Assumptions Outputs Assumptions Outcomes Assumptions Goal Impact Training and Materials Training of MTs consisting of DTST in EGR and use of program-developed teaching and learning materials Teacher guides with scripted lessons in the 11 official Ghanaian languages Pupil books in the 11 Ghanaian languages Classroom kits including alphabet strips, flashcards, posters, and readers Development of ASER tool for continuous pupil assessment MTs successfully train HTs, CLs, and teachers on reading instructional model ● Program materials are designed, printed, and distributed in sufficient numbers and according to plan ● Teachers receive five days of initial in￾service training and termly refresher trainings on phonics￾based instruction from MTs ●● Teachers are provided with teacher guides with scripted lesson plans ●●● Pupils are provided with learning/activity books ●●● Classrooms are provided with supplementary materials and books ●●● Teachers are trained on continuous assessment using ASER ●● Training content is accurately and fully covered using a ToT/cascade model ●● Teachers understand, absorb, and retain the training content and are motivated to put it into practice ●●●● Teachers use scripted lesson plans as prescribed ●●● Pupils regularly access learning books and supplementary readers ●●● Materials are properly cared for and stored ●● Trained teachers do not transfer or leave teaching profession ● Teachers are able to teach in the language of scripted lesson plans in teacher guides ●● Teachers regularly assess pupils ●●● Teachers demonstrate mastery of phonics￾based approach to teaching reading in the Ghanaian LOI ●●●● Classroom time-on￾task improves for pupils and teachers ● Pupils are more likely to engage with books and learning materials, both inside and outside the classroom ●●●● Teachers adapt instructional approach to pupils’ needs through continuous assessment ●●●● Materials and practices are an improvement upon prior materials and practices ●● Class environment is conducive to absorbing lessons ●● Pupils are punctual and attend class with enough regularity to benefit from structured pedagogy ●●● Teachers teach reading lessons with prescribed frequency and duration ●● Pupils understand the LOI and materials ●●●● Classroom-level interventions can lead to improved reading regardless of home environment ● Improved reading outcomes in LOI for P1 and P2 pupils ● Legend ● Administrative data ● Head Teacher interview ● Teacher interview ● Pupil interview ● EGRA assessment ● CL interview ● CS interview ● Stallings Observation Tool ● Structured Observation 76 Table 21: Program Theory of Change/Logic Model and Instrument Mapping – Teacher Coaching and Supervision Inputs Assumptions Outputs Assumptions Outcomes Assumptions Goal Impact Coaching and Monitoring HT and CL receive training on instructional leadership and coaching from MTs Trained DTST￾LSs on providing targeted technical support Trained CSs on program monitoring Tablets and data platform for CS monitoring Facilitator guides for Head Teachers and CLs HTs and CLs are able and willing to observe and coach teachers and facilitate SBI meetings with prescribed frequency ●●● Teachers are able and willing to participate in SBI meetings with prescribed frequency ●●● CSs are able/willing and have sufficient resources to conduct monitoring visits with prescribed frequency ●●●● CSs follow monitoring protocols and record data effectively ●● CSs are effective in identifying gaps in instruction and coaching and are able to rally DTST LSs for additional coaching ● Teachers come to school often enough to be monitored and coached with prescribed frequency ●● HTs and/or CLs observe and coach teachers on material use on a bi-weekly basis ●●●● HTs and/or CLs facilitate school-based inset (SBI) meetings for one hour each week ●●●● CSs conduct biannual monitoring visits to monitor FOI of scripted lesson plans, coaching and SBI meetings ●●● CSs arrange DTST LS support visits (as needed) for weak HTs and CLs ●●● HTs, CLs, DTST LS, and CSs understand, absorb, and retain the training content and materials and are motivated to put them into practice ●●● HT, CL, and DTST LS coaching feedback is understood and useful to teachers ●● SBI meeting content is understood and useful to participants ●●●● Regular monitoring of teachers can improve performance by informing training needs and holding teachers accountable ● Monitoring data is reaching intended recipients and they are able and willing to act on it ●●● Teachers demonstrate mastery of phonics-based approach to teaching reading in the Ghanaian LOI and oral English ●●●● Classroom time￾on-task improves for pupils and teachers ● Teachers adapt instructional approach based on SBI learnings, coaching, and monitoring ●●●● Materials and practices are an improvement upon prior materials and practices ●● Class environment is conducive to absorbing lessons ●● Pupils are punctual and attend class with enough regularity to benefit from structured pedagogy ●●● Teachers teach reading lessons with prescribed frequency and duration ●● Pupils understand the LOI and materials ●●●● Classroom-level interventions can lead to improved reading regardless of home environment ● Improved reading outcomes in LOI for P1 and P2 pupils ● Legend ● Administrative data ● Head Teacher interview ● Teacher interview ● Pupil interview ● EGRA assessment ● CL interview ● CS interview ● Stallings Observation Tool ● Structured Observation 77 ANNEX II. SAMPLING PROCEDURES School Sample Selection Procedure The 100 districts to be targeted by the EGR program were purposively selected by Learning in collaboration with GES and MOE through the establishment of the Committee on Performing and Transitioning Districts (CPTD). The first stage of district selection involved classifying all 216 districts in Ghana into one of three categories: Performing, Transitioning, or Low Transitioning. This classification used district-level quantitative measures of education performance modeled after the Ghana Partnership for Education Grant (GPEG) project, which used similar criteria for classifying districts as deprived. The following performance indicators were selected and weighted by CPTD based on (a) the extent to which they predict BECE pass rates in a regression model and (b) qualitative consultation with stakeholders and committee members: Table 22: Performance Indicators Used by CPTD to Classify Districts as Performing, Transitioning, or Low Transitioning Indicator Description Weight 1. Primary Retention Rate Enrolment in P6 (2014)/Enrolment in P1 (2009) * 100 12% 2. % of Girls Enrolment P6 Girls Enrolment in P6/Total Enrolment in P6 * 100 12% 3. % of Trained Teachers in Primary Number of Trained Teachers in Primary/ Total Number of Teachers in Primary * 100 8% 4. % of Pupils having English Textbooks Number of Primary English Textbooks/Total primary Enrolment * 100 15% 5. % of Pupils having Maths Textbooks Number of Primary Maths Textbooks/Total primary Enrolment * 100 10% 6. % of Pupils having Writing Places Number of writing places in Primary/Total primary Enrolment * 100 6% 7. Primary Pupil-Teacher Ratio Total Primary Enrolment/Total Primary Teachers 9% 8. Average Pass Rate in BECE English Total number of Pupils with Average Pass in English/Total Number of English Candidates 14% 9. Average Pass Rate in BECE Maths Total number of Pupils with Average Pass in Maths/Total Number of Maths Candidates 14% Each of these nine indicators was then broken into four bins and each bin was assigned a score point ranging from 1 to 4. For example, primary retention was scored as 1 if the rate was 59.9 percent and below, 2 if the rate was 60-74.9 percent, 3 if the rate was 75-89.9 percent, and 4 if the rate was 90 percent and above. Districts were all assigned a score point for each indicator, and then given a total score (ranging from 1 to 4) based on the weights specified in Table 2223. Districts were then categorized as performing, transitioning, and low transitioning in accordance with Table 233. 78 Table 23: District Classification Using CTPD Method38 District Weighted Average Score (DWAS) District Classification Number of Districts Total Schools DWAS < 2.0 Low Transitioning 51 3,537 2.0 < = DWAS < 3.0 Transitioning 104 7,064 DWAS > = 3.0 Performing 61 4,075 Initially, all 165 districts classified as Performing and Transitioning were selected to receive the EGR intervention. The rationale for targeting districts with higher education performance is that schools would be functional and sufficiently resourced to be able to make the best use of program inputs and activities, thereby ensuring high FOI and increased likelihood of program success. However, as part of an effort to refocus the USAID/Ghana Learning activity, the number of districts was reduced from 165 to 100 in order to better the balance the breadth of the intervention against the depth of program inputs and activities. As outlined in Table 244, the newly selected 100 intervention districts are comprised of 31 districts from the original 61 Performing districts and 69 districts from the original 104 Transitioning districts: Table 24: Selection of the 100 Districts to Receive the EGR Program Region # of Districts (Learning 165) Performing Transitioning Grand Total Ashanti 29 8 12 20 Brong Ahafo 19 7 7 14 Central 20 3 12 14 Eastern 24 4 3 7 Greater Accra 16 2 3 5 Northern 9 0 5 5 Upper East 6 0 2 2 Upper West 5 1 3 4 Volta 19 1 15 16 Western 18 6 7 13 Grand Total 165 31 69 100 These districts were narrowed down based exclusively on GES and district authorities’ perceived degree of match between the LOI, the language spoken by teachers, and the mother tongue of pupils, as defined as the percentage of schools within a district that have “high match.” As such, the evaluation team exacts match the treatment and comparison sample on the LOI reported in the annual school census. In addition, the degree of language match between the official Ghanaian LOI, teacher language, and pupil mother tongue will be extensively mapped as part of the evaluation baseline, and balance between treatment and comparison schools on language match will play an important role in selecting the final sample of 470 schools to be included in the endline assessment. The sampling frame of treatment schools consists of schools within the 100 districts that have been selected to receive the EGR program. Given the wide performance gaps between the 51 Low Transitioning districts and 165 Transitioning and Performing districts, the evaluation team has opted to restrict the sampling frame of comparison schools to those within the 65 districts that are classified as Transitioning 38 In several cases, districts were classified into one of the three categories using metrics other than District Weighted Average School (DWAS). For example, all non-GPEG districts were qualified under either Performing or Transitioning classification regardless of DWAS. In addition, all GPEG districts with additional support from UNICEF supported by USAID through the Learning Support Activity were removed from the Transitioning classification. 79 and Performing but are not receiving the EGR intervention. Using these criteria, 7,044 public primary schools form the potential sampling frame of treatment schools whereas 4,060 public primary schools form the potential sampling frame of comparison schools.39 The large number of schools in the sampling frame make this evaluation a strong candidate for CEM, a robust and relatively new statistical matching technique through which balance between the treatment and comparison groups is established ex ante using variable cutpoints or bins rather than discovered through repeated propensity score re-estimation which can inadvertently create imbalance and model dependence.40 CEM allows for matching of treatment and comparison units on many different variables (rather than through a single composite propensity score) which is especially useful in cases where a large number of important predictor variables are available. Variables to be used in the CEM model include those which predict treatment status and/or the primary outcome of interest—performance on the EGRA. The first stage of variable selection involved requesting from EMIS a series of purposively selected variables from the 2014/15 Annual School Census, a 17-page HT survey of all Basic Schools in Ghana which collects in-depth data on school infrastructure, management, materials and equipment, textbook access, enrollments and attendance, dropouts and repetition, teacher qualifications and subject areas, and movement of teachers and learners. To ensure some measure of pupil performance is a factor in selection process, the team included average BECE scores at the circuit level— the lowest administrative level through which JHS and primary schools could be mapped to one another.41 The evaluation team first classified schools as treatment or non-treatment based on the district in which they are located and then ran regression models to determine which variables significantly predict treatment status. The team then merged the EMIS and circuit-level BECE data to those schools which were sampled for the 2015 EGRA assessment and ran regression models to determine which variables significantly predicted EGRA performance. In all, 11 variables were robust and significant predictors of treatment status and/or EGRA performance and will therefore be used in the CEM algorithm. The description of these 11 variables can be found in Table 25 below. 39 We excluded schools with Nzema as their language of instructions from the sampling frame because there are no schools with that language found among comparison schools. 40 King, G., & Nielsen, R. (2016). Why propensity scores should not be used for matching. Copy at http://j. mp/1sexgVw Download Citation BibTex Tagged XML Download Paper, 378. 41 On average, there are about 9 to 10 schools within each identified school circuit. We calculated aggregated BECE scores across 4 core subjects from BECE: Science, Social Science, Mathematics and English. 80 Table 25: Variables Selected for CEM Based on Ability to Predict Treatment and/or EGRA Variables42 Type Predicted EGRA Performance43 Predicted Treatment Status44 LOI Indicator Yes (+/-) Yes (+/-) Multilingual Schools Indicator Yes (+/-) Yes (-) Schools in Urban Localities Indicator Yes (+) Yes (+) Vehicle Access Indicator Marginally (-) Yes (+) School in Top 20% of Income Generation Indicator Yes (+) Yes (-) % of Female Primary Teachers Continuous Yes (+) Yes (+) Ratio of Writing Place to Pupils Continuous No Yes (+) Absence Rate (P1and P2) Continuous Yes (-) Yes (-) Primary Pupil Teacher Ratio Continuous Yes (-) Yes (-) % of Female Pupils (P1 to P6) Continuous Marginally (+) Yes (-) Circuit-Level BECE Scores Continuous Yes (+) Yes (+) While external validity of the study would be maximized by randomly drawing a stratified sample of 260 schools from the sampling frame of 7,044 treatment schools, it is important to note that the CEM approach requires pruning of both treatment and comparison units in order to ensure balance. Therefore, the team first narrowed the sampling frame to CEM-matched treatment and comparison schools by coarsening the continuous variables into five equal-sized bins and then exact matching on the indicator variables and coarsened continuous variables described in Table 25. This specification produced a matched sample of 7,247 schools, with 4,523 treatment schools matched to 2,724 comparison schools. From this matched sample, the team then drew a random sample of 260 treatment schools, stratified by language so that the number of treatment schools selected in each language group is proportionately represented in the final sample (see Table 26). The team then randomly dropped excess comparison schools in each sampled treatment strata so that 260 comparison schools are exactly matched with treatment schools. 42 Variables with missing values were imputed beforehand using a predictive mean matching approach drawing from 10 nearest neighbors for each missing value found across the variables. Also, 20 different imputations were specified for each missing value. Indicator variables included in the model did not have any missing values. 43 Ordinary Least Squares (OLS) regressions were estimated individually for seven student-level EGRA subtasks on the same imputed dataset of school level independent variables. The relationship matrix presented in this column denotes their overall significance across the seven different scores. Standard errors are clustered at the school level. Variables that predict EGRA at the margin are defined as having p values < 0.1. 44 A logistic regression with robust standard errors was estimated using the same imputed dataset as the preceding estimation to predict treatment status. 81 Table 26: Distribution of Sample, by Language Group Language Comparison Treatment Total Share of Sample (%) Akwapem 32 32 64 12% Asante 110 110 220 42% Dagbani 17 17 34 7% Dangbe 9 9 18 3% Ewe 28 28 56 11% Fante 34 34 68 13% Ga 10 10 20 4% Gonja 2 2 4 1% Kasem 2 2 4 1% Other 5 5 10 2% Dagaare 9 9 18 3% Missing Languages 2 2 4 1% Total 260 260 520 100% The team concluded by analyzing the final sample in terms of external validity and balance. In terms of external validity, the evaluation first checked for district representation and found that of the 165 districts, 144 districts are represented in the final sample—85 of the 100 treatment districts (85 percent) and 59 of the 65 comparison districts (91 percent). The team then conducted a one sample t-test to check differences between the overall sampling frame and final selected sample across a series of school-level indicators. Table 27: External Validity Check of Final Sample Variables45 Population46 Selected Sample Obs Mean Std. Dev. Obs Mean Std. Dev. T-stat Urban Localities 11,291 0.29 0.45 520 0.22 0.41 3.86 Multi Lingual Schools 11,291 0.01 0.11 520 0.00 0.06 3.28 Vehicle Access 11,291 0.91 0.28 520 0.95 0.22 3.76 Functional Electricity 11,291 0.47 0.50 520 0.49 0.50 0.89 Community Electricity 11,291 0.75 0.43 520 0.76 0.43 0.29 Water Access 11,291 0.42 0.49 520 0.45 0.50 1.29 Library 11,291 0.18 0.38 520 0.19 0.39 0.68 School Management Committee 11,291 0.83 0.37 520 0.86 0.35 1.56 CS Visit 11,291 0.90 0.30 520 0.93 0.26 2.02 School in Top 20% of Income Generation 11,291 0.27 0.44 520 0.28 0.45 0.77 45 The evaluation only included school level indicators for which the team has full information (no imputed values). 46 Population parameters included Nzema schools. 82 Full Capitation Grant 11,291 0.50 0.50 520 0.51 0.50 0.40 Average T-statistic 1.71 Unique Districts 165 143 The team observed some statistically significant differences between the selected sample and the broader sampling frame, such as the over-sampling of schools who were visited by their CS twice or more a term (93 versus 90 percent) and schools with vehicle access (95 versus 91 percent) as well as the under￾sampling of schools from urban localities (22 versus 29 percent) and multilingual schools (1 versus 0.4 percent). However, given that balance was observed on the majority of indicators and the magnitude of observed imbalances is not large, the evaluation team believes threats to the study’s external validity are thus far minimal. To examine the balance between treatment and comparison schools in the final sample, the team calculated the average absolute standardized differences for all of the purposively selected variables considered in the original predictor models (regardless of whether or not they were significant predictors of treatment or EGRA). Several variables exhibited perfect balance because they were indicator variables that were exact matched via the CEM matching process. Only one variable seemed to exhibit an imbalance with a higher absolute standardized difference compared to the rest of the final sample’s variable—the proportion of teachers who passed the training certification. While the team is not concerned about this imbalance because the variable only marginally significantly predicts EGRA scores,47 the team will again check for balance during baseline as well as comparison for the variable during endline, as appropriate. Overall, with an average standardized difference of 0.038 standard deviations across all variables, the final sample exhibits strong balance between treatment and comparison units. Table 28: Final Sample Balance Checks Variables48 Average Absolute Standardized Difference (Cohen’s d) Multilingual School 0.000 Urban Localities 0.000 Vehicle Access 0.000 School with Functional Electricity 0.092 School with Community Electricity 0.009 Water Access 0.008 Library 0.010 School Management Committee 0.077 CS Visit 0.015 Top 20% Income 0.000 English Textbook to Pupil 0.043 Math Textbook to Pupils Ratio 0.039 Ghanaian Language Textbook to Pupils Ratio 0.047 Capitation Grant 0.023 47 Moreover, in a separate logistic regression to predict treatment status in the final sample, the proportion of primary teachers who passed the training certification does not significantly predicting treatment status. 48 The team included a full list of variables that ought to be controlled for future analysis. The additional variables were earlier confirmed not to significantly predict treatment status in the imputed dataset. 83 Writing Place Ratio 0.074 Seating Place Ratio 0.043 % of Prim. Teachers Passed Training 0.124 % of Prim. Teachers with Degree 0.076 % of Female Prim Teachers 0.016 Absence Rate 0.028 % of Female Primary Pupils 0.011 % of PupilsP1 pupils with Preschool 0.078 Primary Pupil to Teacher Ratio 0.084 Circuit Level BECE Scores 0.029 Sample Average 0.038 Balance was observed in the baseline sample, which allows for more robust comparisons across survey waves. Table 29: Balance Between Treatment and Comparison on Proportion of Zero Scores EGRA Subtest Comparison Mean Treatment Mean P-Value Ghanaian Language Listening Comprehension (%) 34.223 30.270 0.146 Letter Sounds (clpm) 0.482 0.393 0.002 Non-Word Reading (cwpm) 0.847 0.822 0.140 Oral Reading Fluency (cwpm) 0.852 0.816 0.054 Reading Comprehension (%) 96.054 94.354 0.049 English Listening Comprehension (%) 73.769 76.108 0.366 Letter Sounds (clpm) 0.435 0.358 0.006 Non-Word Reading (cwpm) 0.817 0.787 0.119 Oral Reading Fluency (cwpm) 0.499 0.465 0.245 Reading Comprehension (%) 92.302 92.833 0.685 Table 30: Balance Between Treatment and Comparison EGRA Scores EGRA Subtest Comparison Mean Treatment Mean P-Value Ghanaian Language Listening Comprehension (%) 37.434 40.518 0.103 Letter Sounds (clpm) 7.190 8.659 0.030 Non-Word Reading (cwpm) 1.500 2.006 0.072 Oral Reading Fluency (cwpm) 1.392 2.227 0.095 Reading Comprehension (%) 1.234 1.633 0.153 English Listening Comprehension (%) 13.096 12.691 0.798 Letter Sounds (clpm) 9.482 11.011 0.052 Non-Word Reading (cwpm) 1.812 2.174 0.148 Oral Reading Fluency (cwpm) 5.894 6.453 0.446 Reading Comprehension (%) 2.914 2.671 0.669 84 Table 31: Balance Between Treatment and Comparison on Other Variables that Predict Assessment Performance in Similar Contexts, Baseline Variable Compariso n Mean Treatmen t Mean P-Value School Functional electricity at school (%) 66.383 54.468 0.008 Library at school (%) 14.894 17.872 0.384 Classroom Textbooks available in local language (%) 73.022 76.744 0.257 Workbooks available in local language (%) 17.347 19.650 0.405 Sufficient desk space for all pupils (%) 61.462 65.058 0.346 Class size 31.351 29.413 0.125 Teacher Completed Teacher Training College (%) 80.473 81.467 0.725 Years of experience 10.287 9.969 0.630 In-Service training in EGR (%) 51.186 47.104 0.257 Absenteeism rate (%) 8.498 4.942 0.010 Pupil Attended preschool or kindergarten (%) 90.885 91.244 0.722 Arrives at school hungry (%) 52.891 52.282 0.798 Read to at home on a regular basis (%) 17.981 18.084 0.946 Once the 470 schools were selected among the 520 schools sampled for baseline, they served as a panel throughout the life of the evaluation, and the same teachers will be tracked from baseline to endline. Given the challenges in tracking pupils across multiple school years, however, we selected a random cross￾section of learners during each wave of data collection to complete the pupil interview and assessments. Statistical Power Analysis to Inform Sample Size The sample size for the purpose of data collection is a function of, among other variables, the MDES for pupil-level reading outcomes. To inform MDES assumptions, Evaluating Systems consulted a September 2016 policy brief published by 3ie49 which features a meta-analysis of 21 IEs of structured pedagogy programs in 11 countries, including Liberia, Kenya, and Uganda. The brief defines structured pedagogy programs as those which typically include the “development of evidence-based curricula and instructional approaches, along with lesson plans and training for teachers in delivering new content and material for pupils.” In addition, such programs can include “regular monitoring and mentoring of, and feedback to, teachers on their delivery of the new material.” This meta-analysis reported the standardized mean difference (Cohen’s d) at 0.23 with a lower bound of 0.13 and an upper bound of 0.34 for language arts (including measures of reading, writing, and literacy). To increase the likelihood of the study being able to detect impacts, we assume an MDES of 0.2 which is slightly below the mean standardized effect size. Because it is not logistically possible to draw a simple random sample of pupils, we will employ a two￾stage clustered design whereby schools (clusters) are selected in the first stage and pupils are selected in the second stage. While we will select classrooms in cases where there is more than one class per grade, the majority of schools have only one class per grade (77 percent for P2 according to 2015 Ghana EGRA data) thus we do not factor in classroom-level clustering in our power analysis. Two-stage clustered sampling requires analysis of the intra-cluster correlation, or ρ, which is the ratio of variability in outcomes between clusters to the total variability in outcomes among the broader sample. We calculate ρ for each of the 2015 EGRA subtasks for Ghanaian language as well as pre-reading subtasks for English using Stata’s loneway command. The results of this analysis are reported in Table 32 below. 49 Snilstveit, B., Stevenson, J., Menon, R., Phillips, D., Gallagher, E., Geleen, M., & Jimenez, E. (2016). The impact of education programmes on learning and school participation in low-and middle-income countries. Retrieved from http://www.3ieimpact.org/media/filer_public/2016/09/20/srs7-education-report.pdf. 85 Table 32: Intra-Cluster Correlations for EGRA Subtasks in Ghanaian Language and English ICC (ρ) EGRA Subtask Ghanaian Language English Listening comprehension 0.40 0.37 Letter-sound identification 0.39 0.41 Non-word reading 0.20 --- Oral passage reading 0.26 --- Reading comprehension 0.24 --- To maximize the study’s power to detect impacts, the team adopted the most conservative ρ value of 0.41 for letter-sound identification in English. In addition, the team assumed an R-squared value of 0.12 which represents the percentage of variation in outcomes of interest that is explained by school-level covariate(s).50 Adopting standard assumptions of 0.8 for statistical power (β) and a partial Bonferroni￾adjusted value of 0.0251 for significance level (α), the team uses Optimal Design to plot power against number of subjects per cluster (n) for a given number of clusters (J) in order to explore the gains in power achieved for each additional pupil that is sampled. 50 R-squared was calculated by (1) regressing 2015 EGRA subtask scores on school-level predictors (e.g., mean learner age, mean preschool participation, learner absenteeism, girls enrollment ratio, pupil-teacher ratio, teacher language match, availability of library books, capitation, etc.), and (2) selecting the mean R-squared value among the regression models for the seven subtasks. R-squared values ranged from 0.047 for reading comprehension to 0.22 for listening comprehension (both in Ghanaian language). 51 Bonferroni correction is applied by dividing the standard α value of 0.05 by the number of hypotheses being tested (we use the number of local language EGRA subtasks) while factoring in the mean correlation between these subtasks (0.47). Bonferroni adjustment is designed to correct for the increased likelihood of false positives when multiple hypotheses are being tested. For more information, see http://www.quantitativeskills.com/sisa/calculations/bonhlp.htm. 86 Figure 45: Power versus Number of Subjects per Cluster Figure 45 shows power versus number of pupils per cluster for a fixed sample of 450 schools. While the curve displayed in Figure 45 moves along the y-axis if the number of clusters (J) is changed, the shape of the curve remains relatively constant, showing rapidly diminishing returns to sampling additional pupils beyond the inflection point and only marginal power gains for sampling above 10 (due primarily to the high rate of ICC among Ghanaian learners). As such, the team adopted a cluster size of 10, which represents the number of pupils to be sampled per grade, per school. To ensure results can be disaggregated by gender, the team sampled five boys and five girls per grade. To determine the number of schools to be sampled, the team then plotted MDES against the total number of clusters (treatment and comparison combined) and determined the number of clusters at which an effect size of 0.2 is achieved. 87 Figure 46: MDES versus Total Number of Clusters As shown in Figure 46, 428 schools (214 treatment and 214 comparison) are required to ensure the study is powered according to the parameters outlined above. However, given that an estimated five percent of teachers in Ghana leave their postings each year,52 it was necessary to inflate the sample by 42 schools (10 percent) to account for this non-compliance (470 schools total). Using the above sample estimates, the team reported the MDES for P253 learners for each Ghanaian language subtask in Cohen’s d form as well as reported the minimum detectable change and percent change in actual EGRA subtask scores: 52 Baah, Y., Otoo, K. & Osei-Boateng, C. (2009). Teacher Attrition in Ghana: Results of a Questionnaire Survey, 2009. Retrieved from http://download.ei-ie.org/Docs/WebDepot/GNAT_2010_TEACHER_ATTRITION__REPORT.pdf. 53 The national EGRA assessment in Ghana targets P2 pupils only. 88 Table 33: Minimum Detectable Change by EGRA Subtasks - Ghanaian Language Ghanaian Language EGRA Subtask Baseline (2015)54 Standard Deviation R-Squared MDES (Cohen’s d) Detectable Change Percent Change Listening comprehension 44.95% 34.89 0.22 0.19 6.63 pp 14.75% Letter-sound identification 9.67 clpm 14.53 0.14 0.19 2.76 clpm 28.54% Non-word reading 2.36 cwpm 6.93 0.06 0.16 1.11 cwpm 47.03% Oral passage reading 3.62 cwpm 10.18 0.05 0.17 1.73 cwpm 47.79% Reading comprehension 3.49% 12.86 0.05 0.17 2.19 pp 62.75% Table 34: Minimum Detectable Change by EGRA Subtasks - English Language English Language EGRA Subtask Baseline (2015)13 Standard Deviation R-Squared MDES (Cohen’s d) Detectable Change Percent Change Listening comprehension 12.20% 25.81 0.16 0.19 4.90 pp 40.16% Letter-sound identification 9.17 clpm 13.97 0.17 0.19 2.65 clpm 28.89% 54 Unweighted scores. 89 While the IE primarily aims to determine if the program positively impacts pupil reading achievement, the team carefully assessed intermediate outcomes such as those at the teacher-level. In addition to serving as a gauge of fidelity of implementation, the team aimed to link teacher-level data to pupil performance during endline analysis which allows for exploration of mediating factors of program impact. To calculate detectable change on illustrative teacher-level outcomes, the team used data from Learning’s School-Based Baseline Study Report – Components 1 & 2 (dated April 24, 2016), which reports the percentage of teachers who incorporated the core reading skills outlined in Table 35 at any point during an observed reading lesson. Table 35: Minimum Detectable Change for Teacher Practices Key Reading Skill Description Baseline Detectable Change Letter sounds Teaching letter sounds and not just letter names 21.4% 10 pp Passage/story reading Teachers or pupils reading a passage (connected sentences, paragraphs, stories) 27.4% 11 pp Reading comprehension Pupils answering questions about sentences, stories or passages that they have read 20.8% 10 pp Dictation Pupils writing letters, words, sentences or passages dictated by the teacher 2.1% 5 pp Assuming β of 0.8, a Bonferroni-adjusted α value of 0.02, ρ of 0.2, a cluster size of 2 teachers55 per school, and 214 schools per study arm, the team used Stata’s clustersampsi to calculate the binomial detectable difference on whether or not teachers focus on the core reading practices outlined above. These minimum detectable differences—expressed percentage point increases—are reported in the “Detectable Change” column of Table 35. Sample Pruning Procedure The 100 treatment districts were selected exclusively based on perceived degree of language match between pupils, teachers, and the official Ghanaian LOI. However, at the time of baseline, no school-level data was available on language match in either treatment or non-treatment districts for use in the CEM algorithm. To help reduce potential selection bias, the team therefore opted to oversample schools by approximately ten percent (50 additional schools, or 520 total) at baseline so that the final sample of 470 schools could be limited to those with maximum baseline balance on both language match and EGRA scores. To identify which schools would be dropped from the sample, the team employed three different strategies and then chose the strategy which resulted in the lowest imbalance between treatment and comparison schools. These three strategies are as follows: 1. Using the mean school-level composite percentage score of all EGRA subtests combined, calculate the distance between all treatment schools and all comparison schools within the same language group and then drop treatment and comparison 55 According to the 2015 EGRA study, 77 percent of schools sampled have only one P2 teacher. The evaluation team therefore assumes that one teacher per grade level will be observed for P1 and P2 (2 teachers total). 90 pairs with the greatest distance in terms of composite EGRA scores. 2. Using school-level EGRA scores and language match category frequencies, generate propensity scores to predict selection into treatment then drop treatment/comparison school pairs (matched on language group) with the greatest distance from one another in terms of propensity scores. 3. Calculate distance between treatment and comparison schools on the proportion of “low match” pupils in the school then drop school pairs (matched on language) with the largest distance (i.e., comparison schools with a large number of “low match” pupils matched to treatment schools with very few “low match” pupils). Of the three strategies, strategy three yielded the greatest balance overall and was thus used to determine which schools (46 of 50) were dropped from the sample. The team dropped the two treatment schools with missing EGRA data and as well as comparison matches from the same language group with the highest proportion of low match pupils (four schools total). Finally, because one sampled treatment school could not be enumerated due to school closure, the team retained the treatment school tagged for dropping which had the highest proportion of low match learners, thus realizing a final sample of 470 schools. Pre￾and post-pruning balance tables are presented below. Table 36: Pre- and Post-Pruning Balance on EGRA Scores Pre-Pruning Post-Pruning EGRA Subtest Units Difference56 P-value Difference P-value Ghanaian LanguageListening Comprehension percent 4.955 0.007 3.084 0.103 Letter Sounds clpm 1.957 0.006 1.469 0.030 Non-Word Reading cwpm 0.656 0.019 0.506 0.072 ORF cwpm 2.107 0.006 0.835 0.095 Reading Comprehension percent 0.620 0.024 0.399 0.153 English Listening Comprehension percent -0.973 0.522 -0.404 0.798 Letter Sounds clpm 1.877 0.018 1.529 0.052 Non-Word Reading cwpm 0.430 0.088 0.362 0.148 ORF cwpm 1.685 0.088 0.559 0.446 Reading Comprehension percent -0.545 0.347 -0.243 0.669 With the full sample of 520 schools, half of EGRA subtests were imbalanced at p<0.02, as shown in Table 36. After dropping 50 schools using strategy 3, however, all EGRA subtests were balanced between treatment and comparison, albeit some differences are significant at the margin. Pruning 50 schools yielded similar results for language match (Table 36). While two-thirds of the language match categories were imbalanced at p<0.05 with the full original sample, only one category was significantly different after pruning. Strategies for dealing with this remaining imbalance are discussed in the Findings section of this report, however overall the strategy of oversampling and pruning yielded considerable improvements in terms of baseline balance. Table 37: Pre- and Post-Pruning Balance on Language Match Pre-Pruning Post-Pruning 56 Coefficient on treatment status. 91 EGRA Subtest Difference57 P-value Difference P-value Overa ll High match proportion 0.124 0.001 0.059 0.140 Medium match proportion 0.018 0.448 -0.001 0.970 Low match proportion -0.142 0.000 -0.058 0.151 Pupil High match proportion 0.089 0.009 0.032 0.374 Medium match proportion 0.019 0.295 0.022 0.252 Low match proportion -0.108 0.001 -0.054 0.128 Teach er High match proportion 0.135 0.000 0.080 0.037 Medium match proportion -0.041 0.094 -0.046 0.070 Low match proportion -0.094 0.004 -0.034 0.291 57 Coefficient on treatment status. 92 ANNEX III. FULL IMPACT ANALYSIS REGRESSION OUTPUTS 93 Figure 47: MDES versus Total Number of Clusters Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 1.587** 0.016 0.983* 0.074 0.109 0.540 0.251 0.363 0.887 0.159 0.980 0.230 0.00731 0.569 0.00154 0.919 0.002 0.395 -0.000577 0.793 Midline 4.037*** 0.000 1.199*** 0.002 0.115 0.429 -0.127 0.367 -0.216 0.356 0.136 0.383 -0.00717 0.537 0.0253* 0.076 0.000 0.904 -0.00284* 0.069 Midline * Treatment 13.09*** 0.000 18.54*** 0.000 2.935*** 0.000 4.043*** 0.000 0.897 0.239 5.168*** 0.000 -0.0247 0.101 0.0617*** 0.002 0.004 0.278 0.0472*** 0.000 Baseline: Pupil Hardly Read -1.190 0.334 -1.898 0.111 -0.535 0.194 -0.660 0.147 -1.829*** 0.008 -0.667 0.384 -0.0427* 0.053 -0.0784*** 0.002 -0.0108** 0.023 0.00232 0.761 Baseline: Pupil Misses School Regularly -1.942 0.583 1.118 0.780 0.390 0.714 0.895 0.500 0.522 0.783 1.408 0.567 -0.0467 0.285 -0.0218 0.714 -0.010 0.283 0.0211 0.297 Pupil Hungry Everyday -1.557*** 0.000 -1.441*** 0.000 -0.473*** 0.000 -0.524*** 0.001 -1.160*** 0.000 -0.891*** 0.006 -0.00522 0.524 -0.0208** 0.042 -0.00444** 0.027 -0.00514** 0.011 Baseline: Pupil Read Books Recently 5.279*** 0.001 4.610*** 0.003 1.376*** 0.009 1.757*** 0.005 3.590*** 0.001 1.935 0.154 0.0169 0.501 0.0168 0.576 0.00962** 0.037 0.00990 0.281 Baseline: Pupil Likes Reading Books 4.365** 0.013 2.859 0.148 0.678 0.360 1.064 0.225 2.557** 0.047 1.310 0.398 -0.0222 0.581 0.0589* 0.098 -0.001 0.808 8.90e-05 0.994 Baseline: Pupil Punished by Teacher -1.143 0.420 -0.583 0.692 -0.463 0.346 -0.701 0.163 -1.320* 0.080 -0.925 0.246 -0.0563** 0.019 -0.0629** 0.024 -0.0115** 0.017 -0.000136 0.981 Baseline: Pupil Rewarded by Teacher -1.519 0.341 -1.446 0.399 -0.318 0.548 -0.198 0.734 -0.648 0.586 0.0321 0.972 -0.0230 0.412 -0.0209 0.538 -0.001 0.817 0.00430 0.577 Baseline: Teacher Education (Base = High School or Less) Diploma or Equivalent 1.515** 0.018 1.518** 0.014 0.532*** 0.003 0.168 0.673 -0.174 0.855 -0.826 0.529 -0.000546 0.962 -0.00771 0.577 0.00229 0.151 0.00262 0.354 Undergraduate Degree 3.039*** 0.000 2.566*** 0.002 0.968*** 0.000 0.548 0.223 0.524 0.600 -0.558 0.676 0.00594 0.669 -0.00549 0.733 0.00512* 0.060 0.00342 0.380 Postgraduate Degree -0.679 0.638 -0.919 0.553 0.569 0.265 0.0630 0.930 0.321 0.806 -1.365 0.379 -0.00382 0.904 -0.0879* 0.085 0.005 0.305 -0.00450 0.425 Baseline: Teacher Belief - "Sound Out" (Base = Strongly Agree) Agree 0.183 0.777 0.114 0.853 0.162 0.470 0.00330 0.991 -0.313 0.590 -0.622 0.410 -0.0193* 0.089 0.000542 0.962 -0.002 0.340 0.00415 0.213 Neither Agree or Disagree 0.0735 0.936 -0.150 0.874 0.192 0.555 0.274 0.513 0.385 0.503 -0.248 0.718 -0.0100 0.584 -0.0926*** 0.006 0.003 0.199 -3.04e-05 0.994 Disagree -1.352* 0.089 -0.674 0.441 -0.476* 0.096 -0.673** 0.047 -1.112* 0.080 -1.127 0.121 -0.00247 0.907 -0.0199 0.392 -0.00471* 0.061 -0.00550 0.121 Strongly Disagree 1.537 0.565 0.447 0.844 -0.0463 0.959 -0.655 0.386 -0.0935 0.943 -0.417 0.755 -0.00826 0.819 -0.0475** 0.049 0.011 0.433 -0.00380 0.631 Baseline: Teacher Belief - Teaching English First (Base = Strongly Agree) Agree -0.600 0.472 -1.174 0.165 -0.227 0.463 -0.0937 0.797 0.352 0.553 0.441 0.538 -0.00661 0.687 -0.0286 0.103 0.00246 0.309 -0.00661 0.113 Neither Agree or Disagree 0.794 0.512 0.608 0.634 -0.307 0.494 -0.260 0.513 0.716 0.394 0.0311 0.964 0.0220 0.448 -0.0283 0.372 0.00728 0.221 -0.00587 0.349 Disagree -1.013 0.210 -1.557* 0.054 -0.686*** 0.009 -0.847*** 0.004 -0.221 0.614 -0.442 0.267 -0.000130 0.994 -0.0214 0.187 0.00404 0.163 -0.00594 0.127 Strongly Disagree -0.00710 0.994 -0.216 0.841 -0.0552 0.872 -0.0868 0.810 -0.479 0.339 -0.0771 0.885 0.0137 0.429 -0.00670 0.740 0.000766 0.788 -0.00156 0.797 Pupil's Proficiency in Ghanaian LOI -0.404 0.243 -0.473 0.148 0.0134 0.915 -0.0358 0.752 0.329 0.113 0.0863 0.654 -0.000223 0.970 -0.0830*** 0.000 0.00244 0.200 -0.000113 0.949 Length of School Day (Hours) 0.383 0.261 0.236 0.452 0.199* 0.070 0.108 0.440 0.0930 0.738 -0.0738 0.826 0.00499 0.443 0.00116 0.845 0.00175 0.174 0.00130 0.467 Rural -1.391* 0.064 -1.252 0.149 -0.635** 0.021 -0.588* 0.056 -1.081*** 0.009 -0.501 0.282 -0.0409*** 0.005 -0.0171 0.211 -0.00916*** 0.004 -0.00683 0.104 Pupil Repeated Grade -1.863*** 0.003 -1.478** 0.018 -0.430* 0.065 -0.419 0.130 -0.714* 0.096 -0.660 0.181 -0.0158 0.165 -0.0115 0.420 -0.00107 0.719 0.00226 0.626 Pupil Age (Imputed) 0.0257 0.836 0.0158 0.900 -0.0195 0.692 -0.00587 0.911 0.0779 0.517 0.127 0.364 0.00104 0.676 0.0169*** 0.000 0.000966 0.145 0.000516 0.546 Female Pupil 1.755*** 0.000 1.485*** 0.000 0.339*** 0.003 0.305** 0.014 0.287 0.104 0.445*** 0.008 -0.00176 0.692 -0.0144* 0.051 0.00124 0.370 0.00168 0.323 Official Language of Instruction (Base = Akuapim Twi) Dagaare -8.433*** 0.000 -8.714*** 0.000 -1.454*** 0.008 -1.680*** 0.007 -2.766*** 0.005 -2.070* 0.054 -0.00112 0.980 -0.0901 0.202 -0.00551 0.369 -0.00246 0.745 Dangme -0.784 0.745 -2.880 0.183 -0.272 0.723 -0.890 0.180 0.686 0.672 1.001 0.255 0.104* 0.052 -0.0756*** 0.004 0.0140 0.341 0.0125 0.147 Ewe -2.100 0.175 -4.626*** 0.006 -0.363 0.435 -0.654 0.208 -0.674 0.381 1.018 0.214 0.0219 0.247 0.108*** 0.000 7.44e-05 0.986 0.0130 0.104 Fante -0.862 0.534 -3.700** 0.024 -0.569 0.185 -0.196 0.665 -0.601 0.356 -1.025* 0.082 -0.0122 0.471 0.184*** 0.000 -0.000190 0.965 -0.00750 0.193 Ga 0.270 0.873 -5.022*** 0.005 0.297 0.682 -0.511 0.473 1.011 0.344 -0.565 0.558 0.180*** 0.000 -0.0520* 0.074 0.00434 0.528 -0.0110 0.149 Gonja -3.858 0.273 -6.527** 0.031 0.222 0.849 -0.796 0.415 0.0547 0.986 -1.917 0.126 0.195* 0.091 0.137* 0.061 0.00282 0.764 -0.0104 0.407 Gurene -3.002* 0.052 -5.698*** 0.004 -0.281 0.586 -0.363 0.507 -1.446 0.167 -0.778 0.507 -0.00248 0.928 -0.163*** 0.000 -0.00153 0.824 0.00965 0.220 Kasem -13.18*** 0.000 -17.35*** 0.000 -2.661*** 0.000 -3.628*** 0.000 -2.921** 0.044 -5.230*** 0.000 -0.0362 0.500 -0.173*** 0.000 -0.00737 0.537 -0.0193** 0.026 Baseline: Percent of Class Activities on Task -4.090 0.387 -3.069 0.475 -2.165 0.160 -2.876* 0.081 -4.284 0.111 -3.214 0.268 -0.222* 0.059 -0.00663 0.924 -0.0599** 0.034 -0.0284 0.263 Pupil Household Asset Index 1.564*** 0.000 1.543*** 0.000 0.517*** 0.000 0.474*** 0.001 0.542** 0.015 0.401 0.152 0.0153** 0.024 0.0207*** 0.007 0.00438** 0.033 0.00623*** 0.008 School Resources Index 0.109 0.725 0.167 0.617 0.00127 0.990 -0.0139 0.895 0.0815 0.587 0.155 0.336 -0.00196 0.676 -0.00291 0.639 -0.000250 0.805 0.00180 0.219 Observations R-squared 8,854 0.119 8,854 0.190 8,849 0.119 8,854 0.059 8,853 0.084 8,848 Standard errors are clustered at the school level (*** p<0.01, ** p<0.05, * p<0.1) Predictors 8,849 0.348 8,845 0.403 Letter Sounds (CLPM) 0.118 English Ghanaian English Ghanaian 8,854 0.079 8,854 0.041 Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian 94 Figure 48: OLS Impact Analysis Model (Female Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 1.896** 0.019 1.092* 0.088 0.245 0.244 0.364 0.280 1.181** 0.026 0.882 0.181 0.0110 0.431 0.0123 0.489 0.00236 0.352 -0.00102 0.000 Midline 4.145*** 0.000 1.254** 0.012 0.266 0.241 -0.00205 0.992 0.209 0.512 0.227 0.299 -0.00153 0.904 0.0258 0.111 0.00184 0.565 -0.00203 0.699 Midline * Treatment 13.62*** 0.000 19.67*** 0.000 3.089*** 0.000 4.275*** 0.000 0.334 0.632 5.938*** 0.000 -0.0343** 0.042 0.0539** 0.029 0.00217 0.638 0.0522*** 0.341 Baseline: Pupil Hardly Read -0.146 0.925 -0.797 0.558 -0.440 0.417 -0.352 0.557 -1.518** 0.019 -0.155 0.852 -0.0375* 0.082 -0.0637** 0.046 -0.00710 0.139 0.00353 0.686 Baseline: Pupil Misses School Regularly -0.969 0.812 2.840 0.494 1.088 0.387 1.489 0.304 1.646 0.389 3.011 0.193 -0.0652 0.150 -0.0739 0.251 0.00287 0.792 0.0332 0.106 Pupil Hungry Everyday -1.889*** 0.001 -1.696*** 0.001 -0.534*** 0.008 -0.626*** 0.004 -1.138*** 0.002 -0.936*** 0.003 0.00350 0.743 -0.0318** 0.025 -0.00310 0.194 -0.00662*** 0.007 Baseline: Pupil Read Books Recently 5.401*** 0.005 4.663** 0.015 1.284* 0.069 1.724** 0.033 3.419*** 0.001 1.550 0.222 0.000800 0.976 0.0307 0.431 0.00815 0.142 0.00923 0.370 Baseline: Pupil Likes Reading Books 5.371** 0.013 4.193* 0.079 1.361 0.155 1.490 0.160 2.439** 0.044 1.769 0.263 -0.00177 0.965 0.0967** 0.021 -0.00428 0.508 0.00573 0.661 Baseline: Pupil Punished by Teacher -1.958 0.257 -1.416 0.412 -0.676 0.287 -1.028 0.106 -1.521* 0.062 -1.201 0.171 -0.0560** 0.025 -0.0785** 0.017 -0.00821 0.118 -0.00631 0.409 Baseline: Pupil Rewarded by Teacher -2.488 0.250 -2.983 0.173 -1.104 0.168 -0.761 0.383 -0.507 0.635 -1.219 0.283 -0.0292 0.353 0.0168 0.692 -0.00429 0.569 -0.00700 0.491 Baseline: Teacher Education (Base = High School or Less) Diploma or Equivalent 1.649** 0.043 1.723** 0.021 0.613*** 0.008 0.303 0.547 0.0782 0.915 -0.518 0.612 -0.00368 0.742 0.00362 0.829 0.00564*** 0.007 -1.20e-05 0.997 Undergraduate Degree 3.727*** 0.000 3.598*** 0.001 1.159*** 0.001 0.700 0.230 0.586 0.467 -0.0719 0.947 0.00514 0.704 0.00247 0.898 0.00599** 0.038 0.00274 0.540 Postgraduate Degree 0.0516 0.978 -1.137 0.555 0.785 0.273 -0.235 0.765 1.355 0.281 -1.222 0.377 0.0304 0.564 -0.0813* 0.094 0.0156*** 0.004 -0.00495 0.441 Baseline: Teacher Belief - "Sound Out" (Base = Strongly Agree) Agree 0.534 0.505 0.333 0.655 0.252 0.364 0.145 0.687 -0.178 0.720 -0.510 0.426 -0.0265** 0.020 -0.00481 0.736 -0.00285 0.268 0.000639 0.870 Neither Agree or Disagree -0.320 0.783 0.0302 0.977 0.284 0.442 0.279 0.515 0.595 0.196 -0.00799 0.990 -0.0142 0.410 -0.0787* 0.063 0.00221 0.431 -0.00101 0.836 Disagree -2.237** 0.020 -1.234 0.244 -0.740** 0.046 -0.895** 0.036 -0.868 0.163 -1.142 0.106 -0.00147 0.946 -0.0242 0.361 -0.00446 0.173 -0.00971** 0.037 Strongly Disagree 0.636 0.839 0.203 0.949 0.132 0.918 -0.517 0.604 0.497 0.792 -0.0213 0.990 0.0117 0.766 -0.0645** 0.032 0.0202 0.371 -0.00326 0.775 Baseline: Teacher Belief - Teaching English First (Base = Strongly Agree) Agree -0.762 0.445 -1.316 0.172 -0.172 0.665 -0.157 0.731 0.242 0.679 0.118 0.859 -0.00404 0.824 -0.0204 0.343 0.00190 0.498 -0.00817* 0.100 Neither Agree or Disagree 0.683 0.631 0.770 0.639 -0.370 0.510 -0.342 0.503 -0.148 0.838 -0.335 0.677 0.0188 0.593 -0.0439 0.255 0.00107 0.794 -0.00775 0.308 Disagree -1.165 0.245 -1.542* 0.100 -0.704** 0.045 -0.940** 0.012 -0.330 0.520 -0.492 0.319 0.00220 0.899 -0.0277 0.169 0.00564 0.104 -0.00547 0.275 Strongly Disagree -0.408 0.741 -0.0627 0.961 -0.0714 0.876 -0.149 0.751 -0.478 0.440 -0.422 0.502 0.00639 0.738 -0.0154 0.519 -0.00115 0.767 -0.00747 0.307 Pupil's Proficiency in Ghanaian LOI -0.346 0.464 -0.221 0.598 -0.00375 0.985 -0.0158 0.930 0.632* 0.077 0.406 0.170 0.00198 0.775 -0.0799*** 0.000 0.00211 0.453 0.00309 0.158 Length of School Day (Hours) 0.627 0.108 0.478 0.176 0.302** 0.034 0.147 0.399 0.187 0.440 0.142 0.638 0.00376 0.543 0.0102 0.177 0.00181 0.306 0.00203 0.394 Rural -1.215 0.194 -0.890 0.398 -0.950** 0.012 -0.694* 0.068 -1.509*** 0.002 -0.674 0.200 -0.0516*** 0.000 0.00464 0.784 -0.00895** 0.016 -0.00814 0.133 Pupil Repeated Grade -3.084*** 0.000 -2.774*** 0.001 -0.623* 0.054 -0.656* 0.067 -1.213** 0.011 -1.016** 0.028 -0.0253* 0.086 -0.0212 0.246 -0.00131 0.761 -0.00235 0.643 Pupil Age (Imputed) -0.175 0.306 -0.164 0.358 -0.0756 0.279 -0.0698 0.353 0.0529 0.671 -0.0532 0.618 -7.50e-05 0.978 0.0125*** 0.003 0.00182* 0.059 0.000899 0.392 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -4.063*** 0.004 -6.428*** 0.000 -0.940** 0.043 -0.876* 0.060 -1.664** 0.019 -0.918 0.125 0.00470 0.767 -0.0287 0.193 -0.00818 0.117 0.00231 0.712 Dagaare -9.323*** 0.000 -10.14*** 0.000 -1.932*** 0.005 -2.194*** 0.002 -3.252*** 0.007 -2.805*** 0.007 0.00847 0.864 -0.0678 0.384 -0.00914 0.278 -0.00788 0.382 Dagbani -7.660*** 0.000 -9.567*** 0.000 -1.000 0.140 -1.455** 0.035 -2.222* 0.097 -1.599 0.320 0.0421 0.399 -0.0570 0.203 0.00535 0.713 0.00260 0.882 Dangme -2.316 0.369 -3.773 0.141 -1.015 0.370 -1.450 0.134 0.401 0.852 0.631 0.587 0.0790* 0.066 -0.119*** 0.000 0.0137 0.465 0.00386 0.732 Ewe -1.087 0.574 -3.647* 0.061 -0.295 0.615 -0.518 0.427 -0.315 0.731 1.891** 0.036 0.0275 0.181 0.107*** 0.001 0.000623 0.925 0.0244** 0.021 Fante -0.832 0.644 -3.745* 0.057 -0.772 0.185 -0.467 0.406 -1.124 0.173 -1.401* 0.052 -0.00312 0.873 0.187*** 0.000 -0.00399 0.499 -0.00726 0.319 Ga 0.814 0.683 -4.437** 0.022 0.334 0.674 -0.503 0.514 0.404 0.726 -0.553 0.584 0.159*** 0.000 -0.0539* 0.098 0.00350 0.636 -0.0163** 0.020 Gonja -4.950 0.123 -6.977** 0.021 0.810 0.546 -1.148 0.322 1.951 0.636 -1.422 0.282 0.265** 0.026 0.135 0.113 0.0102 0.493 -0.00117 0.940 Gurene -3.348* 0.072 -5.893*** 0.006 -0.341 0.569 -0.414 0.536 -1.997* 0.096 -1.164 0.350 -0.0229 0.415 -0.180*** 0.000 -0.00396 0.629 0.00789 0.419 Kasem -13.50*** 0.000 -18.87*** 0.000 -3.212*** 0.000 -4.009*** 0.000 -3.177* 0.061 -6.692*** 0.000 -0.00566 0.934 -0.207*** 0.000 0.00489 0.772 -0.0352*** 0.002 Baseline: Percent of Class Activities on Task -0.228 0.963 -0.580 0.900 -1.348 0.418 -1.481 0.402 -3.943 0.161 -3.058 0.338 -0.137 0.217 -0.0737 0.411 -0.0567** 0.036 -0.0299 0.381 Pupil Household Asset Index 1.089** 0.014 1.127*** 0.005 0.313 0.101 0.308 0.122 0.380 0.138 0.319 0.315 0.0147* 0.063 0.0157 0.110 0.00307 0.263 0.00682** 0.019 School Resources Index 0.367 0.330 0.416 0.278 0.108 0.420 0.0897 0.522 0.345** 0.043 0.295 0.135 -0.00246 0.624 0.00142 0.840 -0.000464 0.686 0.00185 0.329 Observations R-squared 4,410 Standard errors are clustered at the school level (*** p<0.01, ** p<0.05, * p<0.1) 4,410 0.086 4,409 0.126 4,410 0.121 Predictors English Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian 4,407 4,406 4,406 0.357 0.427 0.130 4,410 0.189 0.039 4,410 0.095 4,407 0.124 95 Figure 49: OLS Impact Analysis Model (Male Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 1.324* 0.062 0.911 0.125 -0.00350 0.987 0.167 0.529 0.641 0.413 1.098 0.263 0.00495 0.737 -0.00891 0.636 0.00155 0.615 0.000239 0.928 Midline 3.990*** 0.000 1.210*** 0.007 -0.0131 0.942 -0.229 0.154 -0.586* 0.058 0.0723 0.663 -0.0119 0.411 0.0253 0.178 -0.00204 0.407 -0.00343* 0.063 Midline * Treatment 12.51*** 0.000 17.39*** 0.000 2.769*** 0.000 3.799*** 0.000 1.396 0.158 4.410*** 0.001 -0.0172 0.366 0.0695*** 0.007 0.00582 0.208 0.0422*** 0.000 Baseline: Pupil Hardly Read -2.111* 0.086 -2.816** 0.030 -0.587 0.170 -0.909** 0.046 -2.075** 0.027 -1.075 0.191 -0.0492* 0.067 -0.0923*** 0.002 -0.0150** 0.013 0.00130 0.880 Baseline: Pupil Misses School Regularly -2.629 0.483 -0.234 0.957 -0.156 0.887 0.440 0.750 -0.393 0.856 0.247 0.929 -0.0289 0.590 0.0375 0.623 -0.0230* 0.070 0.0119 0.617 Pupil Hungry Everyday -1.202** 0.032 -1.197** 0.018 -0.440** 0.019 -0.438** 0.041 -1.201*** 0.001 -0.844** 0.031 -0.0145 0.169 -0.00636 0.620 -0.00636** 0.021 -0.00325 0.189 Baseline: Pupil Read Books Recently 5.127*** 0.001 4.503*** 0.005 1.451*** 0.004 1.770*** 0.004 3.716*** 0.008 2.271 0.160 0.0336 0.259 0.00287 0.936 0.0112* 0.066 0.0106 0.298 Baseline: Pupil Likes Reading Books 3.506** 0.040 1.706 0.378 0.135 0.850 0.784 0.370 2.743* 0.089 1.047 0.554 -0.0431 0.332 0.0137 0.730 0.00212 0.771 -0.00487 0.695 Baseline: Pupil Punished by Teacher -0.556 0.693 0.0154 0.992 -0.330 0.506 -0.438 0.419 -1.236 0.167 -0.718 0.412 -0.0565** 0.036 -0.0488 0.111 -0.0150** 0.024 0.00626 0.285 Baseline: Pupil Rewarded by Teacher -0.566 0.700 -0.0256 0.988 0.447 0.417 0.360 0.573 -0.724 0.693 1.243 0.243 -0.0130 0.663 -0.0548 0.161 0.00225 0.711 0.0152* 0.098 Baseline: Teacher Education (Base = High School or Less) Diploma or Equivalent 1.401** 0.044 1.323* 0.051 0.458** 0.026 0.0512 0.883 -0.425 0.722 -1.138 0.486 0.000974 0.941 -0.0206 0.208 -0.000805 0.669 0.00492 0.157 Undergraduate Degree 2.510*** 0.004 1.687** 0.042 0.833*** 0.005 0.458 0.254 0.544 0.662 -0.951 0.556 0.00501 0.760 -0.0143 0.461 0.00441 0.216 0.00398 0.388 Postgraduate Degree -1.256 0.473 -0.815 0.643 0.392 0.584 0.292 0.747 -0.335 0.830 -1.434 0.461 -0.0341 0.245 -0.0953* 0.090 -0.00421 0.547 -0.00353 0.675 Baseline: Teacher Belief - "Sound Out" (Base = Strongly Agree) Agree -0.0512 0.934 0.00871 0.989 0.103 0.653 -0.0979 0.724 -0.443 0.535 -0.694 0.445 -0.0118 0.352 0.00488 0.723 -0.00202 0.512 0.00774** 0.036 Neither Agree or Disagree 0.451 0.658 -0.294 0.772 0.119 0.731 0.305 0.494 0.224 0.798 -0.473 0.586 -0.00841 0.737 -0.109*** 0.004 0.00433 0.246 0.00130 0.772 Disagree -0.479 0.634 -0.156 0.886 -0.229 0.503 -0.449 0.249 -1.363* 0.077 -1.097 0.206 -0.00127 0.956 -0.0167 0.540 -0.00518 0.133 -0.000881 0.845 Strongly Disagree 3.051 0.327 1.025 0.630 -0.303 0.400 -0.846** 0.046 -1.090 0.101 -0.876 0.253 -0.0371 0.316 -0.0157 0.505 -0.00341 0.438 -0.00342 0.480 Baseline: Teacher Belief - Teaching English First (Base = Strongly Agree) Agree -0.507 0.547 -1.114 0.210 -0.313 0.301 -0.0681 0.845 0.431 0.518 0.682 0.409 -0.00978 0.559 -0.0363* 0.077 0.00305 0.326 -0.00562 0.266 Neither Agree or Disagree 0.696 0.605 0.297 0.835 -0.342 0.512 -0.251 0.632 1.399 0.242 0.314 0.690 0.0243 0.392 -0.0148 0.647 0.0125 0.158 -0.00428 0.529 Disagree -0.951 0.246 -1.654* 0.058 -0.706** 0.013 -0.799** 0.013 -0.167 0.733 -0.458 0.268 -0.00207 0.906 -0.0167 0.386 0.00260 0.424 -0.00673 0.136 Strongly Disagree 0.370 0.699 -0.389 0.709 -0.0809 0.825 -0.0539 0.892 -0.535 0.296 0.254 0.651 0.0198 0.275 0.00368 0.874 0.00229 0.548 0.00406 0.520 Pupil's Proficiency in Ghanaian LOI -0.404 0.284 -0.649* 0.097 0.0444 0.745 -0.0375 0.787 0.0802 0.742 -0.198 0.303 -0.00113 0.889 -0.0855*** 0.000 0.00300 0.129 -0.00304 0.190 Length of School Day (Hours) 0.153 0.699 0.0201 0.955 0.110 0.354 0.0808 0.557 -0.00754 0.983 -0.275 0.494 0.00553 0.484 -0.00825 0.309 0.00169 0.247 0.000695 0.681 Rural -1.625** 0.033 -1.657* 0.051 -0.332 0.252 -0.478 0.174 -0.625 0.235 -0.313 0.539 -0.0310* 0.071 -0.0396** 0.021 -0.00907** 0.018 -0.00585 0.193 Pupil Repeated Grade -0.691 0.434 -0.187 0.841 -0.261 0.413 -0.198 0.610 -0.305 0.640 -0.342 0.633 -0.00749 0.619 0.00218 0.917 -0.00108 0.788 0.00642 0.364 Pupil Age (Imputed) 0.226 0.158 0.187 0.248 0.0312 0.646 0.0557 0.471 0.0892 0.591 0.285 0.223 0.00196 0.558 0.0210*** 0.000 0.000226 0.808 0.000128 0.910 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -3.672*** 0.004 -6.594*** 0.000 -0.919** 0.035 -0.911* 0.066 -0.536 0.437 -0.690 0.265 0.0159 0.320 0.0139 0.572 -0.00201 0.651 -0.000815 0.885 Dagaare -7.851*** 0.000 -7.621*** 0.002 -1.080* 0.061 -1.277* 0.069 -2.366** 0.016 -1.399 0.268 -0.0112 0.788 -0.110 0.141 -0.00242 0.720 0.00152 0.864 Dagbani -4.765* 0.063 -6.927*** 0.006 -0.536 0.508 -0.777 0.361 -0.524 0.729 -0.758 0.613 0.0661 0.304 -0.0766* 0.084 0.0144 0.381 -0.00286 0.777 Dangme 0.662 0.797 -2.065 0.306 0.489 0.438 -0.345 0.584 0.890 0.514 1.415 0.137 0.133* 0.054 -0.0318 0.354 0.0144 0.227 0.0231* 0.094 Ewe -2.956** 0.044 -5.435*** 0.001 -0.400 0.439 -0.762 0.192 -0.951 0.247 0.277 0.763 0.0170 0.419 0.109*** 0.001 -4.97e-05 0.991 0.00217 0.783 Fante -0.987 0.469 -3.753** 0.018 -0.391 0.444 0.0422 0.945 -0.158 0.838 -0.687 0.330 -0.0212 0.252 0.178*** 0.000 0.00334 0.652 -0.00779 0.192 Ga -0.176 0.921 -5.536*** 0.003 0.310 0.703 -0.489 0.563 1.551 0.215 -0.455 0.687 0.200*** 0.000 -0.0494 0.207 0.00484 0.641 -0.00625 0.567 Gonja -2.797 0.544 -6.161* 0.079 -0.394 0.720 -0.468 0.654 -1.836 0.379 -2.362 0.102 0.124 0.264 0.139* 0.053 -0.00451 0.536 -0.0194 0.127 Gurene -2.624 0.111 -5.509*** 0.006 -0.203 0.743 -0.286 0.662 -0.920 0.424 -0.372 0.764 0.0156 0.637 -0.151*** 0.000 0.00147 0.861 0.0113 0.161 Kasem -13.09*** 0.000 -16.32*** 0.000 -2.274*** 0.008 -3.394*** 0.000 -2.682* 0.063 -4.129*** 0.005 -0.0692 0.175 -0.143*** 0.001 -0.0179* 0.067 -0.00784 0.448 Baseline: Percent of Class Activities on Task -7.344 0.146 -5.198 0.256 -2.981* 0.086 -4.168** 0.024 -4.631 0.110 -3.425 0.243 -0.300** 0.021 0.0571 0.476 -0.0627** 0.046 -0.0289 0.210 Pupil Household Asset Index 1.990*** 0.000 1.920*** 0.000 0.703*** 0.000 0.620*** 0.000 0.717** 0.015 0.451 0.144 0.0161* 0.050 0.0234** 0.019 0.00586** 0.042 0.00482* 0.083 School Resources Index -0.136 0.690 -0.0535 0.887 -0.1000 0.329 -0.108 0.350 -0.158 0.398 0.0414 0.809 -0.00151 0.780 -0.00612 0.423 -7.64e-05 0.951 0.00204 0.201 Observations R-squared Standard errors are clustered at the school level (*** p<0.01, ** p<0.05, * p<0.1) English Ghanaian English Ghanaian 4,442 4,439 4,442 4,442 4,444 4,444 4,444 4,444 0.343 0.382 0.114 Predictors English Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian 4,444 4,444 0.118 0.049 0.061 0.129 0.202 0.055 0.074 96 Figure 50: Tobit Impact Analysis Model Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 2.872*** 0.008 2.577** 0.013 1.676 0.190 1.352 0.417 2.570* 0.079 6.688 0.122 -0.00191 0.977 -0.0117 0.661 0.0608 0.409 0.0389 0.615 Midline 8.763*** 0.000 3.998*** 0.000 1.073 0.261 -1.082 0.376 -1.044 0.168 1.117 0.582 -0.0681 0.264 0.0450* 0.077 0.0326 0.647 -0.0487 0.460 Midline * Treatment 13.02*** 0.000 21.67*** 0.000 10.82*** 0.000 18.00*** 0.000 1.522 0.287 25.80*** 0.000 -0.167** 0.048 0.0890** 0.011 -0.00255 0.979 0.539*** 0.000 Baseline: Pupil Hardly Read -1.562 0.357 -3.929** 0.035 -3.342 0.139 -4.619* 0.072 -5.837*** 0.003 -3.556 0.314 - - - - - - - - Baseline: Pupil Misses School Regularly -3.986 0.380 0.997 0.852 7.343 0.176 7.040 0.247 1.213 0.817 12.55 0.234 - - - - - - - - Pupil Hungry Everyday -2.428*** 0.000 -2.944*** 0.000 -2.725*** 0.001 -3.421*** 0.002 -3.349*** 0.000 -6.224*** 0.007 -0.0269 0.544 -0.0308 0.118 -0.114* 0.073 -0.113** 0.030 Baseline: Pupil Read Books Recently 7.128*** 0.000 8.042*** 0.000 10.52*** 0.000 11.17*** 0.000 8.873*** 0.001 13.23** 0.030 - - - - - - - - Baseline: Pupil Likes Reading Books 6.444*** 0.006 4.699 0.113 2.235 0.572 4.186 0.353 10.67** 0.010 6.285 0.434 - - - - - - - - Baseline: Pupil Punished by Teacher -1.767 0.355 -1.070 0.638 -3.675 0.166 -4.153 0.151 -3.976* 0.065 -7.817* 0.077 - - - - - - - - Baseline: Pupil Rewarded by Teacher -1.697 0.429 -2.178 0.396 0.247 0.927 -0.753 0.803 -1.613 0.566 -0.668 0.888 - - - - - - - - Baseline: Teacher Education (Base = High School or Less) Diploma or Equivalent 1.996** 0.031 2.152** 0.042 2.673** 0.018 2.270 0.153 1.178 0.453 -1.212 0.765 0.00344 0.962 -0.0165 0.534 0.104 0.171 0.0411 0.455 Undergraduate 4.145*** 0.000 4.149*** 0.001 4.703*** 0.001 4.640*** 0.006 2.652* 0.095 0.968 0.802 0.0291 0.699 -0.00946 0.755 0.131 0.111 0.0519 0.414 Postgraduate Degree -0.797 0.660 -2.038 0.405 1.540 0.456 1.766 0.511 1.156 0.589 -7.395 0.305 -0.0183 0.897 -0.189 0.181 0.149 0.165 -0.141 0.290 Baseline: Teacher Belief - "Sound Out" (Base = Strongly Agree) Agree 0.340 0.687 0.473 0.603 0.651 0.530 0.418 0.733 0.648 0.509 -1.442 0.544 -0.0930 0.132 0.00185 0.931 -0.0454 0.460 0.0592 0.218 Neither Agree or Disagree 0.480 0.724 0.342 0.875 -3.923* 0.099 -1.359 0.615 3.694* 0.065 -4.730 0.313 -0.0480 0.724 -0.198*** 0.004 0.205 0.127 -2.206*** 0.000 Disagree -1.377 0.227 0.458 0.742 -0.862 0.631 -0.493 0.798 -0.359 0.827 -5.819 0.109 -0.0441 0.706 -0.00934 0.815 -0.129 0.207 -0.0557 0.433 Strongly Disagree 2.076 0.516 1.857 0.539 1.216 0.723 0.0185 0.996 1.253 0.692 3.026 0.703 -0.0926 0.650 -0.0870* 0.074 0.218 0.293 0.00325 0.979 Baseline: Teacher Belief - Teaching English First (Base = Strongly Agree) Agree -1.078 0.323 -2.184* 0.086 -2.027 0.171 -1.802 0.281 0.00321 0.998 -1.049 0.678 -0.0550 0.555 -0.0482 0.154 0.0356 0.671 -0.133** 0.049 Neither Agree or Disagree 1.114 0.534 1.592 0.425 -2.256 0.301 -2.056 0.339 -0.677 0.765 3.468 0.301 0.128 0.322 -0.0518 0.509 0.159* 0.096 0.00521 0.960 Disagree -1.429 0.175 -2.308** 0.046 -4.279*** 0.000 -4.176*** 0.003 -1.279 0.258 -2.767 0.219 -0.00936 0.915 -0.0174 0.560 0.0265 0.714 -0.0903* 0.085 Strongly Disagree -0.138 0.909 -0.359 0.808 -0.805 0.590 -1.114 0.513 -1.600 0.223 -0.172 0.944 0.0683 0.449 -0.0176 0.624 0.00551 0.947 -0.0397 0.576 Pupil's Proficiency in Ghanaian LOI -0.715 0.133 -0.960* 0.066 -0.266 0.679 -0.472 0.511 0.158 0.764 -1.079 0.269 -0.0161 0.599 -0.153*** 0.000 0.0342 0.305 -0.0277 0.420 Length of School Day (Hours) 0.504 0.284 0.558 0.249 1.510*** 0.002 1.559*** 0.010 0.740 0.128 1.031 0.321 0.00822 0.782 0.00237 0.835 0.0388 0.182 0.0151 0.614 Rural -1.632* 0.096 -1.578 0.185 -2.146* 0.073 -2.404* 0.078 -3.186*** 0.001 -3.495 0.123 - - - - - - - - Pupil Repeated Grade -2.387*** 0.004 -2.129** 0.021 -2.416** 0.019 -2.233* 0.069 -1.663* 0.082 -4.988** 0.046 - - - - - - - - Pupil Age (Imputed) 0.139 0.406 0.113 0.542 -0.0241 0.922 0.237 0.383 0.292 0.238 0.897 0.139 -0.000996 0.938 0.0303*** 0.000 0.0114 0.397 0.00649 0.610 Pupil Gender 2.014*** 0.000 2.179*** 0.000 1.351** 0.015 1.547** 0.013 1.177*** 0.004 3.390*** 0.000 - - - - - - - - Official Language of Instruction (Base = Akuapim Twi) Asante Twi -4.901*** 0.001 -8.707*** 0.000 -4.215*** 0.007 -4.976*** 0.005 -3.271** 0.014 -12.07*** 0.001 -0.00292 0.969 -0.0252 0.424 -0.227*** 0.005 -0.0549 0.452 Dagaare -14.03*** 0.000 -12.62*** 0.000 -18.92*** 0.002 -16.82*** 0.000 -12.45*** 0.009 -16.75** 0.046 -0.0985 0.598 -0.177 0.141 -0.252 0.201 -2.406*** 0.000 Dagbani -11.12*** 0.001 -14.03*** 0.000 -5.805* 0.096 -10.08** 0.012 -9.127** 0.022 -16.65** 0.046 0.0794 0.739 -0.272*** 0.006 -0.0347 0.831 -0.348* 0.090 Dangme -0.597 0.835 -1.455 0.617 1.261 0.719 -1.561 0.670 2.616 0.350 5.083 0.258 0.509*** 0.006 -0.103** 0.023 0.280* 0.091 0.179 0.151 Ewe -2.137 0.257 -4.251** 0.046 0.304 0.874 -0.822 0.724 -0.294 0.872 3.077 0.391 0.0625 0.574 0.153*** 0.000 -0.0370 0.745 0.0762 0.385 Fante -1.119 0.508 -4.425** 0.037 -2.489 0.190 -0.357 0.859 -1.324 0.414 -11.08** 0.013 -0.172* 0.062 0.277*** 0.000 -0.0559 0.558 -0.173* 0.065 Ga 0.764 0.693 -6.201*** 0.007 2.157 0.365 -1.075 0.696 1.368 0.476 -3.704 0.347 0.620*** 0.000 -0.0885 0.119 0.101 0.313 -0.114 0.292 Gonja -6.520 0.175 -10.50** 0.028 -1.683 0.684 -4.063 0.340 -3.941 0.570 -21.27** 0.021 0.607* 0.074 0.155 0.178 0.0558 0.757 -0.133 0.189 Gurene -6.243*** 0.003 -14.55** 0.024 -0.722 0.803 -10.15 0.209 -3.734 0.463 -7.070 0.484 -0.127 0.407 -0.714*** 0.000 -2.967*** 0.000 -2.058*** 0.000 Kasem -16.71*** 0.000 -28.34*** 0.000 -10.13* 0.087 -17.94** 0.016 -3.166 0.549 -27.31** 0.012 -0.0733 0.849 -0.525*** 0.000 -3.200*** 0.000 -0.102 0.657 Baseline: Percent of Class Activities on Task -5.540 0.387 -6.403 0.317 -10.39 0.103 -12.51* 0.076 -10.25* 0.095 -16.27 0.171 -1.181** 0.020 -0.0368 0.780 -1.322*** 0.000 -0.435 0.142 Pupil Household Asset Index 2.144*** 0.000 2.494*** 0.000 3.204*** 0.000 2.918*** 0.000 2.088*** 0.000 3.295*** 0.000 0.0929*** 0.004 0.0379*** 0.005 0.106** 0.018 0.0853*** 0.008 School Resources Index 0.158 0.690 0.408 0.366 0.247 0.612 0.180 0.737 0.481 0.230 1.030 0.255 -0.00620 0.826 -0.00374 0.746 -0.0313 0.311 0.0230 0.350 Observations Standard errors are clustered at the school level (*** p<0.01, ** p<0.05, * p<0.1) Predictors Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian English 8,849 8,845 8,848 8,849 8,854 8,853 8,677 8,677 8,677 8,677 97 Figure 51: Tobit Impact Analysis Model (Female Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 3.792*** 0.004 3.045*** 0.009 1.870 0.219 1.674 0.371 3.300** 0.011 6.997* 0.054 0.00553 0.939 0.00871 0.790 0.112 0.187 0.0234 0.829 Midline 9.258*** 0.000 4.291*** 0.000 1.145 0.408 -0.586 0.719 -0.249 0.758 2.415 0.305 -0.0346 0.612 0.0452 0.131 0.0614 0.521 -0.0131 0.893 Midline * Treatment 12.87*** 0.000 21.97*** 0.000 11.08*** 0.000 18.00*** 0.000 0.456 0.741 22.65*** 0.000 -0.197** 0.038 0.105** 0.014 -0.00746 0.953 0.698*** 0.000 Baseline: Pupil Hardly Read -0.648 0.771 -2.869 0.178 -2.243 0.459 -2.079 0.530 -5.328*** 0.004 0.110 0.974 -0.267** 0.045 -0.116** 0.045 -0.212 0.160 0.0877 0.512 Baseline: Pupil Misses School Regularly -4.117 0.470 1.952 0.742 8.309 0.165 6.666 0.314 2.200 0.667 15.30* 0.093 -0.416 0.196 -0.136 0.270 0.404 0.185 0.447* 0.091 Pupil Hungry Everyday -3.368*** 0.000 -3.800*** 0.000 -2.874** 0.016 -3.714** 0.010 -2.794*** 0.003 -5.656*** 0.006 0.0287 0.597 -0.0617** 0.021 -0.0558 0.449 -0.132* 0.064 Baseline: Pupil Read Books Recently 7.224*** 0.003 7.690*** 0.004 9.294*** 0.002 10.66*** 0.002 7.853*** 0.001 10.50** 0.042 0.0123 0.934 0.0538 0.414 0.333** 0.037 0.193 0.195 Baseline: Pupil Likes Reading Books 7.165** 0.013 6.911** 0.048 5.161 0.296 5.325 0.286 9.581*** 0.008 7.790 0.296 -0.0399 0.862 0.190** 0.023 -0.345 0.157 0.0568 0.822 Baseline: Pupil Punished by Teacher -2.966 0.201 -2.751 0.281 -5.057 0.102 -7.342** 0.023 -4.858** 0.024 -12.14*** 0.006 -0.297** 0.019 -0.151** 0.012 -0.150 0.310 -0.238* 0.058 Baseline: Pupil Rewarded by Teacher -2.712 0.346 -4.239 0.164 -3.314 0.374 -2.872 0.444 -1.078 0.694 -2.931 0.550 -0.198 0.275 0.0361 0.661 -0.163 0.411 0.0311 0.853 Baseline: Teacher Education (Base = High School or Less) Diploma or Equivalent 2.078* 0.084 2.725** 0.034 3.404** 0.024 4.080** 0.039 1.122 0.406 -0.377 0.910 -0.0327 0.643 0.00610 0.843 0.239** 0.032 0.000706 0.992 Undergraduate Degree 4.929*** 0.000 5.802*** 0.000 5.695*** 0.001 6.068*** 0.005 2.471* 0.079 3.041 0.341 0.0287 0.711 0.00441 0.898 0.230* 0.050 0.0550 0.482 Postgraduate Degree -0.304 0.900 -2.587 0.365 2.751 0.273 -0.419 0.902 2.272 0.289 -7.232 0.225 0.0192 0.924 -0.192 0.121 0.326** 0.016 -0.163 0.313 Baseline: Teacher Belief - "Sound Out" (Base = Strongly Agree) Agree 0.857 0.418 0.978 0.362 1.112 0.380 1.025 0.467 0.696 0.443 -0.281 0.888 -0.137** 0.029 -0.00702 0.788 -0.0517 0.419 0.0109 0.842 Neither Agree or Disagree -0.736 0.720 -0.381 0.905 -4.022 0.241 -3.400 0.256 0.485 0.848 -4.651 0.289 -0.0392 0.732 -0.155* 0.059 0.123 0.468 -2.638*** 0.000 Disagree -2.190 0.125 -0.184 0.905 -2.101 0.337 -1.803 0.395 -0.484 0.775 -5.321 0.108 -0.0872 0.418 -0.0285 0.540 -0.146 0.261 -0.207** 0.026 Strongly Disagree 1.497 0.682 1.682 0.675 -1.180 0.856 -3.092 0.601 1.329 0.773 4.431 0.639 0.0228 0.909 -0.110* 0.073 0.292 0.225 0.00483 0.986 Baseline: Teacher Belief - Teaching English First (Base = Strongly Agree) Agree -1.076 0.410 -2.007 0.159 -1.540 0.388 -2.574 0.181 0.159 0.899 -2.184 0.372 -0.0140 0.885 -0.0495 0.223 0.0219 0.811 -0.176** 0.027 Neither Agree or Disagree 0.958 0.642 2.210 0.360 -1.396 0.547 -1.538 0.479 -1.061 0.595 2.388 0.464 0.0864 0.566 -0.0960 0.265 0.0461 0.657 -0.0726 0.580 Disagree -1.809 0.170 -1.985 0.131 -4.186*** 0.007 -4.301*** 0.010 -1.264 0.279 -3.174 0.162 0.00888 0.921 -0.0421 0.246 0.0347 0.704 -0.122* 0.075 Strongly Disagree -0.702 0.649 -0.0832 0.962 -0.265 0.886 -1.528 0.452 -1.250 0.360 -2.232 0.382 0.0643 0.497 -0.0399 0.345 -0.0373 0.727 -0.154 0.100 Pupil's Proficiency in Ghanaian LOI -0.745 0.257 -0.631 0.349 -0.164 0.858 -0.564 0.598 0.774 0.256 -0.395 0.720 -0.00327 0.925 -0.155*** 0.000 0.0375 0.418 0.0353 0.384 Length of School Day (Hours) 0.882 0.104 0.852 0.115 1.673*** 0.008 1.784** 0.010 0.790* 0.093 1.163 0.267 0.00531 0.854 0.0196 0.172 0.0483 0.200 0.0214 0.553 Rural -1.545 0.208 -1.141 0.419 -3.535** 0.015 -3.021** 0.049 -3.445*** 0.001 -4.136* 0.067 -0.283*** 0.000 0.00556 0.854 -0.217*** 0.005 -0.102 0.160 Pupil Repeated Grade -3.832*** 0.000 -4.079*** 0.002 -4.338*** 0.005 -4.578** 0.012 -2.870** 0.011 -6.054** 0.015 -0.159** 0.043 -0.0417 0.197 -0.00510 0.959 -0.0655 0.396 Pupil Age (Imputed) -0.0924 0.690 -0.106 0.682 -0.414 0.228 -0.185 0.642 0.145 0.574 0.145 0.768 0.00154 0.919 0.0224*** 0.004 0.0392* 0.058 0.0201 0.195 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -4.937*** 0.003 -8.248*** 0.000 -3.960** 0.031 -4.313** 0.025 -4.082*** 0.004 -11.53*** 0.000 0.0143 0.870 -0.0438 0.241 -0.256*** 0.007 -0.0593 0.554 Dagaare -14.17*** 0.000 -13.84*** 0.000 -79.86*** 0.000 -21.17*** 0.000 -13.39*** 0.002 -20.58*** 0.008 0.103 0.677 -0.183 0.234 -3.085*** 0.000 -2.962*** 0.000 Dagbani -13.26*** 0.000 -16.88*** 0.000 -8.270** 0.027 -11.43*** 0.006 -9.579** 0.015 -16.20** 0.031 0.157 0.475 -0.206* 0.051 0.0617 0.737 -0.306 0.199 Dangme -2.062 0.506 -2.858 0.396 -1.758 0.721 -2.386 0.613 1.667 0.608 4.166 0.350 0.409** 0.028 -0.168*** 0.003 0.305 0.171 0.173 0.343 Ewe -0.586 0.803 -2.836 0.249 0.820 0.734 0.970 0.733 0.510 0.789 6.014* 0.081 0.175 0.150 0.166*** 0.001 0.0502 0.725 0.216* 0.082 Fante -0.640 0.774 -4.001 0.114 -2.247 0.370 0.0939 0.970 -1.922 0.277 -10.22*** 0.009 -0.0833 0.471 0.307*** 0.000 -0.0656 0.626 -0.265* 0.051 Ga 1.379 0.557 -5.189** 0.036 1.571 0.549 -0.391 0.898 -0.0714 0.971 -2.934 0.444 0.543*** 0.000 -0.0854 0.161 0.0733 0.513 -0.221* 0.084 Gonja -8.271* 0.064 -9.176** 0.038 -0.155 0.972 -6.386 0.236 -0.148 0.984 -16.62* 0.088 1.049*** 0.002 0.233 0.135 0.358 0.195 -0.0482 0.826 Gurene -6.088** 0.016 -12.43* 0.068 -0.278 0.942 -3.968 0.671 -2.180 0.636 -16.42 0.150 -0.374 0.167 -0.893*** 0.000 -2.970*** 0.000 -2.489*** 0.000 Kasem -18.45*** 0.003 -32.18*** 0.000 -17.31** 0.013 -91.98*** 0.000 -4.000 0.425 -128.3*** 0.000 0.00215 0.996 -0.602*** 0.001 0.392 0.232 -3.435*** 0.000 Baseline: Percent of Class Activities on Task -0.716 0.916 -1.838 0.787 -6.848 0.334 -3.404 0.648 -9.110 0.132 -12.60 0.289 -0.758 0.124 -0.141 0.393 -1.083*** 0.004 -0.450 0.204 Pupil Household Asset Index 1.508*** 0.009 1.722*** 0.004 2.484*** 0.001 2.522*** 0.003 1.481*** 0.002 2.369** 0.024 0.0808** 0.025 0.0309* 0.077 0.0743 0.216 0.135*** 0.002 School Resources Index 0.521 0.294 0.692 0.204 0.989* 0.094 0.544 0.399 1.018** 0.019 1.424 0.115 -0.00852 0.769 0.000389 0.975 -0.00532 0.869 0.00700 0.799 Observations Standard errors are clustered at the school level (*** p<0.01, ** p<0.05, * p<0.1) Ghanaian Reading Comprehension Score (%) Predictors English Ghanaian English Ghanaian English Ghanaian English Ghanaian English Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) 4,407 4,406 4,406 4,407 4,410 4,409 4,410 4,410 4,410 4,410 98 Figure 52: Tobit Impact Analysis Mode (Male Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 2.065* 0.076 2.165* 0.060 1.504 0.313 1.062 0.573 1.748 0.317 6.385 0.228 -0.0283 0.717 -0.0340 0.324 -0.00332 0.971 0.0314 0.709 Midline 8.375*** 0.000 3.890*** 0.000 1.128 0.366 -1.228 0.424 -1.647 0.111 -0.134 0.958 -0.103 0.186 0.0275 0.407 -0.0393 0.650 -0.145* 0.073 Midline * Treatment 13.03*** 0.000 21.18*** 0.000 10.40*** 0.000 17.75*** 0.000 2.430 0.215 29.15*** 0.000 -0.0862 0.419 0.129*** 0.003 0.185 0.140 0.650*** 0.000 Baseline: Pupil Hardly Read -2.397 0.153 -4.810** 0.018 -4.114* 0.080 -6.818** 0.011 -5.987** 0.022 -7.927* 0.098 -0.310* 0.060 -0.179*** 0.001 -0.445*** 0.005 -0.0723 0.583 Baseline: Pupil Misses School Regularly -3.803 0.397 0.248 0.963 6.848 0.249 7.421 0.260 0.780 0.894 9.815 0.435 -0.0253 0.937 0.0786 0.606 -0.389 0.545 0.408 0.195 Pupil Hungry Everyday -1.502* 0.059 -2.158** 0.018 -2.615** 0.028 -2.867** 0.038 -3.880*** 0.003 -6.318** 0.040 -0.0997* 0.099 -0.00781 0.737 -0.184** 0.031 -0.0791 0.234 Baseline: Pupil Read Books Recently 6.964*** 0.001 8.272*** 0.001 11.45*** 0.000 11.45*** 0.000 9.905*** 0.006 15.53* 0.050 0.200 0.223 0.0126 0.843 0.395** 0.015 0.256* 0.093 Baseline: Pupil Likes Reading Books 5.827** 0.012 2.755 0.355 -0.0761 0.985 4.125 0.411 11.91** 0.026 5.731 0.565 -0.402 0.102 0.0349 0.645 -0.126 0.638 -0.173 0.423 Baseline: Pupil Punished by Teacher -0.941 0.622 0.224 0.924 -2.558 0.355 -1.227 0.701 -2.945 0.243 -2.274 0.637 -0.283* 0.055 -0.0849 0.152 -0.287* 0.064 0.00775 0.954 Baseline: Pupil Rewarded by Teacher -0.597 0.767 -0.198 0.941 3.601 0.227 1.406 0.658 -2.068 0.582 1.376 0.817 -0.111 0.524 -0.112 0.127 0.00940 0.961 0.334** 0.044 Baseline: Teacher Education (Base = High School or Less) Diploma or Equivalent 1.982** 0.044 1.709 0.132 2.110 0.124 0.775 0.644 1.179 0.553 -1.957 0.701 -0.0206 0.817 -0.0291 0.343 -0.0532 0.554 0.0831 0.314 Undergraduate Degree 3.577*** 0.002 2.752** 0.033 3.984** 0.014 3.664** 0.039 2.922 0.146 -1.103 0.829 -0.0262 0.778 -0.0203 0.568 -0.0121 0.900 0.0490 0.604 Postgraduate Degree -1.232 0.582 -1.746 0.550 0.412 0.883 3.143 0.324 0.406 0.882 -7.164 0.451 -0.141 0.330 -0.230 0.135 -0.181 0.194 -0.159 0.359 Baseline: Teacher Belief - "Sound Out" (Base = Strongly Agree) Agree -0.0312 0.969 0.105 0.910 0.341 0.754 -0.109 0.933 0.629 0.605 -2.771 0.398 -0.0682 0.313 -0.00137 0.956 -0.0204 0.781 0.109** 0.050 Neither Agree or Disagree 1.564 0.377 0.902 0.650 -3.698 0.316 0.677 0.883 7.097* 0.052 -4.640 0.579 -0.0817 0.711 -0.189** 0.020 0.231 0.103 -2.304*** 0.000 Disagree -0.477 0.728 1.052 0.535 0.221 0.918 0.723 0.767 -0.160 0.931 -6.429 0.177 -0.0946 0.469 -0.0280 0.542 -0.206 0.123 -0.0104 0.915 Strongly Disagree 3.312 0.340 2.371 0.359 3.473* 0.088 3.169 0.172 1.309 0.483 0.578 0.921 -0.280 0.323 -0.0147 0.724 -0.152 0.530 -0.0231 0.837 Baseline: Teacher Belief - Teaching English First (Base = Strongly Agree) Agree -1.115 0.315 -2.416* 0.074 -2.622* 0.094 -1.277 0.467 -0.216 0.887 0.0391 0.990 -0.125 0.180 -0.0671* 0.089 -0.00563 0.956 -0.178** 0.024 Neither Agree or Disagree 0.965 0.612 0.780 0.731 -3.129 0.255 -2.580 0.429 -0.659 0.835 3.795 0.447 0.104 0.467 -0.0509 0.489 0.157 0.145 -0.0229 0.842 Disagree -1.186 0.264 -2.721** 0.032 -4.375*** 0.001 -4.102** 0.012 -1.394 0.314 -2.577 0.364 -0.0687 0.444 -0.0250 0.486 -0.0295 0.742 -0.128* 0.060 Strongly Disagree 0.353 0.767 -0.715 0.631 -1.391 0.385 -0.844 0.643 -2.097 0.181 2.614 0.391 0.0644 0.499 -0.00375 0.928 0.00111 0.990 0.0560 0.466 Pupil's Proficiency in Ghanaian LOI -0.641 0.224 -1.216* 0.060 -0.217 0.775 -0.341 0.706 -0.438 0.536 -1.675 0.298 -0.0418 0.292 -0.154*** 0.000 0.0701* 0.089 -0.0864* 0.063 Length of School Day (Hours) 0.136 0.797 0.242 0.663 1.321** 0.019 1.323* 0.059 0.634 0.315 1.036 0.406 0.0105 0.772 -0.0134 0.354 0.0376 0.256 0.0311 0.315 Rural -1.828* 0.067 -2.108* 0.086 -0.893 0.502 -1.887 0.235 -2.968** 0.015 -2.612 0.350 -0.181** 0.025 -0.0610** 0.039 -0.165** 0.033 -0.0728 0.237 Pupil Repeated Grade -0.924 0.425 -0.195 0.879 -0.845 0.530 -0.201 0.903 -0.444 0.752 -4.070 0.264 -0.0485 0.535 -0.00649 0.854 0.0761 0.440 0.0226 0.781 Pupil Age (Imputed) 0.379* 0.076 0.328 0.180 0.315 0.315 0.591 0.115 0.452 0.216 1.662* 0.100 0.00343 0.846 0.0398*** 0.000 -0.0159 0.467 -0.00354 0.848 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -4.889*** 0.001 -9.201*** 0.000 -4.558*** 0.008 -5.766*** 0.006 -2.338 0.135 -12.11*** 0.007 0.0530 0.561 0.0229 0.579 -0.120 0.255 -0.0825 0.304 Dagaare -14.25*** 0.000 -11.89*** 0.004 -13.15** 0.037 -14.05** 0.012 -11.02* 0.055 -11.45 0.237 -0.103 0.754 -0.248* 0.094 -0.173 0.375 -2.550*** 0.000 Dagbani -9.419*** 0.008 -11.83*** 0.002 -4.004 0.321 -9.531* 0.054 -8.584* 0.059 -17.29* 0.089 0.257 0.357 -0.261** 0.013 0.111 0.539 -0.348 0.143 Dangme 0.704 0.818 -0.217 0.936 3.966 0.154 -0.770 0.816 3.324 0.245 5.878 0.278 0.620*** 0.008 -0.0294 0.612 0.325* 0.067 0.304** 0.016 Ewe -3.479* 0.050 -5.475** 0.012 -0.0697 0.973 -2.572 0.313 -1.004 0.634 -0.407 0.930 0.0850 0.509 0.155*** 0.004 -0.0648 0.636 -0.0293 0.784 Fante -1.679 0.296 -5.020** 0.015 -2.794 0.141 -0.885 0.695 -0.775 0.688 -12.00** 0.039 -0.270** 0.020 0.278*** 0.000 -0.140 0.260 -0.164* 0.094 Ga 0.211 0.919 -7.176*** 0.004 2.699 0.318 -1.667 0.592 2.890 0.229 -4.067 0.422 0.711*** 0.000 -0.0692 0.325 0.0885 0.535 -0.108 0.465 Gonja -5.013 0.405 -11.87** 0.039 -3.815 0.454 -2.522 0.618 -8.317 0.225 -27.32** 0.046 0.554 0.206 0.221* 0.093 -2.992*** 0.000 -0.364* 0.082 Gurene -6.126*** 0.005 -17.03*** 0.008 -1.511 0.836 -71.32*** 0.000 -5.558 0.411 -0.288 0.981 0.0279 0.879 -0.617*** 0.000 -2.854*** 0.000 -2.165*** 0.000 Kasem -15.32*** 0.000 -25.61*** 0.000 -6.385 0.252 -13.93** 0.044 -1.866 0.773 -18.16* 0.082 -0.555 0.265 -0.427*** 0.000 -3.457*** 0.000 0.0728 0.756 Baseline: Percent of Class Activities on Task -9.532 0.153 -10.20 0.130 -13.59* 0.053 -20.47** 0.011 -11.34 0.106 -20.75 0.130 -1.372*** 0.008 0.0874 0.545 -1.396*** 0.000 -0.446 0.159 Pupil Household Asset Index 2.700*** 0.000 3.215*** 0.000 3.850*** 0.000 3.213*** 0.000 2.787*** 0.000 4.324*** 0.003 0.0879** 0.038 0.0422** 0.014 0.145** 0.013 0.0856** 0.028 School Resources Index -0.202 0.642 0.123 0.805 -0.486 0.397 -0.218 0.744 -0.146 0.777 0.524 0.662 -0.00609 0.849 -0.0119 0.384 -0.0505 0.213 0.0335 0.297 Observations Standard errors are clustered at the school level (*** p<0.01, ** p<0.05, * p<0.1) Ghanaian Reading Comprehension Score (%) Predictors English Ghanaian English Ghanaian English Ghanaian English Ghanaian English Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) 4,442 4,439 4,442 4,442 4,444 4,444 4,444 4,444 4,444 4,444 99 ANNEX IV. FULL PROGRAM PREDICTORS OF SUCCESS REGRESSION OUTPUT Figure 53: OLS Predictors of Assessment Scores Model Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Pupil Weekly Reading (Base = Hardly) 2.pp_read_weekly 0.0853 0.828 0.521 0.179 0.254 0.243 -0.372* 0.069 -0.109 0.423 -0.111 0.473 0.0102 0.142 0.0164* 0.078 0.00175 0.349 -0.00307 0.169 3.pp_read_weekly 0.729 0.167 1.373** 0.016 0.157 0.614 -0.0103 0.978 0.0947 0.638 0.137 0.542 -0.00676 0.382 0.0359*** 0.003 -0.000127 0.950 0.00286 0.413 Pupil Missess School Regularly -1.479 0.135 -1.820* 0.053 -0.579 0.157 -0.731* 0.063 -0.654*** 0.002 -0.724*** 0.002 0.0293* 0.081 -0.0125 0.573 -0.00103 0.602 -0.00953*** 0.000 Pupil Hungry Everyday -2.797*** 0.000 -2.831*** 0.000 -1.223*** 0.000 -1.304*** 0.000 -0.658*** 0.000 -0.771*** 0.000 -0.00118 0.886 -0.0219** 0.043 -0.00494** 0.017 -0.00788*** 0.000 Pupil Read Books Recently 5.521*** 0.000 6.548*** 0.000 2.271*** 0.000 2.763*** 0.000 1.782*** 0.000 2.032*** 0.000 0.0152** 0.015 0.0247*** 0.004 0.0100*** 0.000 0.0192*** 0.000 Pupil Likes Reading 3.192*** 0.000 2.729*** 0.000 1.339*** 0.000 1.203*** 0.000 0.788*** 0.000 0.917*** 0.000 0.0148* 0.054 0.0481*** 0.000 0.00492*** 0.000 0.00674*** 0.000 Pupil Puinished by Teacher -1.769*** 0.001 -2.141*** 0.000 -1.029*** 0.000 -1.167*** 0.000 -0.707*** 0.000 -1.073*** 0.000 -0.00834 0.226 -0.0161* 0.095 -0.00752*** 0.002 -0.0124*** 0.000 Pupil Rewarded by Teacher 0.108 0.833 0.246 0.648 0.612 0.116 0.603* 0.055 0.298* 0.059 0.469** 0.018 0.00424 0.630 0.0530*** 0.000 0.00386** 0.038 0.00624** 0.014 Length of School Day (Hours) -0.000345 0.999 -0.372 0.391 -0.0245 0.932 -0.309 0.388 0.0794 0.505 -0.0653 0.669 0.00443 0.504 0.000341 0.955 0.00144 0.310 -0.000124 0.948 Rural -1.943** 0.036 -1.940* 0.082 -0.836* 0.098 -0.258 0.665 -0.666** 0.025 -0.572 0.118 -0.0417*** 0.009 -0.0213 0.132 -0.00831** 0.016 -0.00677 0.146 Pupil Repeated Grade -1.528** 0.020 -1.349** 0.047 -0.739* 0.080 -0.653 0.178 -0.448* 0.062 -0.438 0.128 -0.0179 0.145 -0.0110 0.454 -0.00178 0.577 0.00179 0.713 Pupil Age (Imputed) -0.0207 0.884 -0.00196 0.989 0.0687 0.585 0.127 0.384 -0.0227 0.645 -0.00459 0.933 0.000710 0.762 0.0167*** 0.000 0.000922 0.165 0.000596 0.482 Pupil Gender 1.541*** 0.000 1.257*** 0.000 0.164 0.364 0.323* 0.055 0.256** 0.028 0.202 0.106 -0.00246 0.587 -0.0157** 0.036 0.000389 0.787 0.000506 0.764 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -4.375*** 0.002 -7.156*** 0.000 -1.221** 0.046 -0.848 0.122 -1.058*** 0.009 -1.047** 0.017 0.00297 0.825 -0.0221 0.261 -0.00487 0.201 0.00127 0.777 Dagaare -8.450*** 0.000 -9.094*** 0.000 -2.437*** 0.004 -1.843* 0.054 -1.479*** 0.007 -1.773*** 0.006 -0.0142 0.553 -0.0817 0.206 -0.00184 0.754 -0.00499 0.448 Dagbani -5.602** 0.030 -8.194*** 0.001 -0.942 0.500 -0.385 0.794 -0.545 0.467 -0.915 0.208 0.0521 0.436 -0.0777* 0.050 0.0122 0.500 0.00437 0.742 Dangme -0.999 0.665 -3.845* 0.058 1.211 0.411 0.856 0.231 -0.283 0.688 -0.941 0.103 0.106* 0.057 -0.0860*** 0.001 0.0152 0.311 0.00890 0.254 Ewe -1.907 0.267 -4.264** 0.024 -0.669 0.349 1.388** 0.044 -0.361 0.437 -0.521 0.305 0.0104 0.563 0.105*** 0.000 -0.00199 0.639 0.0154** 0.040 Fante -1.114 0.472 -3.597* 0.052 -0.558 0.383 -0.642 0.223 -0.521 0.237 -0.0892 0.853 -0.0183 0.204 0.178*** 0.000 0.000182 0.967 -0.00352 0.481 Ga 0.575 0.785 -4.246* 0.068 1.886 0.131 0.489 0.676 0.735 0.376 0.115 0.887 0.176*** 0.000 -0.0665** 0.039 0.00607 0.490 -0.00601 0.529 Gonja -4.110 0.233 -8.095** 0.018 0.0951 0.971 -2.703*** 0.001 -0.283 0.778 -1.373 0.106 0.173* 0.093 0.100 0.112 -0.00210 0.747 -0.00985 0.125 Gurene -6.919*** 0.000 -9.002*** 0.000 -2.023* 0.075 -0.720 0.467 -0.847 0.127 -0.903 0.113 -0.0180 0.609 -0.198*** 0.000 0.000232 0.980 0.00732 0.342 Kasem -7.020*** 0.000 -9.847*** 0.000 -0.133 0.890 -0.215 0.865 -0.721 0.287 -0.747 0.343 0.0246 0.594 -0.0873 0.108 0.00156 0.855 0.0105 0.181 Class Size 0.0232 0.315 0.0134 0.595 0.0139 0.172 0.0126 0.241 0.00421 0.527 0.00719 0.336 0.000532 0.163 9.66e-05 0.778 8.85e-05 0.203 3.44e-05 0.680 Pupil's Proficiency in Ghanaian LOI (Base = High Proficiency) Medium Proficiency 0.573 0.352 0.444 0.479 0.663** 0.047 0.843** 0.043 0.390* 0.060 0.349* 0.100 0.0152 0.128 -0.0650*** 0.000 0.00433** 0.031 0.00484 0.106 Low Proficiency -3.531*** 0.000 -4.168*** 0.000 -0.243 0.656 -1.417*** 0.006 -0.697** 0.036 -1.070*** 0.001 -0.0102 0.589 -0.183*** 0.000 -0.000819 0.899 -0.00996* 0.084 Pupil Household Asset Index 0.636* 0.092 0.334 0.375 0.453* 0.066 -0.0369 0.907 0.301** 0.036 0.194 0.175 0.0180** 0.017 0.0168** 0.029 0.00447* 0.050 0.00257 0.296 School Resources Index 0.124 0.516 0.0324 0.878 -0.0375 0.679 0.0236 0.801 -0.0182 0.737 0.000952 0.987 0.00117 0.731 0.000675 0.830 -0.000334 0.671 0.000235 0.774 Number of Teachers at School 0.104 0.434 0.129 0.398 0.134* 0.080 0.0638 0.284 0.0580 0.168 0.0566 0.210 0.00132 0.547 0.00106 0.679 0.000205 0.598 0.000232 0.675 Proportion of Female PI Teachers 1.263* 0.086 0.653 0.426 1.450*** 0.001 0.919 0.143 0.643*** 0.000 0.592** 0.014 0.0225** 0.048 0.00394 0.774 0.00653*** 0.009 0.00334 0.378 Sufficient Materials at School 10.02*** 0.000 11.89*** 0.000 0.614 0.207 3.900*** 0.000 2.090*** 0.000 2.668*** 0.000 -0.0158 0.209 0.0517*** 0.001 0.00334 0.390 0.0271*** 0.000 School Library Available 1.683* 0.071 1.882* 0.054 0.253 0.623 0.974 0.100 0.434 0.142 0.237 0.461 0.00702 0.696 0.0217 0.133 0.00360 0.420 0.00881* 0.087 Constant 8.915** 0.038 13.06*** 0.003 -0.210 0.921 0.337 0.860 0.174 0.879 0.966 0.441 -0.0231 0.766 0.153** 0.012 -0.0196 0.296 -0.00555 0.779 Observations R-squared *** p<0.01, ** p<0.05, * p<0.1 8,679 8,684 8,684 8,684 0.096 0.111 0.180 0.041 8,684 0.198 0.224 0.063 0.067 0.110 8,679 8,675 8,684 8,683 8,678 0.061 Predictors Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian English 100 Figure 54: OLS FOI Predictors of Assessment Scores Model Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Fidelity of Implementation * Survey Wave Low Implementation * Midline 0.760 0.390 0.818 0.495 2.531 0.419 3.982 0.274 0.903 0.397 0.916 0.286 -0.0356* 0.095 -0.0325* 0.077 0.00472 0.248 0.00151 0.537 Partial Implementation * Midline -0.546 0.337 1.421* 0.077 9.709*** 0.000 12.65*** 0.000 0.597 0.197 1.361** 0.013 -0.0629* 0.061 -0.00703 0.867 -0.00860 0.103 0.00910 0.181 Near-Full Implementation * Midline 1.289 0.229 5.476*** 0.000 13.24*** 0.000 19.40*** 0.000 3.392*** 0.000 4.391*** 0.000 -0.0145 0.377 0.0815*** 0.000 0.00942** 0.044 0.0555*** 0.000 Full Implementation * Midline 0.896 0.157 7.579*** 0.000 16.17*** 0.000 21.43*** 0.000 3.429*** 0.000 5.276*** 0.000 -0.0256 0.221 0.0354 0.371 -0.00351 0.597 0.0500*** 0.002 Fidelity of Implementation Low Implementation 0.448 0.519 1.560* 0.058 1.345 0.188 2.606*** 0.005 0.640* 0.052 1.004*** 0.007 -0.00387 0.909 -0.0618** 0.037 -0.00385 0.570 0.00783 0.138 Partial Implementation 0.0786 0.877 0.456 0.232 -0.112 0.902 0.276 0.748 0.196 0.522 0.197 0.490 0.0262 0.372 0.0152 0.600 0.00540 0.270 0.00348 0.377 Near-Full Implementation 1.104 0.230 1.217 0.325 2.054*** 0.008 1.260** 0.044 0.0452 0.816 0.281 0.454 0.0114 0.448 -0.00245 0.885 -7.49e-05 0.974 -0.00281 0.218 Full Implementation 0.756 0.201 0.150 0.778 1.449 0.277 1.057 0.382 0.172 0.675 0.0868 0.841 -0.0229 0.123 0.0184 0.537 0.00319 0.599 0.00229 0.584 Midline -0.144 0.549 0.207 0.211 4.151*** 0.000 1.262*** 0.001 0.153 0.293 -0.0922 0.516 -0.00590 0.619 0.0278* 0.052 0.000151 0.943 -0.00231 0.152 Rural -0.758 0.154 0.00554 0.992 -1.301* 0.065 -1.223 0.126 -0.570** 0.038 -0.430 0.203 -0.0420*** 0.009 -0.0184 0.188 -0.00846** 0.018 -0.00540 0.196 Pupil Gender 0.262 0.142 0.424** 0.010 1.687*** 0.000 1.431*** 0.000 0.322*** 0.005 0.287** 0.021 -0.00171 0.703 -0.0150** 0.046 0.00124 0.380 0.00140 0.408 Pupil's Proficiency in Ghanaian LOI 0.197 0.353 0.0731 0.700 -0.453 0.220 -0.501 0.149 0.0162 0.904 -0.0579 0.640 -0.00200 0.776 -0.0842*** 0.000 0.00136 0.532 0.000374 0.854 Pupil Hungry Everyday -1.161*** 0.000 -0.922*** 0.004 -1.549*** 0.000 -1.436*** 0.000 -0.459*** 0.001 -0.512*** 0.002 -0.00338 0.673 -0.0171* 0.099 -0.00449** 0.027 -0.00503** 0.010 Pupil Rewarded by Teacher 0.706* 0.088 0.775** 0.025 0.911** 0.046 0.909** 0.046 0.407*** 0.008 0.534*** 0.005 0.00548 0.543 0.0580*** 0.000 0.00376** 0.029 0.00609*** 0.010 Length of School Day (Hours) 0.0652 0.814 -0.0806 0.807 0.392 0.251 0.255 0.416 0.178 0.110 0.0893 0.527 0.00433 0.532 0.00186 0.760 0.00160 0.281 0.00140 0.426 Pupil Repeated Grade -0.643 0.127 -0.579 0.206 -1.755*** 0.005 -1.409** 0.024 -0.413* 0.079 -0.375 0.171 -0.0156 0.192 -0.0109 0.441 -0.00107 0.731 0.00238 0.608 Pupil Age (Imputed) 0.0745 0.553 0.143 0.329 0.0108 0.933 -0.00527 0.966 -0.0136 0.785 0.00348 0.948 0.00138 0.562 0.0175*** 0.000 0.000914 0.177 0.000673 0.424 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -1.401** 0.027 -0.811 0.109 -4.325*** 0.001 -6.989*** 0.000 -1.061*** 0.010 -1.013** 0.021 -0.00345 0.802 -0.0242 0.223 -0.00600 0.132 0.00157 0.711 Dagaare -2.828*** 0.003 -2.059* 0.059 -9.091*** 0.000 -9.557*** 0.000 -1.676*** 0.005 -1.865*** 0.009 0.00223 0.948 -0.0742 0.269 -0.00429 0.455 -0.00561 0.417 Dagbani -1.633 0.235 -0.799 0.582 -6.408*** 0.006 -8.835*** 0.000 -0.776 0.276 -1.030 0.145 0.0349 0.560 -0.0865** 0.030 0.00777 0.646 0.00325 0.793 Dangme 1.311 0.397 1.307 0.120 0.105 0.966 -2.367 0.271 0.00428 0.995 -0.516 0.434 0.0950* 0.095 -0.0786*** 0.002 0.0157 0.307 0.0138** 0.047 Ewe -1.042 0.198 0.970 0.219 -2.564 0.114 -5.028*** 0.004 -0.503 0.308 -0.713 0.191 -0.000300 0.987 0.0953*** 0.000 -0.00346 0.446 0.0134* 0.082 Fante -1.080 0.111 -1.278** 0.019 -2.378* 0.095 -4.927*** 0.003 -0.852** 0.049 -0.509 0.281 -0.0233* 0.088 0.168*** 0.000 -0.00171 0.706 -0.00725 0.121 Ga 1.395 0.242 -0.220 0.806 -0.295 0.876 -5.309*** 0.006 0.325 0.675 -0.286 0.700 0.158*** 0.000 -0.0766** 0.014 0.00376 0.633 -0.0105 0.170 Gonja 0.600 0.815 -1.496* 0.059 -2.195 0.481 -5.754** 0.035 0.427 0.673 -0.472 0.549 0.165 0.109 0.0943 0.168 -8.51e-05 0.990 -0.00118 0.851 Gurene -2.611** 0.024 -0.969 0.326 -6.686*** 0.000 -8.376*** 0.000 -1.143** 0.041 -1.085* 0.059 -0.0357 0.266 -0.205*** 0.000 -0.00504 0.563 0.00458 0.531 Kasem -1.373 0.242 -2.598** 0.013 -12.57*** 0.000 -16.06*** 0.000 -1.940*** 0.005 -2.308*** 0.002 -0.000739 0.986 -0.114** 0.016 0.00336 0.702 -0.00643 0.528 Class Size 0.00938 0.332 0.00792 0.390 0.0168 0.366 0.00745 0.699 0.00166 0.789 0.00402 0.556 0.000412 0.282 1.37e-05 0.966 7.07e-05 0.271 5.67e-06 0.943 Pupil Household Asset Index 0.771*** 0.002 0.387 0.207 1.833*** 0.000 1.750*** 0.000 0.576*** 0.000 0.529*** 0.000 0.0192** 0.014 0.0261*** 0.001 0.00580** 0.020 0.00591** 0.011 School Resource Index -0.0133 0.887 0.0272 0.765 0.0534 0.715 -0.0199 0.895 -0.0181 0.725 0.00508 0.925 0.00151 0.652 6.51e-05 0.983 -0.000336 0.670 0.000146 0.844 Number of Teachers at School 0.134* 0.075 0.0416 0.427 0.0569 0.607 0.0715 0.472 0.0472 0.258 0.0423 0.302 0.00176 0.410 0.00112 0.667 0.000245 0.515 6.77e-05 0.900 Proportion of Female PI Teachers 1.406*** 0.001 1.311** 0.018 2.372*** 0.000 2.040*** 0.000 0.821*** 0.000 0.858*** 0.000 0.0201* 0.077 0.0115 0.394 0.00646** 0.010 0.00660* 0.061 School Library Available 0.212 0.699 0.792 0.171 1.273 0.157 1.368 0.114 0.388 0.194 0.148 0.642 0.0107 0.554 0.0209 0.158 0.00408 0.383 0.00797* 0.093 Constant 0.00913 0.997 -2.886 0.127 2.646 0.454 5.801 0.101 -0.566 0.624 -0.520 0.682 0.0446 0.540 0.272*** 0.000 -0.0131 0.436 -0.0212 0.268 Observations R-squared Predictors English Ghanaian English Ghanaian English Ghanaian English Ghanaian English Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) 8,839 8,839 8,839 8,839 0.050 0.091 0.346 0.405 0.120 0.120 8,839 8,838 8,834 8,830 8,833 8,834 0.107 0.186 0.033 0.087 *** p<0.01, ** p<0.05, * p<0.1 101 Figure 55: OLS Predictors of Assessment Scores Model (Rural Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 0.895 0.279 0.911 0.409 0.612 0.393 0.319 0.594 -0.0957 0.598 0.190 0.588 -0.00304 0.822 0.00143 0.936 0.000425 0.809 -0.00223 0.350 Midline -0.177 0.496 0.357* 0.069 4.447*** 0.000 1.767*** 0.000 0.175 0.235 0.00967 0.952 -0.00969 0.506 0.0344** 0.045 0.000318 0.857 -0.00121 0.420 Treatment * Midline 0.262 0.779 4.036*** 0.002 12.60*** 0.000 17.39*** 0.000 2.432*** 0.000 3.182*** 0.000 -0.0232 0.191 0.0446* 0.068 0.000519 0.844 0.0425*** 0.000 Pupil Gender 0.0668 0.736 0.300 0.103 1.786*** 0.000 1.539*** 0.000 0.168 0.137 0.272** 0.041 -0.00361 0.389 -0.00427 0.609 0.00167 0.131 0.00136 0.451 Pupil's Proficiency in Ghanaian LOI (Base = High Proficiency) Medium Proficiency 0.651* 0.063 0.781* 0.076 0.426 0.493 0.302 0.588 0.334 0.105 0.320 0.133 0.00579 0.523 -0.0689*** 0.000 0.00345** 0.048 0.00525* 0.097 Low Proficiency -0.304 0.416 -0.318 0.335 -1.552** 0.037 -1.640** 0.027 -0.286 0.183 -0.484** 0.049 -0.0118 0.392 -0.139*** 0.000 0.00113 0.753 -0.00160 0.685 Pupil Hungry Everyday -0.672** 0.013 -0.697** 0.037 -0.953** 0.020 -0.889** 0.020 -0.275** 0.034 -0.320* 0.051 0.00503 0.502 -0.00387 0.740 -0.00201* 0.073 -0.00304* 0.080 Pupil Rewarded by Teacher 0.243 0.587 0.867** 0.017 0.714 0.145 0.911* 0.055 0.313** 0.033 0.411** 0.035 0.00729 0.354 0.0658*** 0.000 0.00192 0.189 0.00569** 0.034 Length of School Day (Hours) -0.146 0.704 -0.368 0.461 0.284 0.426 0.111 0.758 0.0798 0.509 0.0184 0.924 -0.00142 0.785 0.00206 0.784 0.00116 0.209 5.27e-05 0.978 Pupil Repeated Grade 0.0192 0.971 0.0909 0.884 -1.043 0.144 -0.879 0.222 -0.00850 0.976 0.0175 0.960 -0.00735 0.484 0.00138 0.938 0.00516 0.118 0.0114* 0.078 Pupil Age (Imputed) 0.0509 0.736 0.0953 0.594 -0.0571 0.689 -0.0883 0.524 -0.0403 0.460 0.00570 0.923 0.00107 0.607 0.0141*** 0.000 0.000140 0.738 -0.000858 0.265 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -1.625* 0.055 -1.031 0.127 -5.857*** 0.000 -9.028*** 0.000 -1.333*** 0.004 -1.207** 0.012 -0.00143 0.914 -0.0269 0.345 -0.00862* 0.087 -0.00256 0.585 Dagaare -3.285*** 0.001 -2.205** 0.023 -10.72*** 0.000 -11.70*** 0.000 -1.956*** 0.001 -1.997*** 0.002 0.00745 0.827 -0.0822 0.268 -0.00857 0.134 -0.00791 0.220 Dagbani -2.446** 0.013 -2.090** 0.014 -9.168*** 0.000 -12.17*** 0.000 -1.319** 0.022 -1.367** 0.020 0.00667 0.817 -0.143*** 0.004 -0.00690 0.270 -0.00967 0.203 Dangme 0.589 0.637 1.365* 0.085 -1.402 0.581 -4.035* 0.091 -0.328 0.627 -0.610 0.349 0.0810* 0.053 -0.0840** 0.012 0.00746 0.484 0.0120 0.160 Ewe -0.880 0.339 1.517** 0.037 -2.930 0.127 -5.614*** 0.006 -0.383 0.475 -0.450 0.419 0.0137 0.451 0.0934*** 0.004 -0.00469 0.401 0.0148* 0.056 Fante -1.288 0.130 -1.005* 0.075 -3.234* 0.072 -5.873*** 0.004 -0.877* 0.089 -0.461 0.380 -0.0208 0.131 0.163*** 0.000 -0.00752 0.141 -0.00620 0.203 Ga 7.450* 0.093 8.560 0.167 2.650 0.446 0.202 0.974 3.708* 0.086 4.525 0.164 0.360** 0.035 -0.0694 0.206 0.0386 0.160 -0.00322 0.854 Gonja -1.886 0.200 -1.726* 0.076 -3.956 0.369 -7.633** 0.043 -0.252 0.808 -0.816 0.300 0.177 0.193 -0.0449 0.424 0.00156 0.864 -0.00550 0.457 Gurene -3.005*** 0.010 -1.301 0.123 -8.628*** 0.000 -11.09*** 0.000 -1.416** 0.014 -1.180** 0.044 -0.0347 0.107 -0.224*** 0.000 -0.0105 0.113 0.00233 0.727 Kasem -1.687* 0.072 -2.667*** 0.002 -14.60*** 0.000 -18.48*** 0.000 -2.203*** 0.000 -2.360*** 0.000 -0.0190 0.323 -0.153** 0.013 -0.000551 0.949 -0.0111* 0.080 Class Size 0.0142 0.133 0.0113 0.303 0.0388* 0.054 0.0345 0.104 0.00453 0.478 0.00599 0.404 -2.38e-05 0.941 -4.14e-05 0.907 1.04e-05 0.821 -4.40e-06 0.966 Pupil Household Asset Index 0.339 0.157 0.191 0.593 1.678*** 0.000 1.471*** 0.000 0.422*** 0.005 0.458*** 0.004 0.00620 0.362 0.0307*** 0.001 0.000113 0.929 0.00559** 0.038 School Resources Index -0.0369 0.675 -0.0178 0.851 -0.00445 0.976 -0.0758 0.625 -0.0454 0.345 -0.0109 0.832 0.000642 0.811 0.00225 0.552 -0.000520 0.224 -0.000496 0.513 Number of Teachers at School 0.181** 0.035 0.0935 0.110 0.0976 0.413 0.134 0.203 0.0575 0.191 0.0703 0.110 0.00154 0.494 0.000927 0.765 0.000358 0.304 0.000347 0.563 Proportion of Female PI Teachers 1.135*** 0.009 0.746 0.224 1.950*** 0.001 1.561** 0.011 0.548*** 0.003 0.524** 0.026 0.0151 0.109 0.00972 0.503 0.00333** 0.011 0.00200 0.594 School Library Available 0.355 0.513 0.822 0.191 1.929** 0.044 2.107** 0.031 0.625** 0.036 0.379 0.259 -0.00757 0.543 0.0183 0.293 0.00178 0.505 0.00935* 0.092 Constant 1.401 0.526 -0.0340 0.983 1.986 0.565 5.547 0.123 0.0548 0.959 -0.628 0.604 0.0430 0.367 0.140* 0.058 -0.00418 0.646 -0.00189 0.912 Observations R-squared Predictors Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian English 6,774 6,774 6,774 6,774 0.032 0.066 0.340 0.400 0.099 0.088 6,774 6,773 6,769 6,766 6,768 6,769 0.061 0.195 0.026 0.077 *** p<0.01, ** p<0.05, * p<0.1 102 Figure 56: OLS Predictors of Assessment Scores Model (Urban Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 0.799 0.304 0.846 0.102 4.323*** 0.007 3.130** 0.015 0.635 0.178 0.266 0.565 0.0511* 0.094 0.00908 0.748 0.000800 0.910 0.00319 0.592 Midline -0.597 0.372 -0.370 0.320 3.029*** 0.006 -0.235 0.824 -0.0137 0.972 -0.431 0.208 -0.0126 0.625 -0.00695 0.804 -0.00617 0.344 -0.00635 0.197 Treatment * Midline 3.240** 0.012 8.881*** 0.000 15.79*** 0.000 23.16*** 0.000 4.717*** 0.000 6.867*** 0.000 -0.0121 0.738 0.103*** 0.008 0.0231 0.113 0.0680*** 0.000 Pupil Gender 0.820* 0.054 0.863** 0.016 1.625** 0.016 1.325** 0.025 0.817** 0.013 0.426 0.150 0.00304 0.801 -0.0422*** 0.008 0.000122 0.977 0.00169 0.671 Pupil's Proficiency in Ghanaian LOI (Base = High Proficiency) Medium Proficiency 0.688 0.361 0.953 0.149 0.814 0.444 0.992 0.339 0.594 0.242 0.561 0.240 0.0304 0.257 -0.0509* 0.055 0.00463 0.453 0.00463 0.452 Low Proficiency 0.409 0.723 -0.810 0.188 -1.680 0.323 -1.901 0.168 -0.380 0.592 -0.505 0.407 -0.0117 0.677 -0.236*** 0.000 0.00481 0.639 -0.00241 0.796 Pupil Hungry Everyday -2.176*** 0.000 -1.451*** 0.001 -3.415*** 0.000 -3.224*** 0.000 -0.962*** 0.008 -1.105*** 0.004 -0.0201 0.339 -0.0315 0.117 -0.0109** 0.048 -0.00949** 0.025 Pupil Rewarded by Teacher 2.372*** 0.000 0.832* 0.078 1.924* 0.096 1.464 0.133 0.915** 0.027 1.186*** 0.002 0.00493 0.845 0.0421* 0.061 0.0110** 0.015 0.00861* 0.085 Length of School Day (Hours) 0.413 0.237 0.418* 0.067 0.715 0.271 0.623 0.265 0.405* 0.075 0.265 0.207 0.0179 0.132 0.00255 0.781 0.00261 0.331 0.00375 0.266 Pupil Repeated Grade -1.555** 0.015 -1.721*** 0.001 -2.783** 0.018 -2.202* 0.052 -1.131*** 0.009 -1.049** 0.021 -0.0232 0.358 -0.0183 0.418 -0.0107* 0.090 -0.0140** 0.018 Pupil Age (Imputed) 0.0300 0.869 0.165 0.257 0.0650 0.783 0.0833 0.733 -0.00154 0.988 -0.0474 0.673 -0.00235 0.644 0.0257*** 0.000 0.00235 0.114 0.00452** 0.022 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -1.304 0.122 -0.804 0.268 -2.819* 0.084 -4.347** 0.023 -0.838 0.187 -0.857 0.217 -0.0203 0.345 -0.0413 0.109 -0.00732 0.205 0.00173 0.833 Dagbani 2.797 0.394 5.713** 0.042 2.519 0.607 1.469 0.702 2.181 0.124 1.700 0.214 0.210 0.159 0.0880 0.117 0.0688 0.125 0.0548** 0.039 Dangme 11.94*** 0.000 3.019*** 0.002 11.85*** 0.000 7.167*** 0.006 4.799*** 0.000 1.412 0.129 0.473*** 0.000 -0.0841** 0.017 0.125*** 0.000 0.0168 0.136 Ewe -5.275*** 0.000 -6.001*** 0.003 -14.22*** 0.000 -19.13*** 0.000 -4.775*** 0.001 -6.380*** 0.000 -0.112*** 0.000 0.0608 0.182 -0.0273** 0.013 -0.0426 0.123 Fante 0.205 0.876 -1.244 0.298 -0.626 0.789 -4.093 0.182 -0.878 0.310 -0.0525 0.953 -0.0743 0.117 0.152*** 0.000 0.0184* 0.078 -0.0189 0.253 Ga 1.639 0.322 0.288 0.799 2.251 0.352 -2.259 0.326 1.024 0.359 0.205 0.837 0.153*** 0.000 -0.0232 0.457 0.00574 0.660 -0.00355 0.776 Gonja 4.152 0.315 -2.474 0.149 -1.012 0.754 -5.124 0.110 0.855 0.500 -1.116 0.338 0.169* 0.099 0.334*** 0.000 -0.00224 0.843 -0.00368 0.815 Gurene -4.272*** 0.006 -0.496 0.653 -9.681*** 0.000 -10.43*** 0.000 -1.893** 0.032 -1.690* 0.051 0.0998** 0.025 -0.176*** 0.000 -0.0149 0.304 -0.00215 0.884 Kasem 4.435** 0.010 1.485 0.260 -1.461 0.623 -6.258** 0.022 1.336 0.220 0.00134 0.999 0.410*** 0.000 0.0992*** 0.009 0.00832 0.637 0.0220 0.197 Class Size -0.00313 0.910 0.00438 0.817 -0.0465 0.350 -0.0769* 0.065 -0.00419 0.801 1.98e-06 1.000 0.00194** 0.047 0.000817 0.274 0.000229 0.229 0.000173 0.356 Pupil Household Asset Index 1.575*** 0.000 0.768** 0.027 2.014*** 0.002 2.205*** 0.000 0.842*** 0.001 0.657*** 0.002 0.0374*** 0.004 0.0157 0.233 0.0150*** 0.002 0.00577 0.119 School Resource Index 0.129 0.508 0.159 0.333 0.316 0.282 0.200 0.520 0.0818 0.497 0.0569 0.665 0.00769 0.312 -0.00131 0.794 0.000796 0.666 0.00210 0.164 Number of Teachers at School 0.118 0.357 0.0753 0.499 0.0874 0.722 0.0332 0.892 0.134 0.144 0.0656 0.484 0.00752 0.167 0.00612 0.136 0.00185** 0.023 0.00136 0.301 Proportion of Female PI Teachers 2.291** 0.017 3.064*** 0.004 3.368** 0.030 2.700** 0.045 2.034*** 0.002 1.971*** 0.005 0.0211 0.606 0.00351 0.925 0.0232** 0.042 0.0301*** 0.002 School Library Available -0.641 0.519 -0.177 0.812 -0.764 0.640 -1.384 0.272 -0.535 0.391 -0.819 0.150 0.0142 0.673 0.00927 0.703 0.00384 0.620 -0.00166 0.839 Constant -7.488* 0.066 -10.43*** 0.007 -5.473 0.421 -1.243 0.837 -7.120*** 0.002 -4.474** 0.049 -0.259 0.142 0.124 0.208 -0.101** 0.019 -0.128** 0.018 Observations R-squared Predictors English Ghanaian English Ghanaian English Ghanaian English Ghanaian English Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) 2,065 2,065 2,065 2,065 0.086 0.244 0.342 0.421 0.137 0.195 2,065 2,065 2,065 2,064 2,065 2,065 0.133 0.201 0.067 0.115 *** p<0.01, ** p<0.05, * p<0.1 103 Figure 57: OLS Predictors of Assessment Scores Model (Pupils with Low Proficiency in Official Ghanaian LOI Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 1.190** 0.047 1.001** 0.032 1.845 0.133 2.524** 0.011 0.213 0.486 -0.111 0.681 0.0455 0.234 0.000729 0.983 0.00858 0.109 0.00209 0.632 Midline 0.728 0.192 0.107 0.710 4.027*** 0.000 1.310** 0.045 0.269 0.345 -0.157 0.482 -0.0405* 0.087 -0.00193 0.946 -0.00588 0.323 -0.00552** 0.031 Treatment * Midline 0.775 0.566 3.527*** 0.007 8.171*** 0.000 10.06*** 0.000 1.545* 0.076 2.635*** 0.001 0.0323 0.410 -0.0378 0.331 0.0107 0.374 0.00904 0.157 Rural -2.299 0.226 0.867 0.662 -0.677 0.731 0.353 0.876 -0.0931 0.915 0.308 0.746 -0.0771 0.201 -0.00369 0.930 -0.0261 0.134 -0.0175** 0.027 Pupil Gender 0.481 0.222 0.377 0.159 1.120 0.154 0.745 0.257 -0.124 0.658 -0.116 0.641 0.0128 0.345 0.000766 0.959 0.00115 0.760 0.00185 0.468 Pupil Hungry Everyday -1.095** 0.018 -0.379 0.236 -0.206 0.820 -1.058 0.102 -0.626*** 0.009 -0.439** 0.029 0.000235 0.991 -0.0100 0.633 -0.00308 0.435 -0.00319 0.156 Pupil Rewarded by Teacher 1.153** 0.047 -0.0954 0.797 0.996 0.201 0.119 0.880 0.294 0.297 0.398 0.125 -0.0488* 0.073 -0.0169 0.282 0.00521 0.225 0.00153 0.426 Length of School Day (Hours) 0.449 0.256 -0.0416 0.869 -0.401 0.519 -0.373 0.417 0.127 0.453 0.00274 0.985 -0.0117 0.417 -0.00304 0.791 0.00447 0.135 0.000349 0.873 Pupil Repeated Grade -0.151 0.883 -1.146* 0.053 -2.940** 0.044 -0.835 0.619 -0.613 0.373 -0.266 0.663 -0.0206 0.680 -0.0617** 0.017 0.00421 0.720 -0.00652 0.121 Pupil Age (Imputed) 0.304 0.140 0.142 0.309 0.144 0.596 0.0614 0.782 0.123 0.255 0.0373 0.706 -0.00274 0.578 0.0221*** 0.003 0.00216* 0.096 0.00159* 0.081 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -4.473** 0.022 -3.187** 0.031 -9.261*** 0.001 -11.29*** 0.000 -3.128*** 0.002 -3.397*** 0.004 -0.0655 0.237 -0.00501 0.936 -0.0369* 0.054 -0.0238 0.293 Dagaare -4.124* 0.077 -2.096 0.286 -11.58*** 0.000 -10.48*** 0.001 -2.794** 0.019 -3.128** 0.016 -0.0816 0.207 -0.327*** 0.000 -0.0258 0.270 -0.0228 0.326 Dagbani -3.628 0.161 -2.647 0.259 -11.21*** 0.000 -12.45*** 0.000 -2.377** 0.044 -3.126** 0.019 -0.0663 0.447 -0.199*** 0.003 -0.0189 0.481 -0.0191 0.447 Dangme 4.231 0.530 -0.0474 0.982 -1.887 0.755 -3.578 0.431 0.0301 0.992 -1.561 0.376 0.269*** 0.003 -0.210*** 0.000 0.0656 0.375 0.00501 0.872 Ewe -5.051** 0.032 -3.930** 0.016 -15.10*** 0.000 -11.53*** 0.000 -3.974*** 0.000 -3.947*** 0.001 -0.139* 0.089 -0.222*** 0.009 -0.0441** 0.034 -0.0299 0.183 Fante -3.639* 0.062 -2.581* 0.092 -5.008* 0.073 -7.837*** 0.009 -2.475** 0.018 -2.902** 0.015 -0.0871 0.128 0.0611 0.410 -0.0329 0.102 -0.0242 0.277 Ga -3.123 0.171 -0.920 0.573 -4.886* 0.064 -7.633*** 0.007 -0.881 0.415 -1.532 0.231 0.0649 0.297 -0.240*** 0.000 -0.0472** 0.039 -0.0401 0.114 Gonja 0.853 0.795 -3.807 0.174 -3.583 0.402 6.366 0.482 5.399 0.138 0.967 0.682 0.541*** 0.000 0.212** 0.013 -0.0419* 0.066 -0.0195 0.391 Gurene -5.030** 0.033 -2.702 0.133 -13.58*** 0.000 -14.45*** 0.000 -3.134*** 0.006 -3.470*** 0.008 -0.0801 0.245 -0.384*** 0.000 -0.0290 0.236 -0.0266 0.291 Kasem -4.517** 0.039 -4.354** 0.017 -14.07*** 0.000 -17.11*** 0.000 -3.427*** 0.002 -4.085*** 0.002 -0.0506 0.546 -0.212*** 0.000 -0.0293 0.206 -0.0199 0.400 Class Size 0.0107 0.434 0.0161 0.213 0.0224 0.317 0.0170 0.460 0.00362 0.603 0.00814 0.285 0.000255 0.573 0.000788 0.208 0.000161* 0.099 3.51e-05 0.398 Pupil Household Asset Index 0.698 0.301 0.684* 0.099 2.083** 0.041 1.886*** 0.010 0.519 0.109 0.324 0.204 0.0466* 0.076 0.0374* 0.084 0.00834 0.267 0.00181 0.524 School Resource Index 0.00699 0.977 -0.243 0.249 -0.0529 0.855 -0.177 0.494 -0.173* 0.083 -0.0593 0.511 0.00280 0.761 -0.00423 0.609 -0.000372 0.884 0.000546 0.604 Number of Teachers at School 0.314* 0.075 0.109 0.403 0.898*** 0.001 0.711*** 0.001 0.146* 0.059 0.0726 0.324 0.00997 0.222 0.0154* 0.054 0.00107 0.411 0.00140 0.102 Proportion of Female PI Teachers 1.364* 0.063 1.539** 0.016 2.728*** 0.004 1.461 0.124 0.988*** 0.003 0.791** 0.015 -0.0228 0.369 0.0304 0.328 0.0137* 0.064 0.00209 0.501 School Library Available 0.721 0.583 1.750 0.137 1.259 0.477 1.712 0.276 0.362 0.567 0.429 0.517 0.0376 0.386 0.0334 0.367 0.00699 0.502 0.00538 0.268 Constant -0.890 0.827 -1.855 0.613 4.338 0.502 6.943 0.187 0.517 0.790 1.218 0.571 0.259 0.134 0.0668 0.672 0.00588 0.877 0.0255 0.529 Observations R-squared Predictors English Ghanaian English Ghanaian English Ghanaian English Ghanaian English Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) 1,242 *** p<0.01, ** p<0.05, * p<0.1 1,242 1,242 1,242 1,240 1,242 0.170 0.167 0.325 0.303 0.144 0.240 0.224 0.097 0.059 1,241 1,242 1,242 1,242 0.135 104 Figure 58: OLS Predictors of Assessment Scores Model (Pupils with Medium Proficiency in Official Ghanaian LOI Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 0.459 0.785 2.334 0.343 0.271 0.854 0.263 0.813 0.153 0.699 0.196 0.695 0.00768 0.751 -0.0219 0.464 -0.00522 0.301 -0.000820 0.867 Midline -0.932 0.235 0.145 0.751 4.189*** 0.000 0.851 0.368 0.423 0.256 0.138 0.690 0.00707 0.767 -0.0527* 0.054 0.000752 0.920 -0.00401 0.321 Treatment * Midline 1.369 0.439 4.165 0.118 15.14*** 0.000 21.17*** 0.000 3.247*** 0.000 4.471*** 0.000 -0.0584* 0.056 0.171*** 0.000 0.00230 0.777 0.0584*** 0.000 Rural 0.809 0.388 0.506 0.579 -0.426 0.784 0.0251 0.986 -0.0468 0.936 -0.0235 0.970 -0.0467* 0.099 -0.0348 0.281 0.00403 0.448 0.00469 0.593 Pupil Gender 0.936 0.183 1.628*** 0.002 2.055*** 0.004 2.419*** 0.001 0.827*** 0.004 1.026*** 0.004 -0.0161 0.173 -0.0218 0.176 -0.000513 0.880 0.00725* 0.088 Pupil Hungry Everyday -1.131* 0.076 -1.209 0.102 -1.596* 0.077 -1.365 0.111 -0.266 0.525 -0.306 0.452 0.00309 0.884 -0.0156 0.483 -0.00540 0.158 0.00239 0.680 Pupil Rewarded by Teacher 1.401 0.140 1.627 0.227 1.701 0.135 1.810 0.112 0.672 0.128 0.571 0.319 0.0185 0.346 0.0930*** 0.004 -0.00235 0.640 0.00624 0.462 Length of School Day (Hours) -0.780 0.343 -0.960 0.375 0.532 0.355 0.678 0.195 0.0768 0.714 0.00761 0.977 -0.00944 0.347 -0.00685 0.570 0.00131 0.546 0.000717 0.814 Pupil Repeated Grade -0.873 0.498 -2.570* 0.058 -2.010 0.212 -1.934 0.234 -1.318** 0.044 -1.264* 0.077 -0.000750 0.978 0.0105 0.760 0.00274 0.738 -0.0128 0.192 Pupil Age (Imputed) 0.0956 0.747 0.519 0.164 0.571** 0.041 0.248 0.289 0.0810 0.477 0.188 0.109 -2.87e-06 0.999 0.00908 0.114 0.00115 0.305 0.00188 0.292 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -1.861* 0.097 -1.649* 0.096 -5.347*** 0.003 -7.095*** 0.000 -1.572** 0.013 -1.834*** 0.005 -0.0179 0.452 0.0152 0.678 -0.0111 0.153 -0.00363 0.595 Dagbani -4.347** 0.018 -2.062 0.223 -4.844 0.207 -8.221** 0.023 -0.855 0.509 -1.728 0.151 0.161 0.134 -0.286*** 0.000 -0.0154 0.155 -0.0183* 0.092 Dangme -0.547 0.809 -0.394 0.840 -0.199 0.972 -2.943 0.410 -2.016* 0.075 -3.215** 0.032 0.142 0.118 -0.164* 0.076 0.0201 0.364 -0.0115 0.637 Ewe -2.306 0.105 -1.438 0.213 -5.278* 0.063 -6.735** 0.011 -1.307 0.262 -1.939** 0.048 0.0163 0.699 0.0755 0.314 -0.0109 0.191 -0.0157* 0.068 Fante -0.996 0.474 -0.234 0.853 -1.644 0.473 -3.833 0.110 -0.256 0.756 -0.0213 0.981 -0.0424* 0.092 0.198*** 0.000 -0.0118 0.202 0.00538 0.633 Ga 9.096** 0.034 6.368 0.105 2.226 0.572 2.019 0.659 3.390 0.164 3.227 0.220 0.226*** 0.001 -0.0643 0.382 0.0649** 0.045 0.0432 0.203 Gonja -0.205 0.926 -5.063** 0.040 9.192*** 0.003 6.673** 0.026 -0.757 0.508 1.006 0.570 0.343*** 0.002 0.0482 0.662 0.0277* 0.056 -0.0324** 0.050 Gurene -4.434 0.107 -2.474 0.348 -3.999 0.330 -7.586* 0.079 -2.393 0.129 -2.309 0.194 0.183** 0.011 -0.183** 0.027 -0.0308 0.125 -0.0148 0.533 Class Size 0.0428* 0.058 0.0317 0.202 0.0421 0.313 0.0181 0.645 0.0273* 0.073 0.0252 0.148 0.00126* 0.092 0.000473 0.564 0.000240 0.118 0.000337 0.115 Pupil Household Asset Index 1.086** 0.030 -0.271 0.688 1.960*** 0.006 1.308* 0.054 0.475* 0.096 0.319 0.339 0.0119 0.415 0.0243 0.146 0.000741 0.866 0.000569 0.902 School Resource Index -0.138 0.412 0.0595 0.767 -0.422 0.137 -0.279 0.266 -0.0901 0.344 -0.0529 0.599 -0.00227 0.569 0.00873 0.111 -0.00103 0.177 -0.00184 0.159 Number of Teachers at School 0.135 0.421 0.317** 0.035 0.536** 0.033 0.637*** 0.002 0.201** 0.026 0.185* 0.057 0.00184 0.651 0.00302 0.617 0.000115 0.856 0.00383*** 0.004 Proportion of Female PI Teachers 1.749* 0.060 2.187 0.100 3.622*** 0.002 3.326*** 0.002 1.106*** 0.008 1.075** 0.021 0.0255 0.132 -0.0339 0.190 0.00554* 0.067 0.00975 0.128 School Library Available 1.441 0.293 0.164 0.903 3.384** 0.050 4.440** 0.023 1.281* 0.086 0.883 0.263 0.0519 0.152 0.0788** 0.013 0.0201** 0.043 0.0111 0.323 Constant 1.359 0.851 -5.539* 0.064 -7.063 0.284 -7.144 0.252 -3.775* 0.098 -4.114 0.111 0.170* 0.084 0.189 0.149 -0.0129 0.479 -0.0671* 0.053 Observations R-squared Predictors Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian English 1,581 1,581 1,581 1,581 0.068 0.062 0.362 0.458 0.163 0.152 1,581 1,580 1,578 1,577 1,578 1,578 0.169 0.111 0.094 0.119 *** p<0.01, ** p<0.05, * p<0.1 105 Figure 59: OLS Predictors of Assessment Scores Model (Pupils with High Proficiency in Official Ghanaian LOI Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 1.128* 0.064 0.774 0.264 2.205*** 0.005 1.263* 0.054 0.144 0.525 0.407 0.236 0.0109 0.428 0.0172 0.352 0.00458* 0.072 0.000122 0.962 Midline -0.116 0.657 0.304 0.168 4.424*** 0.000 1.455*** 0.003 0.0975 0.592 -0.0882 0.627 0.0100 0.517 0.0599*** 0.001 0.00341* 0.054 -0.000809 0.683 Treatment * Midline 0.752 0.314 5.532*** 0.000 13.37*** 0.000 19.39*** 0.000 3.044*** 0.000 4.071*** 0.000 -0.0387** 0.039 0.0398 0.105 0.000602 0.870 0.0504*** 0.000 Rural -0.986** 0.048 -0.356 0.460 -1.551** 0.040 -1.630* 0.062 -0.732** 0.014 -0.667* 0.064 -0.0367*** 0.006 -0.00734 0.622 -0.00803*** 0.003 -0.00367 0.442 Pupil Gender 0.0259 0.930 0.163 0.532 1.798*** 0.000 1.432*** 0.000 0.321** 0.034 0.229 0.137 -0.000119 0.981 -0.0195** 0.043 0.00153 0.394 -0.000344 0.872 Pupil Hungry Everyday -0.980*** 0.005 -0.845*** 0.004 -1.855*** 0.000 -1.566*** 0.001 -0.393** 0.026 -0.542*** 0.010 -0.00540 0.496 -0.0182 0.174 -0.00333* 0.072 -0.00637*** 0.002 Pupil Rewarded by Teacher 0.504 0.344 1.010*** 0.001 0.634 0.267 0.867 0.115 0.394** 0.037 0.623*** 0.003 0.0202** 0.022 0.0701*** 0.000 0.00519*** 0.002 0.00775** 0.010 Length of School Day (Hours) 0.133 0.624 0.103 0.731 0.610 0.105 0.496 0.184 0.241* 0.070 0.167 0.343 0.00784 0.189 0.00732 0.306 0.000608 0.660 0.00299 0.193 Pupil Repeated Grade -0.805** 0.048 -0.133 0.788 -1.821*** 0.006 -1.844*** 0.008 -0.245 0.388 -0.301 0.336 -0.0244** 0.029 -0.0140 0.446 -0.00409 0.179 0.00643 0.311 Pupil Age (Imputed) -0.00810 0.948 0.0118 0.925 -0.219 0.152 -0.0783 0.630 -0.0854 0.158 -0.0650 0.329 0.000932 0.701 0.0166*** 0.000 2.81e-05 0.970 -7.26e-05 0.941 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -1.131** 0.048 -0.702 0.138 -3.853*** 0.003 -6.909*** 0.000 -0.872** 0.036 -0.693 0.117 0.0129 0.344 -0.0274 0.184 -0.00244 0.406 0.00312 0.520 Dagaare -2.922*** 0.003 -2.370** 0.028 -9.739*** 0.000 -11.17*** 0.000 -1.602** 0.014 -1.804** 0.025 0.0271 0.522 -0.0294 0.750 -0.000244 0.947 -0.00671 0.380 Dagbani 2.478 0.324 4.113 0.237 1.387 0.719 -2.572 0.498 1.191 0.364 1.230 0.403 0.231* 0.052 0.142** 0.036 0.0664 0.139 0.0463 0.176 Dangme 0.775 0.417 1.582* 0.057 -0.164 0.947 -2.895 0.205 0.000734 0.999 -0.181 0.791 0.0779 0.100 -0.0630** 0.019 0.00562 0.455 0.0136* 0.075 Ewe -0.550 0.444 1.549** 0.028 -1.813 0.256 -4.559** 0.010 -0.185 0.704 -0.217 0.685 0.0179 0.310 0.106*** 0.000 0.00107 0.750 0.0166** 0.025 Fante -0.801 0.213 -1.090* 0.050 -1.922 0.182 -4.602*** 0.008 -0.678 0.130 -0.119 0.813 -0.0103 0.485 0.174*** 0.000 0.00276 0.519 -0.00738 0.159 Ga 1.339 0.272 -0.918 0.192 1.323 0.577 -5.161** 0.035 0.505 0.646 -0.303 0.745 0.167*** 0.000 0.0417 0.170 0.00443 0.533 -0.00783 0.464 Gonja 1.069 0.698 -0.558 0.439 -2.607 0.321 -6.778*** 0.003 0.745 0.469 -0.123 0.875 0.148 0.122 0.127* 0.094 0.00132 0.821 0.00750 0.328 Gurene -2.095*** 0.006 0.147 0.828 -6.642*** 0.000 -6.933*** 0.004 -0.822 0.114 -0.465 0.439 -0.0550*** 0.002 -0.253*** 0.000 0.000428 0.911 0.0106* 0.076 Kasem 0.598 0.424 -0.264 0.692 -7.931*** 0.000 -9.101*** 0.000 -0.523 0.317 -0.0949 0.863 -0.0155 0.476 0.129*** 0.000 0.00638 0.219 0.00647 0.428 Class Size 0.00119 0.913 -0.000535 0.964 0.0105 0.662 -0.000477 0.985 -0.00640 0.449 -0.00556 0.543 0.000266 0.530 -0.000564 0.219 2.64e-05 0.762 -9.62e-05 0.496 Pupil Household Asset Index 0.655*** 0.010 0.590** 0.032 1.748*** 0.000 1.839*** 0.000 0.628*** 0.000 0.658*** 0.000 0.0113* 0.094 0.0213** 0.018 0.00520** 0.014 0.00817*** 0.004 School Resource Index 0.0527 0.566 0.0670 0.438 0.215 0.192 0.102 0.582 0.0380 0.559 0.0328 0.636 0.00185 0.541 0.000977 0.767 0.000189 0.744 0.000773 0.393 Number of Teachers at School 0.128 0.112 0.0122 0.815 -0.0783 0.510 -0.0391 0.730 0.0297 0.499 0.0348 0.416 0.000154 0.944 0.00129 0.624 0.000271 0.472 -0.000326 0.598 Proportion of Female PI Teachers 1.223*** 0.001 0.718 0.171 1.755*** 0.009 1.494** 0.024 0.565*** 0.007 0.651** 0.012 0.0255** 0.016 0.0174 0.236 0.00381** 0.049 0.00511 0.285 School Library Available -0.138 0.750 0.767 0.144 1.162 0.161 1.115 0.167 0.267 0.333 0.0254 0.934 -0.00846 0.496 0.00595 0.692 -0.000978 0.732 0.00900 0.116 Constant 1.111 0.594 -2.074 0.294 2.726 0.474 5.108 0.203 -0.214 0.881 -0.338 0.827 0.000329 0.996 0.124* 0.094 -0.00587 0.736 -0.0313 0.197 Observations R-squared Predictors Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian English 6,016 6,016 6,016 6,016 0.036 0.103 0.343 0.397 0.107 0.107 6,016 6,016 6,014 6,013 6,013 6,015 0.076 0.102 0.039 0.090 *** p<0.01, ** p<0.05, * p<0.1 106 Figure 60: OLS Predictors of Assessment Scores Model (Female Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 1.092** 0.033 0.720 0.247 1.797** 0.027 1.068* 0.094 0.153 0.473 0.275 0.418 0.0112 0.422 0.00683 0.709 0.00198 0.415 -0.00175 0.489 Midline 0.328 0.330 0.272 0.238 4.277*** 0.000 1.275** 0.014 0.280 0.226 0.00707 0.972 -0.000245 0.985 0.0301* 0.067 0.00227 0.480 -0.00165 0.456 Treatment * Midline 0.374 0.598 5.992*** 0.000 13.85*** 0.000 19.93*** 0.000 3.163*** 0.000 4.338*** 0.000 -0.0339** 0.049 0.0500** 0.046 0.00265 0.571 0.0525*** 0.000 Rural -1.342** 0.021 -0.411 0.457 -1.367 0.128 -1.210 0.220 -0.958** 0.012 -0.616 0.123 -0.0523*** 0.001 0.00616 0.728 -0.00747* 0.068 -0.00682 0.218 Pupil's Proficiency in Ghanaian LOI (Base = High Proficiency) Medium Proficiency 1.194* 0.063 1.594** 0.024 0.455 0.471 0.764 0.217 0.591** 0.048 0.687** 0.023 0.00623 0.551 -0.0590*** 0.000 0.00237 0.356 0.00955** 0.015 Low Proficiency 0.335 0.653 -0.453 0.325 -1.600 0.164 -1.729* 0.077 -0.627 0.166 -0.795* 0.055 -0.00185 0.917 -0.180*** 0.000 0.00254 0.754 -9.20e-05 0.987 Pupil Hungry Everyday -1.034*** 0.007 -0.821*** 0.009 -1.662*** 0.004 -1.455*** 0.003 -0.418** 0.044 -0.518** 0.018 0.00581 0.590 -0.0296** 0.041 -0.00262 0.268 -0.00572** 0.016 Pupil Rewarded by Teacher 0.990*** 0.003 0.710* 0.051 0.455 0.495 0.433 0.474 0.189 0.399 0.336 0.185 0.00750 0.525 0.0647*** 0.000 0.00359 0.107 0.00690** 0.020 Length of School Day (Hours) 0.168 0.502 0.120 0.694 0.583 0.135 0.428 0.218 0.275* 0.053 0.116 0.506 0.00379 0.561 0.0101 0.195 0.00189 0.318 0.00210 0.388 Pupil Repeated Grade -1.148** 0.017 -0.976** 0.035 -2.863*** 0.000 -2.694*** 0.001 -0.607* 0.068 -0.638* 0.078 -0.0251* 0.088 -0.0217 0.236 -0.00167 0.709 -0.00217 0.668 Pupil Age (Imputed) 0.0389 0.755 -0.0551 0.610 -0.195 0.265 -0.195 0.274 -0.0795 0.260 -0.0751 0.324 0.000136 0.960 0.0142*** 0.001 0.00184* 0.054 0.00102 0.357 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -1.819** 0.014 -1.067* 0.062 -4.989*** 0.001 -7.525*** 0.000 -1.165** 0.012 -1.112** 0.019 -0.00410 0.791 -0.0389* 0.096 -0.00803 0.175 0.00111 0.846 Dagaare -2.860*** 0.007 -1.800 0.103 -9.551*** 0.000 -10.35*** 0.000 -1.642** 0.022 -1.792** 0.029 0.0288 0.477 -0.0302 0.703 -0.00781 0.319 -0.00328 0.723 Dagbani -1.874 0.190 -0.259 0.866 -7.664*** 0.001 -9.687*** 0.000 -0.438 0.545 -0.722 0.294 0.0378 0.491 -0.0357 0.480 0.00615 0.730 0.0108 0.564 Dangme 1.167 0.580 1.137 0.338 -1.255 0.639 -3.095 0.246 -0.567 0.617 -0.879 0.369 0.0723 0.113 -0.107*** 0.001 0.0132 0.499 0.00482 0.690 Ewe -0.236 0.787 2.143*** 0.007 -1.548 0.432 -4.068** 0.038 -0.279 0.611 -0.371 0.528 0.0114 0.539 0.107*** 0.001 -0.00241 0.733 0.0246** 0.010 Fante -1.316* 0.096 -1.447** 0.016 -2.145 0.208 -4.896*** 0.009 -0.936* 0.073 -0.566 0.286 -0.0201 0.237 0.181*** 0.000 -0.00489 0.471 -0.00668 0.284 Ga 1.341 0.320 0.429 0.687 0.370 0.869 -4.448** 0.033 0.711 0.444 0.145 0.866 0.153*** 0.000 -0.0558 0.115 0.00459 0.620 -0.0120 0.131 Gonja 2.418 0.515 -1.076 0.183 -3.853 0.158 -6.468** 0.014 0.921 0.429 -0.837 0.345 0.232** 0.043 0.121 0.132 0.0109 0.345 0.00426 0.681 Gurene -2.420** 0.049 -0.629 0.490 -8.166*** 0.000 -8.894*** 0.000 -0.844 0.170 -0.812 0.180 -0.0339 0.285 -0.215*** 0.000 -0.00612 0.568 0.00689 0.521 Kasem -1.759 0.114 -3.750*** 0.007 -14.72*** 0.000 -19.04*** 0.000 -2.604*** 0.001 -2.926*** 0.002 0.0221 0.682 -0.104 0.146 0.0105 0.495 -0.0169 0.160 Class Size 0.0127 0.234 0.0120 0.220 0.0338 0.112 0.0202 0.326 0.00260 0.725 0.00673 0.382 0.000195 0.597 0.000145 0.775 2.34e-05 0.746 5.77e-05 0.555 Pupil Household Asset Index 0.650*** 0.009 0.409 0.191 1.503*** 0.001 1.447*** 0.000 0.432** 0.015 0.429** 0.021 0.0186** 0.033 0.0248** 0.011 0.00382 0.190 0.00701** 0.023 School Resources Index 0.00923 0.928 0.0437 0.664 0.0447 0.794 0.0135 0.940 -0.019 0.776 0.0163 0.814 0.00184 0.551 0.00174 0.627 -0.000485 0.574 0.000215 0.826 Number of Teachers at School 0.0572 0.374 0.0312 0.641 0.0326 0.836 0.0791 0.569 0.037 0.509 0.0310 0.587 0.000723 0.749 0.00278 0.298 0.000321 0.539 9.12e-05 0.899 Proportion of Female PI Teachers 1.607*** 0.000 1.342** 0.010 2.877*** 0.000 2.643*** 0.000 0.950*** 0.000 0.967*** 0.001 0.0152 0.171 0.0177 0.271 0.00780*** 0.003 0.00922** 0.036 School Library Available 0.509 0.362 0.734 0.230 1.704 0.113 1.209 0.241 0.560 0.122 0.103 0.784 0.00543 0.716 0.0120 0.515 0.00510 0.278 0.00686 0.270 Constant 1.127 0.611 -1.428 0.525 6.172 0.123 8.121** 0.033 0.828 0.568 0.752 0.630 0.0793 0.235 0.0531 0.528 -0.0168 0.383 -0.0318 0.260 Observations R-squared Predictors Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian English 4,400 4,400 4,400 4,400 0.074 0.127 0.349 0.418 0.123 0.118 4,400 4,399 4,397 4,396 4,396 4,397 0.109 0.183 0.030 0.095 *** p<0.01, ** p<0.05, * p<0.1 107 Figure 61: OLS Predictors of Assessment Scores Model (Male Subgroup) Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Treatment 0.654 0.422 0.992 0.327 1.320* 0.067 0.952 0.116 -0.0262 0.901 0.134 0.616 0.006 0.705 -0.00158 0.933 0.000779 0.813 -0.000455 0.857 Midline -0.578* 0.060 0.119 0.478 4.071*** 0.000 1.283*** 0.005 0.0198 0.913 -0.201 0.220 -0.010 0.480 0.0259 0.169 -0.00157 0.515 -0.00286 0.136 Treatment * Midline 1.441 0.136 4.437*** 0.000 12.49*** 0.000 17.38*** 0.000 2.773*** 0.000 3.806*** 0.000 -0.018 0.360 0.0687*** 0.008 0.00563 0.229 0.0419*** 0.000 Rural -0.430 0.503 0.00866 0.990 -1.863** 0.018 -1.886** 0.026 -0.354 0.259 -0.477 0.234 -0.0332* 0.091 -0.0450*** 0.009 -0.00936** 0.038 -0.00601 0.183 Pupil's Proficiency in Ghanaian LOI (Base = High Proficiency) Medium Proficiency 0.218 0.655 0.0746 0.788 0.290 0.681 -0.167 0.802 0.201 0.371 0.0313 0.902 0.0227* 0.090 -0.0717*** 0.000 0.00680** 0.013 0.000872 0.786 Low Proficiency -0.549 0.363 -0.928* 0.084 -2.162** 0.027 -2.264** 0.029 -0.156 0.659 -0.377 0.289 -0.033 0.200 -0.189*** 0.000 -0.00263 0.681 -0.00873 0.236 Pupil Hungry Everyday -1.199*** 0.001 -0.862** 0.021 -1.206** 0.030 -1.204** 0.016 -0.432** 0.020 -0.419** 0.047 -0.013 0.195 -0.000655 0.960 -0.00672** 0.016 -0.00309 0.188 Pupil Rewarded by Teacher 0.385 0.560 0.813** 0.035 1.339** 0.022 1.360** 0.026 0.620*** 0.002 0.746*** 0.002 0.002 0.839 0.0527*** 0.000 0.00373* 0.075 0.00598* 0.072 Length of School Day (Hours) -0.0342 0.921 -0.279 0.465 0.124 0.762 0.0287 0.938 0.0894 0.466 0.0593 0.676 0.005 0.557 -0.00699 0.389 0.00147 0.388 0.000700 0.687 Pupil Repeated Grade -0.308 0.627 -0.343 0.611 -0.744 0.409 -0.274 0.773 -0.289 0.369 -0.198 0.611 -0.00990 0.536 0.00189 0.928 -0.00108 0.799 0.00579 0.410 Pupil Age (Imputed) 0.0808 0.655 0.277 0.262 0.151 0.340 0.127 0.434 0.0238 0.732 0.0527 0.516 0.00125 0.686 0.0193*** 0.000 -0.000106 0.913 -4.02e-06 0.997 Official Language of Instruction (Base = Akuapim Twi) Asante Twi -1.045 0.128 -0.918 0.116 -4.542*** 0.001 -7.386*** 0.000 -1.188*** 0.008 -1.226** 0.015 0.00981 0.552 -0.0128 0.603 -0.00336 0.439 -0.00111 0.827 Dagaare -2.539** 0.016 -2.019 0.101 -8.805*** 0.000 -9.108*** 0.001 -1.616** 0.011 -1.878** 0.012 -0.000262 0.994 -0.104 0.135 0.00163 0.805 -0.00753 0.307 Dagbani -0.615 0.694 -0.439 0.772 -5.324* 0.073 -8.116*** 0.005 -0.651 0.482 -0.962 0.319 0.0769 0.327 -0.113*** 0.009 0.0164 0.402 0.000420 0.968 Dangme 1.497 0.223 1.435* 0.089 0.953 0.719 -2.340 0.227 0.534 0.366 -0.289 0.623 0.141* 0.051 -0.0446 0.174 0.0199* 0.099 0.0228* 0.067 Ewe -1.329 0.126 0.277 0.749 -3.341** 0.031 -5.764*** 0.001 -0.586 0.291 -0.911 0.143 0.0106 0.614 0.0920*** 0.005 -0.00193 0.632 0.00421 0.593 Fante -0.621 0.440 -0.707 0.283 -2.158 0.137 -4.519*** 0.006 -0.580 0.269 -0.221 0.719 -0.0201 0.226 0.163*** 0.000 0.00256 0.703 -0.00536 0.302 Ga 2.111 0.141 0.0311 0.979 -0.633 0.737 -5.831*** 0.005 0.363 0.672 -0.350 0.692 0.198*** 0.000 -0.0744* 0.063 0.00768 0.540 -0.00524 0.677 Gonja -0.952 0.549 -1.749** 0.047 -1.094 0.802 -5.534* 0.096 -0.0868 0.930 -0.138 0.882 0.118 0.229 0.0715 0.276 -0.00909 0.186 -0.00690 0.331 Gurene -1.998 0.111 -0.311 0.776 -4.434** 0.033 -7.172*** 0.001 -0.941 0.162 -0.870 0.227 0.00179 0.968 -0.172*** 0.000 0.00222 0.837 0.00769 0.354 Kasem -0.790 0.492 -1.692* 0.073 -11.34*** 0.000 -14.60*** 0.000 -1.493** 0.037 -2.035*** 0.005 0.0217 0.620 -0.104** 0.029 0.00220 0.819 0.000169 0.987 Class Size 0.0102 0.348 0.0120 0.320 0.0160 0.472 0.0114 0.629 0.00462 0.527 0.00626 0.459 0.000546 0.216 -3.58e-05 0.930 0.000120 0.166 1.02e-05 0.913 Pupil Household Asset Index 0.979*** 0.004 0.499 0.148 2.391*** 0.000 2.273*** 0.000 0.775*** 0.000 0.704*** 0.000 0.0213** 0.028 0.0283*** 0.006 0.00803** 0.021 0.00511* 0.063 School Resources Index -0.0160 0.886 0.0203 0.842 0.0744 0.662 -0.0302 0.857 -0.0117 0.838 -0.00166 0.979 0.00170 0.681 -0.000759 0.839 -3.85e-05 0.966 0.000257 0.748 Number of Teachers at School 0.217* 0.057 0.0824 0.172 0.126 0.234 0.124 0.249 0.0809* 0.053 0.0810* 0.064 0.00178 0.462 -0.000135 0.967 0.000198 0.639 0.000379 0.512 Proportion of Female PI Teachers 1.145** 0.026 1.040 0.121 1.592*** 0.010 1.092* 0.076 0.590*** 0.005 0.602** 0.015 0.0255* 0.054 0.00422 0.782 0.00512* 0.078 0.00249 0.522 School Library Available -0.0145 0.981 1.019 0.118 1.166 0.183 1.840** 0.036 0.291 0.380 0.302 0.404 0.00983 0.658 0.0277 0.123 0.00261 0.626 0.0100** 0.049 Constant 0.204 0.950 -2.269 0.343 4.699 0.279 7.967* 0.060 -0.782 0.587 -0.692 0.656 0.00279 0.976 0.278*** 0.002 -0.00660 0.749 -0.00425 0.805 Observations R-squared Predictors Ghanaian Letter Sounds (CLPM) Invent Word (CWPM) Oral Reading Fluency (CWPM) Listening Comprehension Score (%) Reading Comprehension Score (%) English Ghanaian English Ghanaian English Ghanaian English Ghanaian English 4,439 4,439 4,439 4,439 0.037 0.058 0.331 0.376 0.107 0.110 4,439 4,439 4,437 4,434 4,437 4,437 0.109 0.194 0.039 0.070 *** p<0.01, ** p<0.05, * p<0.1 108 Figure 62: OLS Predictors of FOI Index and Sub-Indices Predictors FOI Index Performance Sub-Index Materials Sub-Index Training Sub-Index Coeff. Sig. Coeff. Sig. Coeff. Sig. Coeff. Sig. Number of Teachers 0.0594 0.465 0.0363** 0.033 0.0632 0.160 -0.0401 0.444 Length of School Day Hours 0.377* 0.099 0.122** 0.019 0.266** 0.032 -0.00998 0.945 School Resources Index 0.160** 0.046 0.00977 0.626 0.0744 0.106 0.0755 0.131 Teacher Received a College Education or Higher 0.394 0.250 0.0226 0.801 -0.0515 0.796 0.423** 0.035 Number of Dropouts -0.153** 0.027 0.0237 0.304 -0.0742* 0.084 -0.102*** 0.001 Baseline: Pupil Read Books Recently 0.231 0.633 -0.00663 0.960 0.134 0.617 0.104 0.727 Baseline: Pupil Punished by Teacher 0.954 0.153 -0.0980 0.660 -0.312 0.523 1.364*** 0.000 Baseline: Pupil Rewarded by Teacher 0.0798 0.904 -0.0495 0.806 -0.403 0.270 0.532 0.230 Baseline: School has Sufficient Resources -0.970 0.199 0.0912 0.563 -0.269 0.445 -0.792 0.208 Proportion of Female Pupils 1.921* 0.077 0.644 0.126 -0.137 0.872 1.414* 0.070 Average Pupil Household Asset Index 1.056*** 0.005 0.178 0.123 0.603*** 0.009 0.275 0.289 Female Teacher -0.236 0.446 0.0525 0.523 0.0801 0.666 -0.369* 0.057 Constant 12.95*** 0.000 3.232*** 0.000 3.637*** 0.000 6.082*** 0.000 Observations 233 233 233 233 R-squared 0.200 0.106 0.149 0.160 109 ANNEX V. OUTCOMES AND MEASUREMENT FRAMEWORK Data Source Illustrative Indicators / Topics Timeline Administrative Data Program monitoring data to be collected by Learning Midline and Endline (treatment only):  Number of Master Trainers (CSs and LSs) trained in sampled school areas.  Names and attendance records of sampled school teachers, HTs, and CLs reached through cascade training model.  Numbers of learner books and classrooms kits provided to sampled schools and delivery dates.  Frequency of CS monitoring visits in program schools and classrooms. Midline Endline Head Teacher Interview Structured interviews with HTs at target schools Anticipated duration: 45 minutes Baseline, Midline, and Endline (treatment and comparison):  Teacher numbers and attendance data for sampled P1 and P2 teachers.  Language background of head teacher, teachers, and pupils.  School resources, reading resources, and in-service professional development.  Information on pupil enrolments, attendance, tardiness, repetition, and dropouts.  Beliefs on current models of instruction for improving early grade reading.  Frequency and content of HT coaching and (if appropriate) extent to which HTs feel coaching is positively received by teachers and has an impact on pupil learning.  Extent to which teachers are engaging in continuous assessment and how assessment data is used to inform lesson planning. Midline and Endline (treatment only):  Extent to which teachers are employing new curricula and materials.  Challenges teachers and learners face in using new curricula and materials.  Frequency and content of coaching and SBI meetings as well as challenges faced in implementation.  Frequency of CS monitoring and LS coaching visits.  How monitoring data is being used to improve instructional practice. Baseline Midline Endline Midline Endline 110 Teacher Interview Structured interviews with P1 and P2 teachers Anticipated duration: 45 minutes Baseline and Endline (treatment and comparison), Midline (P1 treatment only):  Review of teacher guides and lesson plans and current scope and sequence position.  Methods and frequency of continuous assessment and (if appropriate) how teacher modifies approach in response to assessment data.  Availability of learner textbooks, activity books, and supplementary materials in the classroom and extent of learner access and use.  Teacher knowledge, attitudes, and practices on efficacy of phonics-based instruction vis-à-vis other methods (e.g., whole language).  Teacher and pupil language skills.  Classroom enrolments and attendance (including tardiness). Midline and Endline (treatment only):  Challenges teachers and learners face in using new curricula and materials.  Frequency, content, and perceived utility of HT/CL coaching and SBI meetings.  Frequency, content, and perceived utility of CS monitoring and LS coaching visits.  How monitoring and coaching feedback is used to improve instructional practice. Baseline Midline Endline Midline Endline Pupil Interview Structured interview with P1 and P2 pupils Anticipated duration: 10 minutes Baseline, Midline (P1 only), and Endline (treatment and comparison):  Methods and frequency of pupil assessment.  Availability books/readers at school and for home use (and extent of use).  Extent to which child is read to at home and has help with homework.  Language skills, including first language, language(s) spoken at home, language(s) spoken on the playground, and language(s) used in the classroom.  Self-reported household assets (for socio-economic index). Baseline Midline Endline EGRA Ghanaian language and English subtasks Anticipated duration: 20 minutes Baseline, Midline (P1 only), and Endline (treatment and comparison): • Initial reading/decoding skills (letter-sound and non￾word reading) and fluency and comprehension (oral passage reading, reading comprehension) in LoI and English for P1 and P2. Baseline Midline Endline 111 Stallings Observation System Protocol examining teacher and learner time-on￾task in P1 and P2 classrooms Anticipated duration: 90 minutes Baseline and Endline (treatment and comparison), Midline (P1 treatment only):  Teacher and learner time-on-task during a reading lesson.  Use of materials throughout reading lesson including textbooks, notebooks, learning aides, etc.  Distribution of class time across the following categories: reading aloud, lecture, discussion/debate, practice/drill, assignment/class work, copying, verbal instruction, and classroom management.  Classroom climate including disruption and teacher use of discipline. Baseline Midline Endline Structured Observations Structured observations of reading lessons Anticipated duration: varies Baseline and Endline (treatment and comparison): • Availability of learner materials/books and whether or not teachers use lesson plans, scripted lesson plans, and/or guidebooks. • Teacher use of best practices in teaching reading including assessing pupil understanding, connecting lessons to prior learnings, encouraging learners to “sound out” words they don’t know, etc. • Language of instruction as well as non-LoI languages used/spoken by teachers and learners. Midline and Endline (treatment only): • Extent to which teachers are adhering to the scripted lesson plans in terms of schedule, timing, activities, steps, and materials usage (FOI) Baseline Endline Midline Endline CL Interview Structured interviews with CLs Anticipated duration: 30 minutes Midline only (treatment only): • Extent to which teachers are employing new curricula and materials. • Challenges teachers and learners face in using new curricula and materials. • Frequency and content of coaching and SBI meetings as well as challenges faced in implementation. • Extent to which CLs feel their coaching is positively received by teachers and has an impact on pupil learning. • Extent to which teachers are engaging in continuous assessment and how assessment data is used to inform lesson planning. • How monitoring data is being used to improve instructional practice. Midline 112 Circuit Supervisor Interview Structured and semi-structured interviews with Circuit Supervisors Anticipated duration: 45 minutes Baseline and Endline (treatment and comparison): • Extent to which Circuit Supervisors feel equipped to engage in their monitoring duties. • Extent to which Circuit Supervisors feel their monitoring is positively received by teachers and has an impact on pupil learning. • Extent to which teachers are engaging in continuous assessment and how assessment data is used to inform lesson planning. Midline only (treatment only): • Extent to which teachers are employing new curricula and materials. • Challenges teachers and learners face in using new curricula and materials. • Regularly of, and challenges with, recording and transmitting monitoring data using tablets. • How monitoring data is being used to improve instructional practice. • Procedures for and frequency of coordinating DTST LS visits. Baseline Endline Midline 113 ANNEX VI. DATA COLLECTION TOOLS AND CONSENT FORMS Early Grade Reading Program Impact Evaluation Midline Circuit Supervisor Questionnaire General Background Information Team name: Enumerator ID: Today’s date (DD-MM-YY): - - Start time (HH:MM): : School name: School ID: Region: District: Circuit: Circuit code: Locality: Was the consent form administered and signed? O Yes O No (DO NOT PROCEED) Circuit Supervisor Identifying Information First name: Last name: Sex: O Male O Female Telephone: 114 A. Circuit Supervisor Background Information 1. How long have you been the Circuit Supervisor for this school? Years: Months: 2. How long have you been a Circuit Supervisor overall? Years: Months: 3. How long have you been in the education field overall? Years: Months: 4. Have you completed teacher training college? O Yes O No O Decline to answer 5. What is the highest level of education that you have completed? (select one) [Do not read options]  O Middle School O Senior High School O Ordinary Level O Advance Level O Certificate A O Diploma in Basic Education O Higher National Diploma (HND) O Technical/Vocational (NVTI) O Bachelor’s of Education (B.Ed) O Other Bachelor’s Degrees O Post Graduate Diploma in Education O Masters of Education (M.Ed) O Other Master’s Degree O Ph.D. O Decline to answer O Other (specify): ________________________. 6. How many times in the current school year (2017/18) have you visited this school?   7. Is this number of visits higher, lower, or just the right amount of visits required?  O Higher than required  skip to B1 O Just the right amount  skip to B1 O Lower than required O I’m not sure what is required  skip to B1 O Decline to answer  skip to B1 8. [If lower than required in question A7] Why are you unable to complete the number of required visits? (select all that apply) [Do not prompt, record any relevant answers]  I have too much work to do in the office There are too many schools in my circuit It is too hard to get to the school physically I don’t have the resources (money, fuel, etc.) to visit the school 115 My visits don’t do any good here My visits are not well received here The school year is not yet finished Other (specify): ____________________________________________________________. B. Language Background [READ] I will now ask you some questions about the use of different languages in this school. The purpose of this module is to help us understand how Ghana’s linguistic diversity affects the ability of teachers to teach reading as well as the ability of pupils to become fluent readers. 1. For each of the following languages, please state whether you speak the language fluently, partially, or not at all [Enumerator: Please name each language and ask for an answer for that language before moving on. You must read out all of the languages] Language Fluently Partially Not at all Akuapem Twi O O O Asante Twi O O O Dagaare O O O Dagbani O O O Dangme O O O English O O O Ewe O O O Fante O O O Ga O O O Gonja O O O Kasem O O O Nzema O O O 2. What is the GES-approved Ghanaian language of instruction for lower primary at this school? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Kasem O Nzema O Decline to answer O Don’t know O Other (specify): ________________________.  3. Do you think this school is facing any challenges in using the GES-approved Ghanaian language as the medium of instruction? O Yes O No  skip to B5 O Decline to answer  skip to B5 116 4. [If yes to question B3] Please describe the challenges this school faces in using the GES￾approved Ghanaian language as the medium of instruction:  5. [If language barrier is specified in question B4] What strategies do teachers at this school employ to help overcome this language barrier when teaching reading in the GES-approved Ghanaian language of instruction? (select all that apply)  Teach in another Ghanaian language Teach in English Have me or other teachers who speak the language of instruction help them Have pupils translate for them Have pupils translate for one another Use visual cues N/A Other (specify): ____________________________________________________________. C. In-Service Training and Professional Development [READ] This next module will focus on any in-service training and professional development (including coaching) that is provided to teachers at this school. Your answers to these questions will help us identify current gaps in in￾service training and teacher coaching support in Ghana. 1. Have you attended any formal in-service training or professional development sessions on early grade reading in the past school year (2017/18)?  O Yes O No  skip to C6 O Don’t know O Decline to answer  2. About how many formal in-service training days did you receive in the current school year (2017/18)? 3. How many of these training days, if any, were for Ghanaian language reading instruction? [Enter 0 if none]. 4. Please specify the training provider or providers: GES Master Trainer NGO Master Trainer Learning Master Trainer 117 District Teacher Support Team Other (specify): ____________________________________________________________. 5. Do you feel this training was enough for you to be able to use these methods correctly in your work? O Yes O No O Don’t know O Decline to answer  6. Do you feel you need more training? O Yes O No  skip to C8 O Don’t know  skip to C8 O Decline to answer  skip to C8 7. [If yes to C6] In which topics would you like to receive more training? 8. In the past year, have you provided teachers in this school with any mentoring, coaching, or methodological support in teaching reading? O Yes O No  skip to D1 O Don’t know  skip to D1 O Decline to answer  skip to D1 9. [If yes to C8] What types of methodological support did you provide? Weekly training session at the school Monthly training session at the school Periodic training sessions at the school (less than monthly) Help with reading lesson planning Coaching based on observed lesson Other (specify): ____________________________________________________________. 10. [If coaching based on observed lesson is selected in question C9] Did teachers adjust their teaching in response to your feedback? O Yes O No  skip to C12 O Don’t know  skip to C13 O Decline to answer  skip to C13  11. [If yes to C10] In what ways did teachers adjust their teaching in response to your feedback? [Do not prompt, record any relevant answers]  skip to C14  Better lesson planning 118 Provide equal opportunities to pupils Group assessment Individual assessment Made class more interactive Other (specify): ____________________________________________________________. 12. [If no to C10] Why did the teachers not adjust their teaching in response to your feedback? [Do not prompt, record any relevant answers] Lack of skill Lack of teaching materials Lack of understanding of the coaching Difference in beliefs on effective teaching Other (specify): ____________________________________________________________. 13. Do you feel this coaching was positively received by the teachers?  O Yes  skip to D1 O No O Don’t know  skip to D1 O Decline to answer  skip to D1 14. [If no to C13] Why was this coaching not received positively?     D. Circuit Supervisor Beliefs on Reading Instructional Methods [READ] I will now read a number of statements about different approaches to teaching reading and ask you to state whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree with that statement. For this module, we want to understand your personal beliefs on teaching reading effectively, even if those beliefs differ from what you actually do or guide others to do. All of your responses are confidential so please answer truthfully. 1. Children should learn to read new words by visually memorizing them as whole units: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 2. Children in Ghana would learn English better if they were taught how to initially read using their mother tongue: 119 O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 3. In the early grades, it is important to use a lot of class time to teach children to “sound out” words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 4. Children should learn to read syllables before they learn to read whole words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 5. The way I was taught to read as a child is the same approach teachers should use to teach their pupils: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 6. Children in Ghana would benefit more if they were taught how to initially read using English: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree  7. Observing and coaching teachers on their reading instruction practices is an important part of my job as Circuit Supervisor: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 120 8. What was the primary language in which you first learned how to read and write? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Gurene O Kasem O Kusaal O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. E. Pupil Performance and Evaluation  [READ] I will now ask a few questions about pupil performance and assessment in your school. The purpose of these questions is to help us understand why some pupils struggle to become fluent readers in Ghana. We also want to know if and how teachers are using assessment to improve teaching and learning. 1. Do you believe KG and lower primary pupils (KG1-P3) in this school are on track to become fluent readers? O Yes  skip to E3 O No O Don’t know  skip to E4 O Decline to answer  skip to E4 2. [If no to E1] Why do you think lower primary pupils are not on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers] They don’t come to class often enough They don’t come to class on time They don’t pay attention during class Their parents do not support their reading at home They are too hungry to concentrate They don’t care about school They are not able to understand the language of instruction The class is too large for teachers to provide good instruction The lessons are too short Teachers don’t always teach the lessons because they are doing other things Teachers don’t show up for work Teachers don’t speak the language of instruction The teachers don’t do a good job teaching There are not enough teachers at this school Teachers vacate their postings too often We don’t have the teaching resources we need to teach them well We don’t have the training we need to teach them well The pupils don’t have enough learning books The pupils don’t have access to books outside of school Other (specify)_____________________________________________________________.  121 3. How do you know whether they are on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers]  I have personally assessed them I have seen their assessment results Teachers tell me they are doing OK The Head Teacher tells me they are doing OK They seem to be doing OK when I observe them in class I don’t know Other (specify)_____________________________________________________________.  4. Do you think teachers at this school adjust their teaching based on the results of pupil evaluation or assessment? O Yes O No  skip to E6 O Don’t know  skip to E6 O Decline to answer  skip to E6 5. [If yes to E4] In what ways do teachers adjust their teaching based on the results of pupil evaluation or assessment? [Read each option and record relevant answers] Repeat previous lessons Make new lessons to teach difficult content in a different way Pay more attention to struggling learners in class Pay more attention to high performing learners in class Provide struggling learners with tutoring outside of class Arrange for others to provide struggling learners with tutoring outside of class Pair struggling learners with stronger ones during class If learners are doing well, skip lessons that aren’t useful If learners are doing well, go through lessons more quickly Other (specify)_____________________________________________________________. 6. Do teachers at this school track their pupils’ progress over time (e.g., termly)? O Yes O No O Some, but not all O Don’t know O Decline to answer F. EGR Program Implementation [READ] I will now ask a few questions on how the Early Grade Reading program has been going at this school. As with all the questions we are asking, your answers are strictly confidential. Your individual responses will only be viewed by the independent evaluation team, so please share your thoughts freely and openly. Your responses will be compiled with hundreds of other Circuit Supervisors’ responses and will help USAID/Learning improve the Early Grade Reading program. 122 1. Do you believe that teachers and the Head Teacher at this school are motivated and happy about the new Ghanaian language Early Grade Reading program?  O Yes  skip to F3 O No O Don’t know  skip to F3 O Decline to answer  skip to F3 2. [If no in question F1 above] Why are they not motivated and happy about the program?  3. Do you believe that teachers in this school are more likely to use phonics-based methods for teaching reading as a result of the Early Grade Reading Program?  O Yes  skip to F5 O No O Don’t know  skip to F5 O Decline to answer  skip to F5 4. [If no in question F3 above] Why are they not more likely to use phonics-based methods?  5. Do you believe that teachers in this school are more likely to use Ghanaian language for teaching reading as a result of the Early Grade Reading Program?  O Yes  skip to F7 O No O Don’t know  skip to F7 O Decline to answer  skip to F7 6. [If no in question F5 above] Why are they not more likely to use Ghanaian language for teaching reading?  7. Do you believe that teachers in this school are regularly using the scripted lesson plans?  O Yes  skip to F9 O No O Don’t know  skip to F10 O Decline to answer  skip to F10 123 8. [If no in question F7 above] Why are they not regularly using the scripted lesson plans?  9. Are teachers at this school able to maintain the pace of the scripted lesson plans?  O Yes  skip to F11 O No O Don’t know  skip to F11 O Decline to answer  skip to F11 10. [If no in question F9 above] Why do you think they are unable to maintain the pace of the scripted lesson plans?  11. Do you visit this school and observe and assess teachers’ implementation of the Early Grade Reading program using an electronic tablet?  O Yes O No  skip to F15 O Don’t know  skip to F15 O Decline to answer  skip to F15 12. [If yes in question F11 above] Can you describe how (if at all) the electronic monitoring data you collect is shared with the school?  13. [If data is shared in F12 above] How do practices at this school change (if at all) in response to this monitoring data?  14. Do you face any challenges in using the tablet to record and transmit data?  124 15. Please describe additional challenges (if any) this school faces in implementing the Early Grade Reading Program:  16. Please share any additional thoughts on the Early Grade Reading program:  Thank you! That completes the Circuit Supervisor questionnaire. End Time (HH:MM): : 125 Ghana Early Grade Reading Program Impact Evaluation Midline Head Teacher Questionnaire (Treatment) General Background Information Team name: Enumerator ID: Today’s date (DD-MM-YY): - - Start time (HH:MM): : School name: School ID: Region: District: Circuit: Circuit code: Locality: Locality type: O Urban (more than 5,000 people living in community/locality) O Rural (less than 5,000 people living in community/locality) School address: Was the consent form administered and signed? O Yes O No (DO NOT PROCEED) Head Teacher Identifying Information First name: Last name: Sex: O Male 126 O Female Telephone:  A. Head Teacher Background Information 1. Did someone from ILC Africa interview you in May or June of last year?  O Yes  skip to A7 O No O Don’t know O Decline to answer  2. How long have you been the Head Teacher at this school? Years: Months: 3. How long have you been a Head Teacher overall? Years: Months: 4. How long have you been in the teaching profession overall? Years: Months: 5. Have you completed teacher training college? O Yes O No O Decline to answer 6. What is the highest level of education that you have completed? (select one) [Do not read options] O Middle School O Senior High School O Ordinary Level O Advance Level O Certificate A O Diploma in Basic Education O Higher National Diploma (HND) O Technical/Vocational (NVTI) O Bachelor’s of Education (B.Ed) O Other Bachelor’s Degrees O Post Graduate Diploma in Education O Masters of Education (M.Ed) O Other Master’s Degree O Ph.D. O Decline to answer O Other (specify): ________________________. 7. Does your school have more than one Head Teacher?  O Yes O No O Decline to answer  8. Which class(es) are you responsible for as Head Teacher? (select all that apply) KG1 P3 127 KG2 P1 P2 P4 P5 P6  9. In addition to your duties as Head Teacher, do you teach classes at this school?  O Yes O No  skip to A12 O Decline to answer  skip to A12   10. [If yes to A9] In a typical week, approximately how many hours do you work at this school?   11. [If yes to A9] Of these, how many hours are spent teaching?   12. For how many days last week (Monday thru Friday) was this school open / in session? Please include partial days (e.g., 0.5 days). 13. Of the $[A12] days that the school was open last week, for how many days were you absent? Please include partial absences (e.g., 0.5 days).  B. School Background [READ] I will begin by collecting some basic information on this school, including the number of teachers, trained teachers, learners, grades, streams, and shifts for KG1-P3 as well as the duration of lessons and number of repetitions and dropouts. This data will help us understand how reading performance varies across schools with different demographics. Kindly collect any necessary administrative records now so that they can be referenced where needed. 1. In total, how many KG1-P3 teachers are there at this school?  2. For each KG and lower primary grade present at this school, please specify a) how many shifts this school has, b) how many streams the school has, c) how many teachers teach that grade level, and d) total enrollments in that grade for each gender (across all streams/classes and shifts): Grade Present? Number of shifts Number of streams Number of teachers Girls enrolled Boys enrolled KG1 128 KG2 P1 P2 P3 3. Are any of the KG1-P3 classes in this school combined?  O Yes O No  skip to B5 O Don’t know  skip to B5 O Decline to answer  skip to B5  4. [If yes to B3] Which grades are combined?  5. How many hours and minutes do P1 learners spend at this school each day (length of school day)? Hours: Minutes: 6. In a typical 5-day school week, on how many days do P1 learners receive a reading lesson? 7. What is the typical duration of a daily reading lesson for P1 learners? Hours: Minutes: 8. How many P1 teachers teach reading at this school?  9. Of the P1 teachers that teach reading, how many are female? [Enter 0 if none are female]   10. Of the P1 teachers that teach reading at this school, how many were working at this school at the beginning of the 2017/18 academic year?  129 11. Was the P1 teacher selected for this study absent on any days during the most recent completed school week (Monday thru Friday)? O Yes O No  skip to B13 O Don’t know  skip to B13 O Decline to answer  skip to B13 12. [If yes to B11] For how many days last week was he or she absent? Please include partial absences (e.g., 0.5 days). 13. For each KG and lower primary grade level, please indicate the number of pupils that have dropped out in the current academic year (2017/18). Please do not include pupils who have transferred, relocated, or died in these figures. [If no one has dropped out from the school,  skip to B16]  Grade Girl dropouts Boy dropouts KG1 KG2 P1 P2 P3 14. Are these pupils who dropped out counted in the enrollment figures you gave me earlier (question B2)?  O Yes, they are counted in the enrollment figures I provided O No, they have already been subtracted from the enrollment figures I provided O Don’t know O Decline to answer 15. What are the main reasons pupils at this school drop out? [Enumerator: if the Head Teacher mentions transfers, relocations, or deaths, please remind him/her that dropouts should NOT include transfers, relocations, or deaths and go back to question B13 and make any necessary corrections] 16. For each KG and lower primary grade level, please indicate the number of pupils that are repeaters (i.e., were in the same grade level last year): [If no one has repeated,  skip to C1]  Grade Girl repeaters Boy repeaters 130 KG1 KG2 P1 P2 P3 17. What are the main reasons pupils at this school repeat grades? C. Language Background [READ] I will now ask you some questions about the use of different languages in this school. The purpose of this module is to help us understand how Ghana’s linguistic diversity affects the ability of teachers to teach reading as well as the ability of pupils to become fluent readers. 1. What language(s) do children in your school speak on the playground? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 2. What is the GES-approved Ghanaian language of instruction for lower primary at this school? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Kasem O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. 131 3. How many lower primary teachers (KG1-P3) at this school use the GES-approved Ghanaian language of instruction when teaching?   4. Of these, how many teach comfortably in the GES-approved Ghanaian language of instruction?   5. What other languages besides the GES-approved Ghanaian language of instruction do KG and lower primary teachers (P1-P3) use to teach at this school? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________.  6. Is your school facing any challenges in using the GES-approved Ghanaian language as the medium of instruction? O Yes O No  skip to C9 O Don’t know  skip to C9 O Decline to answer  skip to C9 7. [If yes in question C6] Please describe the challenges your school faces in using the GES-approved Ghanaian language as the medium of instruction: 8. [If language barrier is specified in question C7] What strategies do teachers at this school employ to help overcome this language barrier when teaching reading in the GES-approved Ghanaian language of instruction? (select all that apply) [Do not prompt, record any relevant answers]  Teach in another Ghanaian language Teach in English Have me or other teachers who speak the language of instruction help them 132 Have pupils translate for them Have pupils translate for one another Use visual cues N/A Other (specify): ____________________________________________________________. 9. In this school, at what level is English used as the primary medium of instruction for reading and writing? (select all that apply) [Do not prompt, record any relevant answers] KG1 KG2 P1 P2 P3 P4 P5 P6 10. In this school, at what level is English used as the primary medium of instruction for subjects other than reading and writing? (select all that apply) [Do not prompt, record any relevant answers]  KG1 KG2 P1 P2 P3 P4 P5 P6 11. Which language or languages would you prefer to be used as the primary language of instruction in your school? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. D. School Resources  [READ] I will now ask several questions about the state of this school, including availability of key resources such as water, toilets, and electricity. For each resource, please state whether that resource is regularly functioning, sometimes not functioning, or rarely or never functioning. This will help us understand if the school environment is conducive for learning. 133 1. School resources roster: [Enumerator: Please name each resource and ask for an answer for that resource before moving on. You must read all out all of the resources listed here] Resource Don’t have Have, and regularly functioning Have, but sometimes not functioning Have, but rarely or never functioning Electricity O O O O Pipe-borne water O O O O Other water (well, borehole, reservoir) O O O O Vehicle access road (paved) O O O O Vehicle access road (unpaved) O O O O Toilets for pupils O O O O Separate toilets for girls O O O O Separate toilets for teachers O O O O School playground O O O O Free meals scheme - breakfast O O O O Free meals scheme - lunch O O O O School Management Committee O O O O Parent Teacher Association O O O O 2. Does this school have an approved School Performance Improvement Plan for this 2017/18 academic year?  O Yes O No O Don’t know O Decline to answer 3. Does this school have a trained Curriculum Leader?  O Yes O No O Don’t know O Decline to answer 4. How often does a Circuit Supervisor visit this school? (select one)  O At least monthly O Twice or more a term 134 O Once a term O Once a year O Hardly ever O To my knowledge, a Circuit Supervisor has never visited this school O Don’t know O Decline to answer  E. Reading Resources [READ] This next module will focus on the availability of teacher guides, lesson plans, learner textbooks, activity books, and supplementary materials and the extent of access and use. The purpose of this module is to identify possible gaps in resources as well as gauge the degree of use of resources that are available. 1. What teaching and learning materials does your school have available in the GES￾approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers]  Teacher's guide Scripted/step-by-step lesson plans Unscripted/generalized lesson plans Lesson notes Pupil readers or textbooks Pupil workbooks Pupil library books Diagraph and trigraph charts Big books Posters Flash Cards Alphabet Strip Material for teacher-made teaching aids Box of materials sent by Learning Other (specify)_______________________. 2. Are the teaching and learning materials at your school sufficient to enable effective teaching in the GES-approved Ghanaian language of instruction? O Yes  skip to E4 O No O Don’t know  skip to E4 O Decline to answer  skip to E4  3. [If no in question E2 above] Why are the teaching and learning materials not sufficient? (select all that apply) [Do not prompt, record any relevant answers]  There are not enough materials for every pupil There are not enough materials for every teacher or class The materials are outdated The materials are damaged We are not able to use the materials because of teachers don’t know the language We are not able to use the materials because pupils don’t know the language Other (specify): ____________________________________________________________.  4. Which of the following specific reading program materials have been provided to your school in the GES-approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers]  Integrated Approach 135 Let's Read and Write National Literacy Acceleration Program (NALAP) pupils' book NALAP Big Book NALAP teachers' book Enlightening the hearts: early grade reading manual for literacy at lower primary Literacy Learning through Play Learning Early Grade Reading Program Teacher’s Guides Learning Early Grade Reading Program Pupil Workbooks Learning Early Grade Reading Program Readers Other (specify): ____________________________________________________________.  5. Does this school have a library? O Yes O No  skip to F1 O Decline to answer  skip to F1 6. [If yes in question E5 above] Does this library include books that are in the GES￾approved Ghanaian language of instruction?  O Yes O No O Don’t know O Decline to answer  7. Do children take library books home with them? O Yes  skip to F1 O No O Don’t know  skip to F1 O Decline to answer  skip to F1  8. [If no in question E7 above] Why don’t children take library books home with them? (select all that apply) [Do not prompt, record any relevant answers]  They are not allowed to because they will lose or damage them They are not allowed to based on the rules of the sponsor They don’t want to take books home No one at home can read with them They don’t understand the language of the books There are not enough books in the library Other (specify): ____________________________________________________________. F. In-Service Training and Professional Development [READ] This next module will focus on any in-service training and professional development (including coaching) that is provided to teachers at this school. Your answers to these questions will help us identify current gaps in in￾service training and teacher coaching support in Ghana. 136 1. During the 2017/18 school year, have you attended any formal in-service training or professional development sessions on early grade reading?  O Yes O No  skip to F4 O Don’t know  skip to F4 O Decline to answer  skip to F4  2. About how many training days did you receive in the current school year (2017/18)? 3. How many of these training days, if any, were focused on Ghanaian language reading instruction? 4. Have teachers in this school attended any formal in-service training or professional development sessions on early grade reading in the current school year (2017/18)? O Yes O No  skip to F9 O Don’t know  skip to F9 O Decline to answer  skip to F9 5. About how many training days did they receive each (average) in the current school year (2017/18)? 6. How many of these training days, if any, were focused on Ghanaian language reading instruction? 7. Do you feel this training was enough for them to be able to use these methods correctly in the classroom? O Yes O No O Don’t know O Decline to answer 8. Please specify the training provider or providers: (select all that apply) GES Master Trainer NGO Master Trainer Learning Master Trainer District Teacher Support Team Other (specify): ____________________________________________________________. 9. Have teachers in this school attended any school-based in-service (SBI) sessions on Ghanaian language reading instruction this year (2017/18)? 137 O Yes O No  skip to F16 O Don’t know  skip to F16 O Decline to answer  skip to F16 10. [If yes to 9] Approximately how many SBI meetings have been held at this school in the 2017/18 school year? 11. [If yes to 9] Do you keep records of dates and/or teacher attendance for these SBI sessions?  O Yes, I keep records of SBI session dates and attendance O Yes, I keep records of SBI session dates only O No, I do not keep records on SBI sessions  skip to F14 O Don’t know  skip to F14 O Decline to answer  skip to F14 12. Are you able to show me these SBI records? Remember, this is only for SBI meetings focused on teaching early grade reading in the Ghanaian language of instruction.  O Yes [ask to retrieve] O No, I am not willing  skip to F14 O No, because I do not have them with me right now  skip to F14 O No, my records are not organized well enough  skip to F14 O I have not separated Ghanaian language SBI data from other SBI we hold at this school  skip to F14 13. [Please record dates for all scheduled SBI meetings, the count of teachers in attendance, and note whether the sampled P1 teacher was in attendance]: SBI meeting date (DD-MM-YY) Total teacher attendance count Was the sampled P1 teacher in attendance? Yes No No record - - - - - - - - - - - - - - - - - - - - 138 - - - - 14. Do you believe that teachers are applying what they discuss in SBI meetings to improve upon their classroom teaching practice? [Read each option and record relevant answer] (select one)  O Yes, very well  skip to F16 O Yes, but there is room for improvement  skip to F16 O Yes, but not very well O No, they are not applying what they discuss O Don’t know  skip to F16 O Decline to answer  skip to F16  15. Why do you feel teachers are not applying what they learn in SBI meetings to improve upon their classroom teaching practice?  16. Do you feel teachers in this school need more training in Ghanaian language reading instruction? O Yes O No  skip to F18 O Don’t know  skip to F18 O Decline to answer  skip to F18 17. [If yes in question F16] In which topics would you like them to receive more training? (select all that apply) Reading Writing Assessment Material Usage Other (specify): ____________________________________________________________. 18. Have teachers in your school ever received training in pupil assessment?  O Yes O No O Don’t know O Decline to answer  19. During the current school year (2017/18), have teachers in this school received any mentoring or coaching in teaching reading? O Yes 139 O No  skip to F27 O Don’t know  skip to G1 O Decline to answer  skip to G1 20. [If yes in question F19] Who in the current school year (2017/18) provided them with mentoring or coaching in teaching reading? (select all that apply) [Read each option and record relevant answers] Head Teacher (myself) Curriculum Lead Circuit Supervisor District Teacher Support Team Other Teacher(s) at this school Someone from GES / MOE An NGO Other (specify): ____________________________________________________________.  21. [If Head Teacher is selected in question F20 above] What types of mentoring or coaching did you provide? (select all that apply) [Do not prompt, record any relevant answers] Weekly training session at the school Monthly training session at the school Periodic training sessions at the school (less than monthly) Help with reading lesson planning Coaching based on observed lesson Other (specify): ____________________________________________________________. 22. [If coaching based on observed lesson is selected in question F21 above] Did teachers change how they teach reading in response to the feedback you provided? O Yes O No  skip to F24 O Don’t know  skip to F25 O Decline to answer  skip to F25 23. [If yes in question F22 above] In what ways did teachers change how they teach reading in response to your feedback? [Do not prompt, record any relevant answers] Better lesson planning Provide equal opportunities to pupils Group assessment Individual assessment Made class more interactive Other (specify): ____________________________________________________________. 24. [If no in question F22 above] Why did the teachers not change their teaching in response to your feedback? [Do not prompt, record any relevant answers] Lack of skill Lack of teaching materials 140 Lack of understanding of the coaching Difference in beliefs on effective teaching Other (specify): ____________________________________________________________. 25. Do you feel this coaching was positively received by the teachers?  O Yes  skip to G1 O No O Don’t know  skip to G1 O Decline to answer  skip to G1 26. [If no in question F25 above] Why was this coaching not received positively?  skip to G1  27. [If Head Teacher was NOT selected in question F20] What are the main reasons you do not provide mentoring or coaching to your teachers in teaching reading? I am too busy with administrative responsibilities I am too busy teaching Providing mentoring or coaching is not my responsibility I don’t have the training or knowledge to provide support I do not want to The teachers do not listen to me when I provide support I do not speak or understand the language that the teacher is using Other (specify): ____________________________________________________________. G. Head Teacher Beliefs on Reading Instructional Methods [READ] I will now read a number of statements about different approaches to teaching reading and ask you to state whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree with that statement. For this module, we want to understand your personal beliefs on teaching reading effectively, even if those beliefs differ from what you actually do, are required to do, or guide others to do. All of your responses are confidential so please answer truthfully. 9. Children should learn to read new words by visually memorizing them as whole units: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 141 10. Children in Ghana would learn English better if they were taught how to initially read using their mother tongue: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 11. In the early grades, it is important to use a lot of class time to teach children to “sound out” words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 12. Children should learn to read syllables before they learn to read whole words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 13. The way I was taught to read as a child is the same approach I should use to teach my pupils: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 14. Children in Ghana would benefit more if they were taught how to initially read using English: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 15. Observing reading lessons and providing coaching and feedback to teachers on their instructional practice is part of my regular duties as Head Teacher:  O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 142 16. What was the primary language in which you first learned how to read and write? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Gurene O Kasem O Kusaal O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. H. Pupil Performance and Evaluation  [READ] I will now ask a few questions about pupil reading performance in your school. The purpose of these questions is to help us understand why some pupils struggle to become fluent readers in Ghana. 1. Do you believe KG and lower primary pupils (KG1-P3) in this school are on track to become fluent readers? O Yes  skip to H3 O No O Don’t know  skip to I1 O Decline to answer  skip to I1 2. [If no in question H1 above] Why do you think lower primary pupils are not on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers] They don’t come to class often enough They don’t come to class on time They don’t pay attention during class Their parents do not support their reading at home They are too hungry to concentrate They don’t care about school They are not able to understand the language of instruction The class is too large for teachers to provide good instruction The lessons are too short Teachers don’t always teach the lessons because they are doing other things Teachers don’t show up for work Teachers don’t speak the language of instruction The teachers don’t do a good job teaching There are not enough teachers at this school Teachers vacate their postings too often We don’t have the teaching resources we need to teach them well We don’t have the training we need to teach them well The pupils don’t have enough learning books The pupils don’t have access to books outside of school Other (specify)_____________________________________________________________.  143 3. How do you know whether they are on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers]  We assess them regularly Teachers tell me they are doing OK They seem to be doing OK when I observe them in class I don’t know Other (specify)_____________________________________________________________. I. EGR Program Implementation  [READ] I will now ask a few questions on how the Early Grade Reading program has been going at this school. As with all the questions we are asking, your answers are strictly confidential. Your individual responses will only be viewed by the independent evaluation team, so please share your thoughts freely and openly. Your responses will be compiled with hundreds of other Head Teachers’ responses and will help USAID/Learning improve the Early Grade Reading program. 1. Do you believe that teachers at this school are motivated and happy about the new Ghanaian language Early Grade Reading program?  O Yes  skip to I3 O No O Don’t know  skip to I3 O Decline to answer  skip to I3 2. [If no in question I1 above] Why are they not motivated and happy about the program?  3. Do you believe that teachers in this school are more likely to use phonics-based methods for teaching reading as a result of the Early Grade Reading Program?  O Yes  skip to I5 O No O Don’t know  skip to I5 O Decline to answer  skip to I5 4. [If no in question I3 above] Why are they not more likely to use phonics-based methods?  144 5. Do you believe that teachers in this school are more likely to use Ghanaian language for teaching reading as a result of the Early Grade Reading Program?  O Yes  skip to I7 O No O Don’t know  skip to I7 O Decline to answer  skip to I7 6. [If no in question I5 above] Why are they not more likely to use Ghanaian language for teaching reading?  7. Are Circuit Supervisors at this school conducting visits to monitor implementation of the Early Grade Reading program?  O Yes O No  skip to I11 O Don’t know  skip to I11 O Decline to answer  skip to I11 8. [If yes in question I7 above] Do they observe and assess teacher performance using an electronic tablet?  O Yes O No  skip to I11 O Don’t know  skip to I11 O Decline to answer  skip to I11 9. [If yes in question I8 above] Can you describe how (if at all) the monitoring data they collect on the tablet is shared with the school?  10. [If data is shared in I9 above] How do practices at this school change (if at all) in response to this monitoring data?  11. Please describe challenges (if any) this school faces in implementing the Early Grade Reading Program:  145 12. Besides the Ghana Early Grade Reading Program run by Learning, have any other early grade reading programs operated at this school in the 2017/18 academic year?  O Yes O No  skip to J1 O Don’t know  skip to J1 O Decline to answer  skip to J1 13. [If yes in question I12 above] Please share with us the names of the program(s), the organization leading the program(s), and provide a brief description:  J. Teacher Rationalization and Deployment  [READ] In this final module, I will ask some questions on teacher recruitment, deployment, and retention. Your answers will help us understand how the movement of teachers influences early grade reading performance. 1. How do you determine the number of teachers you need at this school?  2. When a teaching position is vacant in your school, how do you go about filling it? (select all that apply) [Do not prompt, record any relevant answers]  I report the vacancy to the district Director of Education and request for a teacher I recruit the teacher and submit the name to the district director for approval I request the School Management Committee to recruit a replacement teacher I request for teachers from the District Assembly I request for teachers from the National Service Secretariat I request for teachers from National Youth Employment I don’t know Other (specify)_____________________________________________________________. 3. In your opinion, do some schools in this district have a surplus of teachers while other schools in the district have a shortage?  O Yes 146 O No O Don’t know  skip to J5 O Decline to answer  skip to J5 4. [If yes in J3] Do you know the reasons why some schools in this district have more teachers than they need while others do not have enough?  5. Of the new teachers posted to this school during your tenure, how many are professionally trained or have been duly registered as professional teachers by GES: none, some, most, or all? [select one]  O None O Some (less than half) O Most (more than half) O All O I have not observed new teachers posted here during my tenure O Don’t know O Decline to answer 6. Of the new teachers posted to this school during your tenure, how many have the appropriate competencies and skills to teach reading in lower primary using the GES-approved Ghanaian language of instruction of the school: none, some, most, or all? (select one)  O None O Some (less than half) O Most (more than half) O All O I have not observed new teachers posted here during my tenure O Don’t know O Decline to answer 7. During your tenure, how many teachers have transferred or vacated their positions at this school before serving 5 full years?  O None  END O Some (less than half) O Most (more than half) O All O Don’t know  END O Decline to answer  END 8. In your opinion, what are the main reasons teachers have transferred or vacated their positions at this school before serving 5 full years? 147  Thank you! That completes the Head Teacher questionnaire. End Time (HH:MM): : 148 Ghana Early Grade Reading Program Impact Evaluation Midline Head Teacher Questionnaire (Comparison) General Background Information Team name: Enumerator ID: Today’s date (DD-MM-YY): - - Start time (HH:MM): : School name: School ID: Region: District: Circuit: Circuit code: Locality: Locality type: O Urban (more than 5,000 people living in community/locality) O Rural (less than 5,000 people living in community/locality) School address: Was the consent form administered and signed? O Yes O No (DO NOT PROCEED) Head Teacher Identifying Information First name: Last name: Sex: O Male 149 O Female Telephone: A. Head Teacher Background Information 1. Did someone from ILC Africa interview you in May or June of last year?  O Yes  skip to A7 O No O Don’t know O Decline to answer  2. How long have you been the Head Teacher at this school? Years: Months: 3. How long have you been a Head Teacher overall? Years: Months: 4. How long have you been in the teaching profession overall? Years: Months: 5. Have you completed teacher training college? O Yes O No O Decline to answer 6. What is the highest level of education that you have completed? (select one) [Do not read options] O Middle School O Senior High School O Ordinary Level O Advance Level O Certificate A O Diploma in Basic Education O Higher National Diploma (HND) O Technical/Vocational (NVTI) O Bachelor’s of Education (B.Ed) O Other Bachelor’s Degrees O Post Graduate Diploma in Education O Masters of Education (M.Ed) O Other Master’s Degree O Ph.D. O Decline to answer O Other (specify): ________________________. 7. Does your school have more than one Head Teacher?  O Yes O No O Decline to answer  8. Which class(es) are you responsible for as Head Teacher? (select all that apply) 150 KG1 KG2 P1 P2 P3 P4 P5 P6  9. In addition to your duties as Head Teacher, do you teach classes at this school?  O Yes O No  skip to A12 O Decline to answer  skip to A12   10. [If yes to A9] In a typical week, approximately how many hours do you work at this school?   11. [If yes to A9] Of these, how many hours are spent teaching?   12. For how many days last week (Monday thru Friday) was this school open / in session? Please include partial days (e.g., 0.5 days). 13. Of the $[A12] days that the school was open last week, for how many days were you absent? Please include partial absences (e.g., 0.5 days).  B. School Background [READ] I will begin by collecting some basic information on this school, including the number of teachers, trained teachers, learners, grades, streams, and shifts for KG1-P3 as well as the duration of lessons and number of repetitions and dropouts. This data will help us understand how reading performance varies across schools with different demographics. Kindly collect any necessary administrative records now so that they can be referenced where needed. 1. In total, how many KG1-P3 teachers are there at this school?  2. For each KG and lower primary grade present at this school, please specify a) how many shifts this school has, b) how many streams the school has, c) how many teachers teach that grade level, and d) total enrollments in that grade for each gender (across all streams/classes and shifts): 151 Grade Present? Number of shifts Number of streams Number of teachers Girls enrolled Boys enrolled KG1 KG2 P1 P2 P3 3. Are any of the KG1-P3 classes in this school combined?  O Yes O No  skip to B5 O Don’t know  skip to B5 O Decline to answer  skip to B5  4. [If yes to B3] Which grades are combined?  5. How many hours and minutes do P1 learners spend at this school each day (length of school day)? Hours: Minutes: 6. In a typical 5-day school week, on how many days do P1 learners receive a reading lesson? 7. What is the typical duration of a daily reading lesson for P1 learners? Hours: Minutes: 8. How many P1 teachers teach reading at this school?  9. Of the P1 teachers that teach reading, how many are female? [Enter 0 if none are female]   10. Of the P1 teachers that teach reading at this school, how many were working at this school at the beginning of the 2017/18 academic year?  152 11. Was the P1 teacher selected for this study absent on any days during the most recent completed school week (Monday thru Friday)? O Yes O No  skip to B13 O Don’t know  skip to B13 O Decline to answer  skip to B13 12. [If yes to B11] For how many days last week was he or she absent? Please include partial absences (e.g., 0.5 days). 13. For each KG and lower primary grade level, please indicate the number of pupils that have dropped out in the current academic year (2017/18). Please do not include pupils who have transferred, relocated, or died in these figures. [If no one has dropped out from the school,  skip to B16]  Grade Girl dropouts Boy dropouts KG1 KG2 P1 P2 P3 14. Are these pupils who dropped out counted in the enrollment figures you gave me earlier (question B2)?  O Yes, they are counted in the enrollment figures I provided O No, they have already been subtracted from the enrollment figures I provided O Don’t know O Decline to answer 15. What are the main reasons pupils at this school drop out? [Enumerator: if the Head Teacher mentions transfers, relocations, or deaths, please remind him/her that dropouts should NOT include transfers, relocations, or deaths and go back to question B13 and make any necessary corrections] 16. For each KG and lower primary grade level, please indicate the number of pupils that are repeaters (i.e., were in the same grade level last year): [If no one has repeated,  skip to C1]  Grade Girl repeaters Boy repeaters 153 KG1 KG2 P1 P2 P3 17. What are the main reasons pupils at this school repeat grades? C. Language Background [READ] I will now ask you some questions about the use of different languages in this school. The purpose of this module is to help us understand how Ghana’s linguistic diversity affects the ability of teachers to teach reading as well as the ability of pupils to become fluent readers. 1. What language(s) do children in your school speak on the playground? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 2. What is the GES-approved Ghanaian language of instruction for lower primary at this school? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Kasem O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. 3. How many lower primary teachers (KG1-P3) at this school use the GES-approved Ghanaian language of instruction when teaching?  154  4. Of these, how many teach comfortably in the GES-approved Ghanaian language of instruction?   5. What other languages besides the GES-approved Ghanaian language of instruction do KG and lower primary teachers (P1-P3) use to teach at this school? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________.  6. Is your school facing any challenges in using the GES-approved Ghanaian language as the medium of instruction? O Yes O No  skip to C9 O Don’t know  skip to C9 O Decline to answer  skip to C9 7. [If yes in question C6] Please describe the challenges your school faces in using the GES-approved Ghanaian language as the medium of instruction: 8. [If language barrier is specified in question C7] What strategies do teachers at this school employ to help overcome this language barrier when teaching reading in the GES-approved Ghanaian language of instruction? (select all that apply) [Do not prompt, record any relevant answers]  Teach in another Ghanaian language Teach in English Have me or other teachers who speak the language of instruction help them Have pupils translate for them Have pupils translate for one another Use visual cues 155 N/A Other (specify): ____________________________________________________________. 9. In this school, at what level is English used as the primary medium of instruction for reading and writing? (select all that apply) [Do not prompt, record any relevant answers] KG1 KG2 P1 P2 P3 P4 P5 P6 10. In this school, at what level is English used as the primary medium of instruction for subjects other than reading and writing? (select all that apply) [Do not prompt, record any relevant answers]  KG1 KG2 P1 P2 P3 P4 P5 P6 11. Which language or languages would you prefer to be used as the primary language of instruction in your school? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. D. School Resources  [READ] I will now ask several questions about the state of this school, including availability of key resources such as water, toilets, and electricity. For each resource, please state whether that resource is regularly functioning, sometimes not functioning, or rarely or never functioning. This will help us understand if the school environment is conducive for learning. 1. School resources roster: [Enumerator: Please name each resource and ask for an answer for that resource before moving on. You must read all out all of the resources listed here] 156 Resource Don’t have Have, and regularly functioning Have, but sometimes not functioning Have, but rarely or never functioning Electricity O O O O Pipe-borne water O O O O Other water (well, borehole, reservoir) O O O O Vehicle access road (paved) O O O O Vehicle access road (unpaved) O O O O Toilets for pupils O O O O Separate toilets for girls O O O O Separate toilets for teachers O O O O School playground O O O O Free meals scheme - breakfast O O O O Free meals scheme - lunch O O O O School Management Committee O O O O Parent Teacher Association O O O O 2. Does this school have an approved School Performance Improvement Plan for this 2017/18 academic year?  O Yes O No O Don’t know O Decline to answer 3. Does this school have a trained Curriculum Leader?  O Yes O No O Don’t know O Decline to answer 4. How often does a Circuit Supervisor visit this school? (select one)  O At least monthly O Twice or more a term O Once a term O Once a year O Hardly ever O To my knowledge, a Circuit Supervisor has never visited this school 157 O Don’t know O Decline to answer  E. Reading Resources [READ] This next module will focus on the availability of teacher guides, lesson plans, learner textbooks, activity books, and supplementary materials and the extent of access and use. The purpose of this module is to identify possible gaps in resources as well as gauge the degree of use of resources that are available. 1. What teaching and learning materials does your school have available in the GES￾approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers]  Teacher's guide Scripted/step-by-step lesson plans Unscripted/generalized lesson plans Lesson notes Pupil readers or textbooks Pupil workbooks Pupil library books Diagraph and trigraph charts Big books Posters Flash Cards Alphabet Strip Material for teacher-made teaching aids Box of materials sent by Learning Other (specify)_______________________. 2. Are the teaching and learning materials at your school sufficient to enable effective teaching in the GES-approved Ghanaian language of instruction? O Yes  skip to E4 O No O Don’t know  skip to E4 O Decline to answer  skip to E4  3. [If no in question E2 above] Why are the teaching and learning materials not sufficient? (select all that apply) [Do not prompt, record any relevant answers]  There are not enough materials for every pupil There are not enough materials for every teacher or class The materials are outdated The materials are damaged We are not able to use the materials because of teachers don’t know the language We are not able to use the materials because pupils don’t know the language Other (specify): ____________________________________________________________.  4. Which of the following specific reading program materials have been provided to your school in the GES-approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers]  Integrated Approach Let's Read and Write National Literacy Acceleration Program (NALAP) pupils' book NALAP Big Book NALAP teachers' book 158 Enlightening the hearts: early grade reading manual for literacy at lower primary Literacy Learning through Play Learning Early Grade Reading Program Teacher’s Guides Learning Early Grade Reading Program Pupil Workbooks Learning Early Grade Reading Program Readers Other (specify): ____________________________________________________________.  5. Does this school have a library? O Yes O No  skip to F1 O Decline to answer  skip to F1 6. [If yes in question E5 above] Does this library include books that are in the GES￾approved Ghanaian language of instruction?  O Yes O No O Don’t know O Decline to answer  7. Do children take library books home with them? O Yes  skip to F1 O No O Don’t know  skip to F1 O Decline to answer  skip to F1  8. [If no in question E7 above] Why don’t children take library books home with them? (select all that apply) [Do not prompt, record any relevant answers]  They are not allowed to because they will lose or damage them They are not allowed to based on the rules of the sponsor They don’t want to take books home No one at home can read with them They don’t understand the language of the books There are not enough books in the library Other (specify): ____________________________________________________________. F. In-Service Training and Professional Development [READ] This next module will focus on any in-service training and professional development (including coaching) that is provided to teachers at this school. Your answers to these questions will help us identify current gaps in in￾service training and teacher coaching support in Ghana. 1. During the 2017/18 school year, have you attended any formal in-service training or professional development sessions on early grade reading?  O Yes O No  skip to F4 159 O Don’t know  skip to F4 O Decline to answer  skip to F4  2. About how many training days did you receive in the current school year (2017/18)? 3. How many of these training days, if any, were focused on Ghanaian language reading instruction? 4. Have teachers in this school attended any formal in-service training or professional development sessions on early grade reading in the current school year (2017/18)? O Yes O No  skip to F9 O Don’t know  skip to F9 O Decline to answer  skip to F9 5. About how many training days did they receive each (average) in the current school year (2017/18)? 6. How many of these training days, if any, were focused on Ghanaian language reading instruction? 7. Do you feel this training was enough for them to be able to use these methods correctly in the classroom? O Yes O No O Don’t know O Decline to answer 8. Please specify the training provider or providers: (select all that apply) GES Master Trainer NGO Master Trainer Learning Master Trainer District Teacher Support Team Other (specify): ____________________________________________________________. 9. Have teachers in this school attended any school-based in-service (SBI) sessions on Ghanaian language reading instruction this year (2017/18)? O Yes O No  skip to F16 O Don’t know  skip to F16 O Decline to answer  skip to F16 160 10. [If yes to 9] Approximately how many SBI meetings have been held at this school in the 2017/18 school year? 11. [If yes to 9] Do you keep records of dates and/or teacher attendance for these SBI sessions?  O Yes, I keep records of SBI session dates and attendance O Yes, I keep records of SBI session dates only O No, I do not keep records on SBI sessions  skip to F14 O Don’t know  skip to F14 O Decline to answer  skip to F14 12. Are you able to show me these SBI records? Remember, this is only for SBI meetings focused on teaching early grade reading in the Ghanaian language of instruction.  O Yes [ask to retrieve] O No, I am not willing  skip to F14 O No, because I do not have them with me right now  skip to F14 O No, my records are not organized well enough  skip to F14 O I have not separated Ghanaian language SBI data from other SBI we hold at this school  skip to F14 13. [Please record dates for all scheduled SBI meetings, the count of teachers in attendance, and note whether the sampled P1 teacher was in attendance]: SBI meeting date (DD-MM-YY) Total teacher attendance count Was the sampled P1 teacher in attendance? Yes No No record - - - - - - - - - - - - - - - - - - - - - - - - 161 14. Do you believe that teachers are applying what they discuss in SBI meetings to improve upon their classroom teaching practice? [Read each option and record relevant answer] (select one)  O Yes, very well  skip to F16 O Yes, but there is room for improvement  skip to F16 O Yes, but not very well O No, they are not applying what they discuss O Don’t know  skip to F16 O Decline to answer  skip to F16 15. Why do you feel teachers are not applying what they learn in SBI meetings to improve upon their classroom teaching practice?  16. Do you feel teachers in this school need more training in Ghanaian language reading instruction? O Yes O No  skip to F18 O Don’t know  skip to F18 O Decline to answer  skip to F18 17. [If yes in question F16] In which topics would you like them to receive more training? (select all that apply) Reading Writing Assessment Material Usage Other (specify): ____________________________________________________________. 18. Have teachers in your school ever received training in pupil assessment?  O Yes O No O Don’t know O Decline to answer 19. During the current school year (2017/18), have teachers in this school received any mentoring or coaching in teaching reading? O Yes O No  skip to F27 O Don’t know  skip to G1 O Decline to answer  skip to G1 162 20. [If yes in question F19] Who in the current school year (2017/18) provided them with mentoring or coaching in teaching reading? (select all that apply) [Read each option and record relevant answers] Head Teacher (myself) Curriculum Lead Circuit Supervisor District Teacher Support Team Other Teacher(s) at this school Someone from GES / MOE An NGO Other (specify): ____________________________________________________________. 21. [If Head Teacher is selected in question F20 above] What types of mentoring or coaching did you provide? (select all that apply) [Do not prompt, record any relevant answers] Weekly training session at the school Monthly training session at the school Periodic training sessions at the school (less than monthly) Help with reading lesson planning Coaching based on observed lesson Other (specify): ____________________________________________________________. 22. [If coaching based on observed lesson is selected in question F21 above] Did teachers change how they teach reading in response to the feedback you provided? O Yes O No  skip to F24 O Don’t know  skip to F25 O Decline to answer  skip to F25 23. [If yes in question F22 above] In what ways did teachers change how they teach reading in response to your feedback? [Do not prompt, record any relevant answers] Better lesson planning Provide equal opportunities to pupils Group assessment Individual assessment Made class more interactive Other (specify): ____________________________________________________________. 24. [If no in question F22 above] Why did the teachers not change their teaching in response to your feedback? [Do not prompt, record any relevant answers] Lack of skill Lack of teaching materials Lack of understanding of the coaching Difference in beliefs on effective teaching Other (specify): ____________________________________________________________. 163 25. Do you feel this coaching was positively received by the teachers?  O Yes  skip to G1 O No O Don’t know  skip to G1 O Decline to answer  skip to G1 26. [If no in question F25 above] Why was this coaching not received positively?  skip to G1  27. [If Head Teacher was NOT selected in question F20] What are the main reasons you do not provide mentoring or coaching to your teachers in teaching reading? I am too busy with administrative responsibilities I am too busy teaching Providing mentoring or coaching is not my responsibility I don’t have the training or knowledge to provide support I do not want to The teachers do not listen to me when I provide support I do not speak or understand the language that the teacher is using Other (specify): ____________________________________________________________. G. Head Teacher Beliefs on Reading Instructional Methods [READ] I will now read a number of statements about different approaches to teaching reading and ask you to state whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree with that statement. For this module, we want to understand your personal beliefs on teaching reading effectively, even if those beliefs differ from what you actually do, are required to do, or guide others to do. All of your responses are confidential so please answer truthfully. 1. Children should learn to read new words by visually memorizing them as whole units: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 2. Children in Ghana would learn English better if they were taught how to initially read using their mother tongue: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 164 3. In the early grades, it is important to use a lot of class time to teach children to “sound out” words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 4. Children should learn to read syllables before they learn to read whole words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 5. The way I was taught to read as a child is the same approach I should use to teach my pupils: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 6. Children in Ghana would benefit more if they were taught how to initially read using English: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 7. Observing reading lessons and providing coaching and feedback to teachers on their instructional practice is part of my regular duties as Head Teacher:  O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 8. What was the primary language in which you first learned how to read and write? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Gonja O Gurene O Kasem O Kusaal O Nzema 165 O Ewe O Fante O Ga O Decline to answer O Don’t know O Other (specify): ________________________. H. Pupil Performance and Evaluation [READ] I will now ask a few questions about pupil reading performance in your school. The purpose of these questions is to help us understand why some pupils struggle to become fluent readers in Ghana. 1. Do you believe KG and lower primary pupils (KG1-P3) in this school are on track to become fluent readers? O Yes  skip to H3 O No O Don’t know  skip to I1 O Decline to answer  skip to I1 2. [If no in question H1 above] Why do you think lower primary pupils are not on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers] They don’t come to class often enough They don’t come to class on time They don’t pay attention during class Their parents do not support their reading at home They are too hungry to concentrate They don’t care about school They are not able to understand the language of instruction The class is too large for teachers to provide good instruction The lessons are too short Teachers don’t always teach the lessons because they are doing other things Teachers don’t show up for work Teachers don’t speak the language of instruction The teachers don’t do a good job teaching There are not enough teachers at this school Teachers vacate their postings too often We don’t have the teaching resources we need to teach them well We don’t have the training we need to teach them well The pupils don’t have enough learning books The pupils don’t have access to books outside of school Other (specify)_____________________________________________________________. 3. How do you know whether they are on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers]  We assess them regularly Teachers tell me they are doing OK They seem to be doing OK when I observe them in class 166 I don’t know Other (specify)_____________________________________________________________. I. Early Grade Reading Programming 1. Have any early grade reading programs operated at this school in the 2017/18 academic year?  O Yes O No  skip to J1 O Don’t know  skip to J1 O Decline to answer  skip to J1 2. [If yes in question I11 above] Please share with us the names of the program(s), the organization leading the program(s), and provide a brief description:  J. Teacher Rationalization and Deployment [READ] In this final module, I will ask some questions on teacher recruitment, deployment, and retention. Your answers will help us understand how the movement of teachers influences early grade reading performance. 1. How do you determine the number of teachers you need at this school?  2. When a teaching position is vacant in your school, how do you go about filling it? (select all that apply) [Do not prompt, record any relevant answers]  I report the vacancy to the district Director of Education and request for a teacher I recruit the teacher and submit the name to the district director for approval I request the School Management Committee to recruit a replacement teacher I request for teachers from the District Assembly I request for teachers from the National Service Secretariat I request for teachers from National Youth Employment I don’t know Other (specify)_____________________________________________________________. 3. In your opinion, do some schools in this district have a surplus of teachers while other schools in the district have a shortage?  167 O Yes O No O Don’t know  skip to J5 O Decline to answer  skip to J5 4. [If yes in J3] Do you know the reasons why some schools in this district have more teachers than they need while others do not have enough?  5. Of the new teachers posted to this school during your tenure, how many are professionally trained or have been duly registered as professional teachers by GES: none, some, most, or all? [select one]  O None O Some (less than half) O Most (more than half) O All O I have not observed new teachers posted here during my tenure O Don’t know O Decline to answer 6. Of the new teachers posted to this school during your tenure, how many have the appropriate competencies and skills to teach reading in lower primary using the GES-approved Ghanaian language of instruction of the school: none, some, most, or all? (select one)  O None O Some (less than half) O Most (more than half) O All O I have not observed new teachers posted here during my tenure O Don’t know O Decline to answer 7. During your tenure, how many teachers have transferred or vacated their positions at this school before serving 5 full years?  O None  END O Some (less than half) O Most (more than half) O All O Don’t know  END O Decline to answer  END 8. In your opinion, what are the main reasons teachers have transferred or vacated their positions at this school before serving 5 full years? 168  Thank you! That completes the Head Teacher questionnaire. End Time (HH:MM): : Ghana Early Grade Reading Program Impact Evaluation Midline Curriculum Lead Questionnaire (Treatment Only) General Background Information Team name: Enumerator ID: Today’s date (DD-MM-YY): - - Start time (HH:MM): : School name: School ID: Region: District: Circuit: Circuit code: Locality: Locality type: O Urban (more than 5,000 people living in community/locality) O Rural (less than 5,000 people living in community/locality) School address: 169 Was the consent form administered and signed? O Yes O No (DO NOT PROCEED) Curriculum Lead Identifying Information First name: Last name: Sex: O Male O Female Telephone: A. Curriculum Lead Background Information  1. How long have you been the Curriculum Lead at this school? Years: Months: 2. How long have you been in the teaching profession overall? Years: Months: 3. Have you completed teacher training college? O Yes O No O Decline to answer 4. What is the highest level of education that you have completed? (select one) [Do not read options] O Middle School O Senior High School O Ordinary Level O Advance Level O Certificate A O Diploma in Basic Education O Higher National Diploma (HND) O Technical/Vocational (NVTI) O Bachelor’s of Education (B.Ed) O Other Bachelor’s Degrees O Post Graduate Diploma in Education O Masters of Education (M.Ed) O Other Master’s Degree O Ph.D. O Decline to answer O Other (specify): ________________________. 5. Does your school have more than one Curriculum Lead?  O Yes O No O Don’t know O Decline to answer  6. Which class(es) are you responsible for as Curriculum Lead? (select all that apply) KG1 KG2 P3 P4 170 P1 P2 P5 P6  7. In addition to your duties as Curriculum Lead, do you teach classes at this school?  O Yes O No  skip to A11 O Decline to answer  skip to A11  8. [If yes to A7] Which grades do you teach? (select all that apply) KG1 KG2 P1 P2 P3 P4 P5 P6  9. [If yes to A7] In a typical week, approximately how many hours do you work at this school?  10. [If yes to A7] Of these, how many hours are spent teaching?  11. For how many days last week (Monday thru Friday) was this school open / in session? Please include partial days (e.g., 0.5 days). 12. Of the $[A11] days that the school was open last week, for how many days were you absent? Please include partial absences (e.g., 0.5 days).  B. Language Background [READ] I will now ask you some questions about the use of different languages in this school. The purpose of this module is to help us understand how Ghana’s linguistic diversity affects the ability of teachers to teach reading as well as the ability of pupils to become fluent readers. 1. For each of the following languages, please state whether you speak the language fluently, partially, or not at all [Enumerator: Please name each language and ask for an answer for that language before moving on. You must read out all of the languages] Language Fluently Partially Not at all Akuapem Twi O O O Asante Twi O O O Dagaare O O O Dagbani O O O 171 Dangme O O O English O O O Ewe O O O Fante O O O Ga O O O Gonja O O O Kasem O O O Nzema O O O 2. What language(s) do children in your school speak on the playground? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 3. What is the GES-approved Ghanaian language of instruction for lower primary at this school? (select one) [Do not prompt] O Akuapim Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Kasem O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. 4. How many lower primary teachers (KG1-P3) at this school use the GES-approved Ghanaian language of instruction when teaching?   5. Of these, how many teach comfortably in the GES-approved Ghanaian language of instruction?   6. What other languages besides the GES-approved Ghanaian language of instruction do KG and lower primary teachers (KG1-P3) use to teach at this school? (select all that apply) [Do not prompt, record any relevant answers]  172 Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 7. Is your school facing any challenges in using the GES-approved Ghanaian language as the medium of instruction? O Yes O No  skip to B10 O Don’t know  skip to B10 O Decline to answer  skip to B10 8. [If yes in question B7] Please describe the challenges your school faces in using the GES-approved Ghanaian language as the medium of instruction: 9. [If language barrier is specified in question B8] What strategies do teachers at this school employ to help overcome this language barrier when teaching reading in the GES-approved Ghanaian language of instruction? (select all that apply) [Do not prompt, record any relevant answers]  Teach in another Ghanaian language Teach in English Have me or other teachers who speak the language of instruction help them Have pupils translate for them Have pupils translate for one another Use visual cues N/A Other (specify): ____________________________________________________________. 10. In this school, at what level is English used as the primary medium of instruction for reading and writing? (select all that apply) [Do not prompt, record any relevant answers] KG1 KG2 P1 P3 P4 P5 173 P2 P6  11. Which language or languages do you prefer to be used as the primary language of instruction in your school? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________.  12. What teaching and learning materials does your school have available in the GES￾approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers]  Teacher's guide Scripted/step-by-step lesson plans Unscripted/generalized lesson plans Lesson notes Pupil readers or textbooks Pupil workbooks Pupil library books Diagraph and trigraph charts Big books Posters Flash Cards Alphabet Strip Material for teacher-made teaching aids Box of materials sent by Learning Other (specify)_______________________. 13. Are the teaching and learning materials at your school sufficient to enable effective teaching in the GES-approved Ghanaian language of instruction? O Yes  skip to C1 O No O Don’t know  skip to C1 O Decline to answer  skip to C1  14. [If no in question B13 above] Why are the teaching and learning materials not sufficient? (select all that apply)  There are not enough materials for every pupil There are not enough materials for every teacher or class The materials are outdated The materials are damaged Other (specify): ____________________________________________________________. C. In-Service Training and Professional Development 174 [READ] This next module will focus on any in-service training and professional development (including coaching) that is provided to teachers at this school. Your answers to these questions will help us identify current gaps in in￾service training and teacher coaching support in Ghana. 1. During the 2017/18 school year, have you attended any formal in-service training or professional development sessions on early grade reading?  O Yes O No  skip to C4 O Don’t know  skip to C4 O Decline to answer  skip to C4  2. About how many training days did you receive in the current school year (2017/18)? 3. How many of these training days, if any, were focused on Ghanaian language reading instruction? 4. Have teachers in this school attended any formal in-service training or professional development sessions on early grade reading in the current school year (2017/18)? O Yes O No  skip to C9 O Don’t know  skip to C9 O Decline to answer  skip to C9 5. About how many training days did they receive each (average) in the current school year (2017/18)? 6. How many of these training days, if any, were focused on Ghanaian language reading instruction? 7. Do you feel this training was enough for them to be able to use these methods correctly in the classroom? O Yes O No O Don’t know O Decline to answer 8. Please specify the training provider or providers: (select all that apply) Myself GES Master Trainer NGO Master Trainer Learning Master Trainer 175 District Teacher Support Team Other (specify): ____________________________________________________________. 9. Have teachers in this school attended any school-based in-service (SBI) sessions on Ghanaian language reading instruction this year (2017/18)? O Yes O No  skip to C16 O Don’t know  skip to C16 O Decline to answer  skip to C16 10. [If yes to C9] Approximately how many SBI meetings have been held at this school in the 2017/18 school year? 11. [If yes to C9] Do you keep records of dates and/or teacher attendance for these SBI sessions?  O Yes, I keep records of SBI session dates and attendance O Yes, I keep records of SBI session dates only O No, I do not keep records on SBI sessions  skip to C14 O Don’t know  skip to C14 O Decline to answer  skip to C14 12. Are you able to show me these SBI records? Remember, this is only for SBI meetings focused on teaching early grade reading in the Ghanaian language of instruction.  O Yes [ask to retrieve] O No, I am not willing  skip to C14 O No, because I do not have them with me right now  skip to C14 O No, my records are not organized well enough  skip to C14 O I have not separated Ghanaian language SBI data from other SBI we hold at this school  skip to C14 13. [Please record dates for all scheduled SBI meetings, the count of teachers in attendance, and note whether the sampled P1 teacher was in attendance]: SBI meeting date (DD-MM-YY) Total teacher attendance count Was the sampled P1 teacher in attendance? Yes No No record - - - - - - - - - - - - 176 - - - - - - - - - - - - 14. Do you believe that teachers are applying what they discuss in SBI meetings to improve upon their classroom teaching practice? [Read each option and record relevant answer] (select one)  O Yes, very well  skip to C16 O Yes, but there is room for improvement  skip to C16 O Yes, but not very well O No, they are not applying what they discuss O Don’t know  skip to C16 O Decline to answer  skip to C16  15. Why do you feel teachers are not applying what they learn in SBI meetings to improve upon their classroom teaching practice?  16. Do you feel teachers in this school need more training in Ghanaian language reading instruction? O Yes O No  skip to C18 O Don’t know  skip to C18 O Decline to answer  skip to C18 17. [If yes in question C16] In which topics would you like them to receive more training? (select all that apply) Reading Writing Assessment Material Usage Other (specify): ____________________________________________________________. 18. Have teachers in your school ever received training in pupil assessment?  O Yes O No O Don’t know 177 O Decline to answer  19. During the current school year (2017/18), have teachers in this school received any mentoring or coaching in teaching reading? O Yes O No  skip to C27 O Don’t know  skip to D1 O Decline to answer  skip to D1 20. [If yes in question C19 above] Who in the current school year (2017/18) provided them with mentoring or coaching in teaching reading? (select all that apply) [Read each option and record relevant answers] Head Teacher Curriculum Lead (myself) Circuit Supervisor District Teacher Support Team Other Teacher(s) at this school Someone from GES / MOE An NGO Other (specify): ____________________________________________________________.  21. [If Curriculum Lead is selected in question C20 above] What types of mentoring or coaching did you provide? (select all that apply) [Do not prompt, record any relevant answers] Weekly training session at the school Monthly training session at the school Periodic training sessions at the school (less than monthly) Help with reading lesson planning Coaching based on observed lesson Other (specify): ____________________________________________________________. 22. [If coaching based on observed lesson is selected in question C21 above] Did teachers change how they teach reading in response to the feedback you provided? O Yes O No  skip to C24 O Don’t know  skip to C25 O Decline to answer  skip to C25 23. [If yes in question C22 above] In what ways did teachers change how they teach reading in response to your feedback? [Do not prompt, record any relevant answers] Better lesson planning Provide equal opportunities to pupils Group assessment Individual assessment Made class more interactive Other (specify): ____________________________________________________________. 178 24. [If no in question C22 above] Why did the teachers not change their teaching in response to your feedback? [Do not prompt, record any relevant answers] Lack of skill Lack of teaching materials Lack of understanding of the coaching Difference in beliefs on effective teaching Other (specify): ____________________________________________________________. 25. Do you feel this coaching was positively received by the teachers?  O Yes  skip to D1 O No O Don’t know  skip to D1 O Decline to answer  skip to D1 26. [If no in question C25 above] Why was this coaching not received positively?  skip to D1  27. [If Curriculum Lead was NOT selected in question C20] What are the main reasons you do not provide mentoring or coaching to your teachers in teaching reading? I am too busy with administrative responsibilities I am too busy teaching Providing mentoring or coaching is not my responsibility I don’t have the training or knowledge to provide support I do not want to The teachers do not listen to me when I provide support I do not speak or understand the language that the teacher is using Other (specify): ____________________________________________________________. 179 D. Curriculum Lead Beliefs on Reading Instructional Methods [READ] I will now read a number of statements about different approaches to teaching reading and ask you to state whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree with that statement. For this module, we want to understand your personal beliefs on teaching reading effectively, even if those beliefs differ from what you actually do, are required to do, or guide others to do. All of your responses are confidential so please answer truthfully. 1. Children should learn to read new words by visually memorizing them as whole units: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 2. Children in Ghana would learn English better if they were taught how to initially read using their mother tongue: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 3. In the early grades, it is important to use a lot of class time to teach children to “sound out” words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 4. Children should learn to read syllables before they learn to read whole words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 5. The way I was taught to read as a child is the same approach I should use to teach my pupils: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 180 6. Children in Ghana would benefit more if they were taught how to initially read using English: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 7. Observing reading lessons and providing coaching and feedback to teachers on their instructional practice is part of my regular duties as Curriculum Lead:  O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 8. What was the primary language in which you first learned how to read and write? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Gurene O Kasem O Kusaal O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. E. Pupil Performance and Evaluation  [READ] I will now ask a few questions about pupil performance and assessment in your school. The purpose of these questions is to help us understand why some pupils struggle to become fluent readers in Ghana. 1. Do you believe KG and lower primary pupils (KG1-P3) in this school are on track to become fluent readers? O Yes  skip to E3 O No O Don’t know  skip to F1 O Decline to answer  skip to F1 2. [If no in question E1 above] Why do you think lower primary pupils are not on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers] They don’t come to class often enough They don’t come to class on time They don’t pay attention during class Their parents do not support their reading at home They are too hungry to concentrate 181 They don’t care about school They are not able to understand the language of instruction The class is too large for teachers to provide good instruction The lessons are too short Teachers don’t always teach the lessons because they are doing other things Teachers don’t show up for work Teachers don’t speak the language of instruction The teachers don’t do a good job teaching There are not enough teachers at this school Teachers vacate their postings too often We don’t have the teaching resources we need to teach them well We don’t have the training we need to teach them well The pupils don’t have enough learning books The pupils don’t have access to books outside of school Other (specify)_____________________________________________________________.  3. How do you know whether they are on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers]  We assess them regularly Teachers tell me they are doing OK They seem to be doing OK when I observe them in class I don’t know Other (specify)_____________________________________________________________.  F. EGR Program Implementation  [READ] I will now ask a few questions on how the Early Grade Reading program has been going at this school. As with all the questions we are asking, your answers are strictly confidential. Your individual responses will only be viewed by the independent evaluation team, so please share your thoughts freely and openly. Your responses will be compiled with hundreds of other Curriculum Leads’ responses and will help USAID/Learning improve the Early Grade Reading program. 1. Do you believe that teachers at this school are motivated and happy about the new Ghanaian language Early Grade Reading program?  O Yes  skip to F3 O No O Don’t know  skip to F3 O Decline to answer  skip to F3 2. [If no in question F1 above] Why are they not motivated and happy about the program?  182 3. Do you believe that teachers in this school are more likely to use phonics-based methods for teaching reading as a result of the Early Grade Reading Program?  O Yes  skip to F5 O No O Don’t know  skip to F5 O Decline to answer  skip to F5 4. [If no in question F3 above] Why are they not more likely to use phonics-based methods?  5. Do you believe that teachers in this school are more likely to use Ghanaian language for teaching reading as a result of the Early Grade Reading Program?  O Yes  skip to F7 O No O Don’t know  skip to F7 O Decline to answer  skip to F7 6. [If no in question F5 above] Why are they not more likely to use Ghanaian language for teaching reading?  7. Are Circuit Supervisors at this school conducting visits to monitor implementation of the Early Grade Reading program?  O Yes O No  skip to F11 O Don’t know  skip to F11 O Decline to answer  skip to F11 8. [If yes in question F7 above] Do they observe and assess teacher performance using an electronic tablet?  O Yes O No  skip to F11 O Don’t know  skip to F11 O Decline to answer  skip to F11 183 9. [If yes in question F8 above] Can you describe how (if at all) the monitoring data they collect on the tablet is shared with the school?  10. [If data is shared in F9 above] How do practices at this school change (if at all) in response to this monitoring data?  11. Please describe challenges (if any) this school faces in implementing the Early Grade Reading Program:  Thank you! That completes the Curriculum Lead questionnaire. End Time (HH:MM): : 184 Ghana Early Grade Reading Program Impact Evaluation Midline Teacher Questionnaire General Background Information Team name: Enumerator ID: Today’s date (DD-MM-YY): - - Start time (HH:MM): : School name: School ID: Region: District: Locality: School address: Was the consent form administered and signed? O Yes O No (DO NOT PROCEED) Teacher Identifying Information First name: Last name: Sex: O Male O Female Telephone: 185 A. Teacher Background Information 1. Did someone from ILC Africa assess your pupils in May or June of last year? O Yes O No O Decline to answer 2. Did someone from ILC Africa interview you in May or June of last year? O Yes  skip to A7 O No O Decline to answer 3. How long have you been a teacher at this school? Years: Months: 4. How long have you been a teacher overall? Years: Months: 5. Have you completed teacher training college? O Yes O No O Decline to answer 6. What is the highest level of education that you have completed? (select one) [Do not read options] O Middle School Leaving O Senior High School O GCE Ordinary Level O GCE Advance Level O Certificate A O Diploma in Basic Education O Higher National Diploma (HND) O Technical/Vocational O Bachelor’s of Education (B.Ed) O Other Bachelor’s Degrees O Post Graduate Diploma in Education O Masters of Education (M.Ed.) O Other Master’s Degree O Ph.D. O Decline to answer O Other (specify): ________________________. 7. What is your current rank? [Do not read options] O Deputy Director O Assistant Director I O Assistant Director II O Principal Superintendent O Senior Superintendent I O Senior Superintendent 1I O Superintendent I O Superintendent II O Pupil Teacher (WASSCE/SSCE) O Not applicable O Don’t know O Decline to answer O Other (specify): ________________________. 8. Which grades do you teach? (select all that apply) KG1 P3 186 KG2 P1 P2 P4 P5 P6 9. For how many days last week (Monday thru Friday) was this school open / in session? Please include partial days (e.g., 0.5 days). 10. Of the $[A9] days that the school was open last week, for how many days were you absent? Please include partial absences (e.g., 0.5 days). B. Language Background [READ] I will now ask you some questions about your language background and skills as well as the use of different languages in this school. The purpose of this module is to help us understand how Ghana’s linguistic diversity affects the ability of teachers to teach reading as well as the ability of pupils to become fluent readers. Remember, your answers are strictly confidential so please feel free to answer honestly. Some of these questions are similar, but there are slight yet important differences so please listen carefully and let me know if you require any clarifications. 1. For each of the following languages, please state whether you speak the language fluently, partially, or not at all [Enumerator: Please name each language and ask for an answer for that language before moving on. You must read out all of the languages] Language Fluently Partially Not at all Akuapem Twi O O O Asante Twi O O O Dagaare O O O Dagbani O O O Dangme O O O English O O O Ewe O O O Fante O O O Ga O O O Gonja O O O Kasem O O O Nzema O O O 2. What language(s) do children in your reading class speak on the playground? (select all that apply) [Do not prompt, record any relevant answers] Ahanta Akuapem Twi Asante Twi Bono Cherepon Fante Ga Hausa Gonja Gua Mol Nabt Sissale Sefwi Tale 187 Dagaare Dagbani Dangme English Ewe Gurene Kasem Kusaal Likpakpaln Mampruli Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 3. What language(s) do you use to explain new material to the pupils? (select all that apply) [Do not prompt, record any relevant answers] Akuapem Twi Asante Twi Dagaare Dagbani Dangme English Ewe Fante Ga Gonja Gurene Kasem Kusaal Nzema Decline to answer Other (specify): ________________________. 4. What languages do you teach your pupils to write in? (select all that apply) [Do not prompt, record any relevant answers] Akuapem Twi Asante Twi Dagaare Dagbani Dangme English Ewe Fante Ga Gonja Gurene Kasem Kusaal Nzema Decline to answer Other (specify): ________________________. 5. In what language(s) are exams given to your pupils? (select all that apply) [Do not prompt, record any relevant answers] Akuapem Twi Asante Twi Dagaare Dagbani Dangme English Ewe Fante Ga Gonja Gurene Kasem Kusaal Nzema Decline to answer Other (specify): ________________________. 6. What is the GES-approved Ghanaian language of instruction for lower primary at this school? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Ga O Gonja 188 O Dagaare O Dagbani O Dangme O Ewe O Fante O Kasem O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. 7. Approximately what percentage of the pupils in your reading class speak the GES-approved Ghanaian language of instruction? 8. How confident do you feel about speaking the GES-approved Ghanaian language of instruction? O Very confident O Somewhat confident O Not very confident O Not at all confident 9. How confident do you feel about reading the GES-approved Ghanaian language of instruction? O Very confident O Somewhat confident O Not very confident O Not at all confident 10. How confident do you feel about teaching in the GES-approved Ghanaian language of instruction? O Very confident O Somewhat confident O Not very confident O Not at all confident 11. Are you facing any challenge(s) in using the GES-approved Ghanaian language as the medium of instruction for your class? O Yes O No  skip to B14 O Don’t know  skip to B14 O Decline to answer  skip to B14 12. [If yes to B11 above] Please describe the challenges you face in using the GES-approved Ghanaian language as the medium of instruction for your class: 189 13. [If language barrier is specified in B12 above] What strategies do you employ to help overcome this language barrier when teaching reading in the GES-approved Ghanaian language of instruction? (select all that apply) [Do not prompt, record any relevant answers] Teach in another Ghanaian language Teach in English Have other teachers who speak the language of instruction help me Have pupils translate for me Have pupils translate for one another Use visual cues N/A or no answer Other (specify): ____________________________________________________________. 14. Are there pupils in your classroom with whom you cannot at all communicate in any Ghanaian language? O Yes O No  skip to C1 O Decline to answer  skip to C1 15. Approximately what percentage of the pupils in your reading class are you unable to communicate with in any Ghanaian language? 16. How do you deal with pupils with whom you cannot communicate in any Ghanaian language? C. Classroom Enrollment, Attendance, and Lesson Scheduling [READ] I will now ask a few quick questions on enrollment, attendance, and lesson scheduling. Class size, attendance patterns, and lesson duration may help explain why some pupils in Ghana become fluent readers and some do not. 1. How many boys are enrolled in your P1 reading class? 2. How many girls are enrolled in your P1 reading class? 3. Please estimate how many of your enrolled pupils are absent from the reading class on a typical day: 190 4. Please estimate how many of your enrolled pupils miss more than 20 minutes of the reading class (i.e., are tardy but not absent) on a typical day? 5. In a typical 5-day school week, on how many days does your P1 class receive a reading lesson? 6. What is the duration in minutes of a typical daily reading lesson for your class? 7. Of this time, how many minutes are devoted to English instruction? D. Reading Resources [READ] This next module will focus on the availability of teacher guides, lesson plans, learner textbooks, activity books, and supplementary materials and the extent of access and use. The purpose of this module is to identify possible gaps in resources as well as gauge the degree of use of resources that are available. 1. What teaching and learning materials do you have available in the GES-approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers] Teacher's guide Scripted/step-by-step lesson plans Unscripted/generalized lesson plans Lesson notes Pupil readers or textbooks Pupil workbooks Pupil library books Diagraph and trigraph charts Big books Posters Flash Cards Alphabet Strip Material for teacher-made teaching aids Box of materials sent by Learning Other (specify)_______________________. 2. For each of the materials you indicated having available in question D1, please specify whether you use it daily, weekly, monthly, termly, or rarely or never: Materials Daily Weekly Monthly Termly Rarely / Never Don’t Have Teacher's guide O O O O O O Scripted/step-by-step lesson plans O O O O O O Unscripted/generalized lesson plans O O O O O O Lesson notes O O O O O O Pupil readers or textbooks O O O O O O Pupil workbooks O O O O O O Pupil library books O O O O O O Diagraph and trigraph charts O O O O O O 191 Big books O O O O O O Posters O O O O O O Flash Cards O O O O O O Alphabet Strip O O O O O O Material for teacher-made teaching aids O O O O O O Box of materials sent by Learning O O O O O O Other (specify)__________________. O O O O O O 3. [For materials reported in D2 as being used rarely or never] Why don’t you regularly use the materials in the classroom? (select all that apply) [Do not prompt, record any relevant answers] There are not enough for everyone The children don’t understand the language of the materials I am not comfortable teaching the language of the materials The children will damage or lose them The materials we have are already damaged The materials are not age or grade appropriate The content is not good or outdated This school does not use the GES-approved Ghanaian language of instruction Other (specify): ____________________________________________________________. 4. Do you have sufficient teaching and learning materials to enable effective teaching in the GES￾approved Ghanaian language of instruction? O Yes  skip to D6 O No O Don’t know  skip to D6 O Decline to answer  skip to D6 5. [If no to D4] Why are the teaching and learning materials not sufficient? (select all that apply) There are no materials There are not enough materials for every pupil There are not enough materials for every teacher or class The materials are not good or outdated The materials are damaged I am not able to use the materials because I don’t understand the language I am not able to use the materials because my pupils don’t understand the language The materials are not age or grade appropriate Other (specify): ____________________________________________________________. 6. Which of the following specific reading program materials have been provided to your school/class in the GES-approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers] Integrated Approach Let's Read and Write National Accelerated Literacy Program (NALAP) pupils' book NALAP Big Book 192 NALAP teachers' book Enlightening the hearts: early grade reading manual for literacy at lower primary Literacy Learning through Play Learning Early Grade Reading Program Teacher’s Guides Learning Early Grade Reading Program Pupil Workbooks Learning Early Grade Reading Program Readers Other (specify): ____________________________________________________________. 7. Which of the following resources are your pupils allowed to take home from school? (select all that apply) [Read each option and record relevant answers] Pupil readers or textbooks Pupil workbooks Library books Flash Cards Homework books Alphabet Strip Other (specify)_______________________. N/A or none of these 8. What other teaching and learning materials do you have available in language(s) other than the GES￾approved Ghanaian language of instruction? (select all that apply) [Read each option and record relevant answers] Teacher's guide Scripted/step-by-step lesson plans Unscripted/generalized lesson plans Lesson notes Pupil readers or textbooks Pupil workbooks Pupil library books Big books Posters Flash Cards Alphabet Strip Material for teacher-made teaching aids Box of materials sent by Learning Other (specify)_______________________. 9. I will now ask a few questions about the lesson plan book or folder that you are presently using to inform your day-to-day teaching. Can I kindly request you to show me the book or folder? O Yes O No, I do not have a lesson plan book or folder  skip to E1 O No, I decline to show you  skip to E1 10. [Record the name of the book or books that the teacher presents]: 11. [Please select the type of item that the teacher presents]: O Scripted/step-by-step daily lesson plan O Unscripted/generalized daily lesson plan  skip to E1 O Lesson notes  skip to E1 O Weekly work schemes  skip to E1 O Other (specify) __________________________________.  skip to E1 12. How effective are the scripted/step-by-step daily lesson plans in helping you be a better reading and writing teacher? 193 O Very effective O Somewhat effective O Neutral / no opinion O Somewhat ineffective O Very ineffective 13. How effective are the scripted/step-by-step daily lesson plans in helping your pupils learn to read and write? O Very effective O Somewhat effective O Neutral / no opinion O Somewhat ineffective O Very ineffective 14. How easy or difficult are the scripted/step-by-step daily lesson plans to use in your classroom? O Very easy O Somewhat easy O Neutral / no opinion O Somewhat difficult O Very difficult 15. For most of my pupils, the pace of the scripted/step-by-step daily lesson plans is: O Much too fast O A little too fast O Just the right pace O A little too slow O Much too slow 16. How often do you go “off script” or make up your own lesson activities that are not in the scripted/step-by-step daily lesson plans? O I go off script the majority of the time O Sometimes, but I stick to the script most of the time O I rarely go off script  skip to D18 O I never go off script  skip to D18 17. Please describe in your own words your reasons for going off script: 18. How much pressure do you feel to maintain the pace of the scripted/step-by-step daily lesson plans (even if your pupils need to go faster or slower)? O No pressure O A little pressure O Neutral/no opinion O Some pressure O A lot of pressure 194 19. Do you like using the scripted/step-by-step lesson plan? O Yes  skip to D21 O No O Decline to answer  skip to D21 20. Why don’t you like using the scripted lesson plan? 21. In what ways, if any, do you think the scripted lesson plans can be improved upon? E. In-Service Training and Professional Development [READ] This next module will focus on any in-service training and professional development (including coaching) that you have received. Your answers to these questions will help us identify current gaps in in-service training and teacher coaching support in Ghana. 1. Have you ever attended any formal in-service training or professional development sessions on early grade reading and writing? O Yes O No  skip to E7 O Don’t know  skip to E7 O Decline to answer  skip to E7 2. Have you attended any formal in-service training or professional development sessions on early grade reading in the current school year (2017/18)? O Yes O No  skip to E7 O Don’t know  skip to E7 O Decline to answer  skip to E7 3. About how many early grade reading training days did you receive in total over the current school year (2017/18)? 4. How many of these training days, if any, were for Ghanaian language reading instruction? [If no training in Ghanaian language reading instruction was provided, enter 0]. 195 5. Do you feel this training was enough for you to be able to use these methods correctly in your classroom? O Yes  skip to E7 O No O Don’t know O Decline to answer  skip to E7 6. What, if anything, can be done to better support you in using these methods correctly? 7. Have you attended any school-based in-service (SBI) sessions on Ghanaian language reading instruction this year (2017/18)? O Yes O No  skip to E12 O Don’t know  skip to E12 O Decline to answer  skip to E12 8. Approximately how many SBI sessions on Ghanaian language reading instruction have you attended this year (2017/18)? 9. Do you believe these SBI sessions have improved your ability to teach reading? O Yes O No  skip to E11 O Don’t know  skip to E12 O Decline to answer  skip to E12 10. [If yes to E9] In what ways have the SBI sessions improved your ability to teach reading? 11. [If no to E9] Why do you think the SBI sessions have not improved your ability to teach reading? 12. Have you ever received training in pupil assessment? O Yes O No O Don’t know O Decline to answer 196 13. During the current school year (2017/18), have you received any mentoring or coaching in teaching reading? O Yes O No  skip to F1 O Don’t know  skip to F1 O Decline to answer  skip to F1 14. [If yes to E13] Who in the current school year (2017/18) provided you with mentoring or coaching in teaching reading? (select all that apply) [Read each option and record relevant answers] Head Teacher Curriculum Lead Circuit Supervisor District Teacher Support Team (DTST) Other Teacher(s) at this school Someone from GES / MOE An NGO Other (specify): ____________________________________________________________. 15. [If yes to E13] What types of mentoring or coaching did you receive? (select all that apply) [Do not prompt, record any relevant answers] Weekly training session at the school Monthly training session at the school Periodic training session at the school (less than monthly) Help with reading lesson planning Coaching based on observed lesson Other (specify): ____________________________________________________________. 16. [If coaching based on observed lesson is selected in E15] When you are being observed by a coach, do you in any way modify your teaching approach, as compared to an ordinary day? O Yes O No  skip to E18 O Don’t know  skip to E18 O Decline to answer  skip to E18 17. [If yes to E16] In what ways do you modify your teaching approach when being observed? 18. I will now ask some questions about the amount of mentoring or coaching that you have received as well as how helpful you found it to be. Remember, your individual responses will not be shared with anyone outside of the research team, so please feel free to share openly and honestly. Of the following list of possible supervision and/or coaching providers, please estimate the approximate number of hours each provider supervised/coached you in the last two full terms (T1 197 and T2 of the 2017/18 school year) and then rate each coaching provider on a scale of 1-5 in accordance with the following scale: 1=very unhelpful, 2=somewhat unhelpful, 3=neither helpful nor unhelpful, 4=somewhat helpful, 5=very helpful. Coaching provider HOURS in the last 2 full terms (T1 and T2) Rating (1=very unhelpful; 5=very helpful) Head Teacher Curriculum Lead Circuit Supervisor District Teacher Support Team Literacy Specialist Other (specify): _________________. 19. What were the most useful aspects of the coaching sessions? 20. What were the least useful aspects of the coaching sessions? F. Teacher Knowledge, Attitudes, and Practices [READ] I will now read a number of statements about different approaches to teaching reading and ask you to state whether you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree with that statement. For this module, we want to understand your personal beliefs on teaching reading effectively, even if those beliefs differ from what you actually do or are told to do. All of your responses are confidential so please answer truthfully. 1. Children should learn to read new words by visually memorizing them as whole units: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 2. Children in Ghana would learn English better if they were taught how to initially read using their mother tongue: O Strongly agree 198 O Agree O Neither agree nor disagree O Disagree O Strongly disagree 3. In the early grades, it is important to use a lot of class time to teach children to “sound out” words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 4. Children should learn to read syllables before they learn to read whole words: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 5. The way I was taught to read as a child is the same approach I should use to teach my pupils: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 6. Children in Ghana would benefit more if they were taught how to initially read using English: O Strongly agree O Agree O Neither agree nor disagree O Disagree O Strongly disagree 7. What was the primary language in which you first learned how to read and write? (select one) [Do not prompt] O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Gurene O Kasem O Kusaal O Nzema O Decline to answer O Don’t know O Other (specify): ________________________. Whereas the previous questions were focused on your personal beliefs, this next series of questions will be focused on your actual practices in the classroom. 199 8. For each of the following practices, please state whether you engage in that practice during your reading lessons daily, weekly, monthly, rarely, or not at all: [Enumerator: Please name each practice and ask for an answer for that practice before moving on. You must read out all of the practices listed here] Practice Daily Weekly Monthly Rarely Not at all Assess learner learning O O O O O Introduce lesson by connecting to what learners have learned previously O O O O O Use a lesson plan O O O O O Have individual learners read aloud O O O O O Engage learners in reading activities or games appropriate to reading level O O O O O Ask learners questions to assess their understanding of something they learn O O O O O Provide learners with structured opportunities to apply understanding and skills to everyday life and problems O O O O O Encourage learners to “sound it out” when they don’t know a word O O O O O Ask learners questions to assess their understanding of stories they hear O O O O O Ask learners to recognize letters and say letter names and/or sounds O O O O O Provide instructions on how to decode syllables and words O O O O O Apply multiple methods to support comprehension, including games, group work, etc. O O O O O Asks learners pre-reading questions O O O O O 9. Which of the following teaching methods do you use during your reading lessons to help teach letters of the alphabet? (select all that apply) [Read each option and record relevant answers] Use letter games Teach letters through songs Use alphabet cards Use word and/or sentence cards Post letters on class wall Present letters and ask pupils to name them Other (specify)_____________________________________________________________. I don’t teach this in my class 10. Which of the following teaching methods do you use during your reading lessons to improve pupils’ vocabulary? (select all that apply) [Read each option and record relevant answers] 200 Teach specific words before reading Repeat vocabulary words in many contexts Use dictionaries and other reference aids Use pictures Write definitions on wall or board Give synonyms Other (specify)_____________________________________________________________. I don’t teach this in my class 11. Which of the following teaching methods do you use during your reading lessons to improve phonic awareness? (select all that apply) [Read each option and record relevant answers] Present letters and ask for the sounds they produce Identify words or sounds that are the same or different Produce words that rhyme Segment or separate words into syllables Clap words or syllables Explain vowels and consonants Identify the first sounds in a word Blend letter sounds and/or syllables into words Other (specify)_____________________________________________________________. I don’t teach this in my class 12. Which of the following teaching methods do you use during your reading lessons to improve reading speed and accuracy? (select all that apply) [Read each option and record relevant answers] Guided oral reading Silent reading Choral reading (reading as a group) Echo reading (read after me) Model reading by teacher Have pupils follow words with a finger Other (specify)_____________________________________________________________. I don’t teach this in my class 13. Which of the following teaching methods do you use during your reading lessons to improve reading comprehension? (select all that apply) [Read each option and record relevant answers] Ask pupils to read and predict what will happen next Ask questions before, after, and during reading Ask pupils to summarize what they read Use visual representations of text Have children write a reaction to the text that they read Other (specify)_____________________________________________________________. I don’t teach this in my class 201 G. Pupil Performance and Evaluation [READ] I will now ask a few questions about pupil performance and assessment in your class. The purpose of these questions is to help us understand why some pupils struggle to become fluent readers in Ghana. We also want to know if and how teachers are using assessment to improve teaching and learning. 1. Do you believe your pupils are on track to become fluent readers? O Yes  G3 O No O Don’t know  G4 O Decline to answer  G4 2. [If no to G1 above] Why do you think your pupils are not on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers] They don’t come to class often enough They don’t come to class on time They don’t pay attention during class Their parents do not support their reading at home They are too hungry to concentrate They don’t care about school They are not able to understand the language of instruction The class is too large for me to provide good instruction The lessons are too short The teacher(s) they had before me did not do a good job I don’t always teach the lessons because I am doing other things I don’t have the teaching resources I need to teach them well I don’t have the training I need to teach them well The pupils don’t have enough learning books The pupils don’t have access to books outside of school Other (specify)_____________________________________________________________. 3. How do you know whether they are on track to become fluent readers? (select all that apply) [Do not prompt, record any relevant answers] I assess them regularly They seem to be doing OK when I observe them in class I don’t know Other (specify)_____________________________________________________________. 4. For each of the following methods of pupil evaluation or assessment, please state whether you employ it daily, weekly, monthly, quarterly, annually, or not at all: [Enumerator: Please name each method and ask for an answer for that method before moving on. You must read out all of the methods listed here] Method Daily Weekly Monthly Quarterly Annually Not at all Written assessments O O O O O O 202 Individual (1:1) oral assessments O O O O O O Checking pupil workbooks O O O O O O Checking pupil homework O O O O O O Other (specify): _____________. O O O O O O 5. Unlike a teacher-designed assessment, a standardized assessment is a type of assessment which uses the same questions across schools and has a standard scoring system. Please describe any STANDARDIZED oral or written assessments that you are currently using to assess pupils: 6. [If ASER is mentioned in G5] Do you keep a log book of ASER results? O Yes O No O Decline to answer 7. Do you adjust your teaching based on the results of individual, one-on-one pupil evaluation or assessment? O Yes O No  skip to G9 O I don’t do individual, one-on-one assessment  skip to G9 O Don’t know  skip to G9 O Decline to answer  skip to G9 8. [If yes to G7] In what ways do you adjust your teaching based on the results of pupil evaluation or assessment? (select all that apply) [Read each option and record relevant answers] I repeat previous lessons I make new lessons to teach difficult content in a different way I pay more attention to struggling learners in class I pay more attention to high performing learners in class I provide struggling learners with tutoring outside of class I arrange for others to provide struggling learners with tutoring outside of class I pair struggling learners with stronger ones during class If learners are doing well, I skip lessons that aren’t useful If learners are doing well, I go through lessons more quickly Other (specify)_____________________________________________________________. 9. Do you penalize or punish pupils who score low or unsatisfactory marks on the assessment? O Yes O No  skip to G11 O Don’t know  skip to G11 O Decline to answer  skip to G11 203 10. [If yes to G9] In what ways do you penalize or punish pupils based on the results of pupil assessment? 11. Do you acknowledge or reward pupils who score high marks on the assessment? O Yes O No  skip to G13 O Don’t know  skip to G13 O Decline to answer  skip to G13 12. [If yes to G11] In what ways do you acknowledge or reward pupils based on the results of pupil assessment? [Do not prompt, record any relevant answers] Food and drink (candies, biscuits, soda, etc.) Gifts Stickers or stars Words of encouragement Giving them a special job Other (specify): _____________________________________________________________. 13. Do you track your pupils’ progress over time (e.g., termly)? O Yes O No O Don’t know O Decline to answer End Time (HH:MM): : Thank you! That completes the teacher interview. 204 Ghana Early Grade Reading Program Impact Evaluation Midline Pupil Questionnaire General Background Information Team name: Enumerator ID: Today’s date (DD-MM-YY): - - Start time (HH:MM): : School name: School ID: Region: District: Locality: Pupil ID from tracking sheet: [Read assent script. Does the child assent to participate?] O Yes O No (DO NOT PROCEED) Pupil first name: Pupil sex: O Male O Female 205 A. Pupil Background Information I will start by asking some questions about you as well as how often you come to school. If you don’t know the answer to any questions or don’t want to answer, that’s OK. None of your answers will be shared with anyone so please answer truthfully. 1. How old are you?  Age in years: 2. In what class were you last year? O KG1  skip to A4 O KG2  skip to A4 O P1 O P2 O P3 O I was not in school last year O Don’t know O Decline to answer 3. Did you go to KG before starting P1?  O Yes O No O Don’t know O Decline to answer  4. Did you go to nursery or pre-school before starting KG or P1? O Yes O No O Don’t know O Decline to answer 5. Think back to last week, Monday through Friday. Were you absent from school on any days last week? O Yes O No  skip to A7 O Don’t know  skip to A7 O Decline to answer  skip to A7 6. [If yes to A5] How many days last week were you absent from school?  7. How often are you absent from school? O I rarely or never miss school  skip to A9 O I sometimes miss school, but I come more than half of the time O I regularly miss school—I am absent at least half of the time O I rarely come to school O Don’t know O Decline to answer 206 8. What are the main reasons that you miss school? (select all that apply) [Do not prompt, record any relevant answers]  I am sick or hurt I am too tired to come It is too far to walk No transportation or money for transportation No money to pay school fees No money to buy food I have to help with household chores I have to care for family members (younger siblings, sick parents/grandparents, etc.) I have to do work for the family (including farm work) I want to play instead The weather is bad Religious services or observances Dirty uniform or no uniform Travel No shoes Other (specify):_____________________________________________________________. 9. How often are you late to school? [Do not prompt, record relevant answer] O I am rarely or never late to school  skip to A11 O I am sometimes late, but I am on time more than half the time O I am regularly late to school—at least half of the time O I rarely come to school on time O Don’t know  skip to A11 O Decline to answer  skip to A11 10. What are the main reasons you have been late to school? (select all that apply) [Do not prompt, record any relevant answers]  I wake up late I have to walk a long time I have to help with household chores I have to babysit younger siblings I have to do work for the family (including farm work) I am delayed in getting ready Other people delay me (siblings, parents, etc.) I want to play instead The weather is bad The classes never start on time because all the other pupils are late The classes never start on time because the teachers are late Other (specify):_____________________________________________________________. 11. How often do you feel hungry when you first get to school in the morning: every day, some days, or rarely/never? O Every day O Some days 207 O Rarely or never O Don’t know O Decline to answer 12. Do you usually eat breakfast at home in the morning? O Yes  skip to A14 O No O Don’t know  skip to B1 O Decline to answer  skip to B1 13. [If no to A12] What is the main reason that you usually do not eat breakfast in the morning? O There is no food available at home  skip to B1 O I don’t have time to eat breakfast  skip to B1 O I don’t feel hungry  skip to B1 O I am not given money for food  skip to B1 O Don’t know  skip to B1 O Decline to answer  skip to B1 O Other (specify): __________________________________________________________. 14. [If yes to A12] What do you usually eat for breakfast in the morning (select all that apply) [Do not prompt, record any relevant answers] Ampesi Banku / kenkey / akple Eggs Fruit Fufu Koko Porridge (koko) Rice or rice water Rice and stew/soup Tea and bread/biscuit Tuo-zaafi (TZ) Waakaye/beans Other (specify):____________. B. Language Background I will now ask some questions about the languages you speak at home and at school. I will also ask about the language your teachers use during class. 1. In which language(s) do your parents speak to you at home most of the time? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer 208 English Ewe Likpakpaln Mampruli Don’t know Other (specify): _____________________.  2. Which language(s) do you use when playing with your friends on the playground at school? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 3. Which language(s) are used in teaching you at school? (select all that apply) [Do not prompt, record any relevant answers]  Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 4. Which language is used most often in teaching you reading and writing? (select one)  O Ahanta O Akuapem Twi O Asante Twi O Bono O Cherepon O Dagaare O Dagbani O Dangme O English O Ewe O Fante O Ga O Hausa O Gonja O Gua O Gurene O Kasem O Kusaal O Likpakpaln O Mampruli O Mol O Nabt O Sissale O Sefwi O Tale O Nzema O Waale O Decline to answer O Don’t know O Other (specify): _____________________. 5. Do you speak that language? Can you read that language?  209 O I can only speak it O I can only read it O I can speak and read it O I can neither speak nor read it O Don’t know O Decline to answer 6. If you could choose, which language would you want your teacher to use when teaching you reading and writing? (select one)  O Ahanta O Akuapem Twi O Asante Twi O Bono O Cherepon O Dagaare O Dagbani O Dangme O English O Ewe O Fante O Ga O Hausa O Gonja O Gua O Gurene O Kasem O Kusaal O Likpakpaln O Mampruli O Mol O Nabt O Sissale O Sefwi O Tale O Nzema O Waale O Decline to answer O Don’t know O Other (specify): _____________________. C. Reading Practices I am now going to ask a few questions about your reading practices in school and at home. 1. Did you read books on your own yesterday (or the most recent school day)?  O Yes O No  skip to C4 O Don’t know  skip to C4 O Decline to answer  skip to C4 2. [If yes to C1] Where did you get the book(s) that you read yesterday? (select all that apply) [Do not prompt, record any relevant answers]  School library Classroom library Belongs to me or my family Belongs to a friend or neighbor Other (specify):_____________________________________________________________. 3. [If yes to C1] Where did you read the book(s): at home, at school, or both home and school?  O At home O At school O At both home and school O Don’t know 210 O Decline to answer O Other (specify):_____________________________________________________________. 4. How often does someone at home read books with you?  O Hardly ever O Only sometimes O 2-3 times per week O Every day O Don’t know O Decline to answer 5. Does anyone at home help you do your homework? O Yes O No O Don’t know O Decline to answer 6. Do you like reading or dislike reading?  O Like it  skip to D1 O Neutral  skip to D1 O Dislike it O Don’t know  skip to D1 O Decline to answer  skip to D1 7. Why don’t you like reading?   D. Reading Assessment Teachers can see how pupils are doing in many different ways. I will now ask you about some different ways that your reading teacher might check in to see how you are learning. For each of these ways, please tell me if your teacher does it every day, every week, every month, or rarely. 1. Does your teacher ever sit with you to test your reading one-on-one?  O Yes O No  skip to D3 O Don’t know  skip to D3 O Decline to answer  skip to D3 2. [If yes to D1] How often does your teacher sit with you to test your reading one-on￾one?  O Every day O Every week O Every month 211 O Rarely O Don’t know O Decline to answer 3. Do you have a pupil workbook? By workbook I mean a book that you can write in, that has reading and writing activities.  O Yes O No  skip to D7 O Don’t know  skip to D7 O Decline to answer  skip to D7 4. [If yes to D3] In what language or languages is your pupil workbook? (select all that apply) [Do not prompt, record any relevant answers]  Akuapem Twi Asante Twi Dagaare Dagbani Dangme English Ewe Fante Ga Gonja Gurene Kasem Kusaal Nzema Decline to answer Don’t know Other (specify): ________________________. 5. [If yes to D3] Does your teacher check or mark your workbook?  O Yes O No  skip to D7 O Don’t know  skip to D7 O Decline to answer  skip to D7 6. [If yes to D5] How often does your teacher check or mark your workbook?  O Every day O Every week O Every month O Rarely O Don’t know O Decline to answer 7. Does your teacher mark your homework?  O Yes O No  skip to D9 O Don’t know  skip to D9 O Decline to answer  skip to D9 8. [If yes to D7] How often does your teacher mark your homework?  212 O Every day O Every week O Every month O Rarely O Don’t know O Decline to answer 9. Does your reading teacher punish you if you perform poorly? O Yes O No  skip to D11 O Don’t know  skip to D11 O Decline to answer  skip to D11 10. [If yes to question D9] In what ways does your teacher punish you? (select all that apply) [Do not prompt, record any relevant answers]  Beats, canes, pinches me or other physical punishment Has other pupils physically punish me Makes me sweep, clean, or pick rubbish Makes me hold an uncomfortable position (kneeling/squatting/hands over head) Insults, hoots, or laughs at me (or tells my peers to do so) Other (specify):_____________________________________________________________. 11. Does your reading teacher reward you if you perform well? O Yes O No  skip to E1 O Don’t know  skip to E1 O Decline to answer  skip to E1 12. [If yes to question D11] In what ways does your teacher reward you? (select all that apply) [Do not prompt, record any relevant answers] Food and drink (candies, biscuits, soda, etc.) Gifts Clapping Encourages me with words Gives me a special task Gives me money Gives me school supplies (pencils, books, etc.) Other (specify):_____________________________________________________________. E. Household Assets  I will now ask some questions about the things you have in your house. [Enumerators: use stimulus sheet. If child reports multiple items, you should probe to find out which one is used MOST often] 1. Where do you normally get drinking water from at home?  213 O River or stream O Well or borehole O Communal tap O Tap in the home O Water tank O Rain water O Bottled or sachet water O Tap water bought outside the home O Other (specify): _______________________________. O Don’t know O Decline to answer 2. Does your home have electricity?  O Yes O No O Don’t know O Decline to answer 3. Where is food normally cooked at your home?  O Outside the house O In a shed O Inside the house O Other (specify): _______________________________. O Don’t know O Decline to answer 4. How is food most often cooked at your home? O Using firewood O Using a coal pot O Using a stove O Using a cooker (including an oven) O Other (specify): _______________________________. O Don’t know O Decline to answer 5. When you are at home, what type of toilet do you use? O A pit toilet O A shared toilet O A communal toilet O A flush toilet outside your house O A flush toilet inside your house O In the bush/free range O Other (specify): _______________________________. O Don’t know O Decline to answer 214 6. Does your family have the following items in your home? (select all that apply) [Read options one by one and ask child for a yes/no answer]  Item Yes No Don’t know Radio O O O Mobile phone O O O Television O O O Computer O O O Refrigerator O O O Bicycle O O O Motorbike O O O Car/truck O O O Thank you! That completes the pupil questionnaire. End Time (HH:MM): : 215 Ghana Early Grade Reading Program Impact Evaluation Midline Fidelity of Implementation Classroom Observation Begin the classroom observation by asking the teacher if he/she can show you all of the lesson books, folders, and/or notes that he/she will be using for today’s reading lesson. If the teacher does not show a copy of a Learning scripted lesson plan, double confirm that he/she will not use any additional books or materials other than what he/she has shown you. If the teacher will not use the Learning scripted lesson plan, you will not complete this observation and instead focus on the Structured Observation only. DO NOT prompt the teacher to use the Learning scripted lesson plan in any way (e.g., by asking about it, revealing your copy, etc.). Only once he/she has shown you the Learning lesson plan, should you continue. Team name: Enumerator ID: Today’s date (DD-MM-YY): - - School name: School ID: Teacher ID: Lesson START time (HH:MM): : 1. Which grades are included in the class? (select all that apply) KG1 KG2 P1 P2 P3 P4 P5 P6 IMPORTANT! Before beginning, it is critical that you verify which term book and language he/she will be using, as well as all lesson number(s) and part(s) that will be covered during the observation. If you record (and therefore follow) the wrong lesson number/part, you will not be able to complete the observation correctly. We thus will be verifying that the term, lesson number, part, number of activities, and number of minutes for each activity corresponds to the official lesson plans. Misaligned or mis￾entered data will be immediately reported to ILC Africa management.  2. What is the language of the scripted lesson plan the teacher is using today? O Akuapem Twi O Asante Twi O Dagaare O Dagbani O Dangme O Ewe O Fante O Ga O Gonja O Kasem 3. What is the term (i.e., T1, T2, or T3) of the scripted lesson plan the teacher is using today? T1 T2 T3 4. Which lesson number(s) and part(s) is the teacher planning on using today? Lesson # Part (A or B) 216 Start the observation when the teacher begins the first activity. Record activities according to the order in which the teacher teachers them. For each activity, you should complete sections A, C, and D prior to completing sections E, F and G (section B can be completed after the lesson is completed if needed). Once the lesson is completed, record the lesson end time at the end of the next page. Note that “Some” is less than half (but more than zero) while “Most” is more than half (but less than 100%). Lesson # and part (e.g., 113B) A. Exact start time of activity (HH:MM) B. Which activity is the teacher doing (activity name)? C. Activity number D. Exact end time of activity (HH:MM) E. Did the teacher follow the steps in the lesson plan? (circle one) F. How many pupils were participating in the activity? (circle one) G. Did the teacher use the appropriate materials? (circle one) : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No 217 : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No : : None Some Most All None Some Most All Yes No 218 After the lesson, ask the pupils to stay in their seats so you can count how many have pupil books, pencils, and notebooks. 5. Ask pupils to hold up their Learning pupil books in the air. How many pupils have the pupil book for this term? 6. Ask the pupils to hold their pen/pencil in the air. How many pupils have a pen or pencil? 7. Ask the pupils to hold their notebooks in the air. How many pupils have a notebook to write in? 8. Randomly select one pupil’s notebook. Open the book and look at some of the pages. Has the pupil practiced letter writing in the notebook? O Yes O No O No pupils in this class have a notebook 9. Did the teacher have the Teacher’s Guide? O Yes O No 10. Did the teacher have their own copy of the pupil book? O Yes O No 11. Did the teacher have an alphabet chart posted (or review letters posted in large print) in the Official Ghanaian Language of Instruction? O Yes O No  skip to 14 12. Is the alphabet chart posted where children can see it? O Yes O No 13. Did the teacher have flash cards? O Yes O No  skip to 16 14. Do the flash cards have a box to store them in? O Yes O No 15. Did the teacher have Big Books? O Yes O No 16. Did the teacher have book of conversation/picture posters? 219 O Yes O No Lesson END time (HH:MM): : 220 Ghana Early Grade Reading Program Impact Evaluation Midline Structured Classroom Observation General Background Information Team name: Enumerator ID: Today’s date (DD-MM-YY): - - Lesson start time (HH:MM): : School name: School ID: Region: District: Locality: Teacher ID from tracking sheet: Was the teacher consent form administered and signed? O Yes O No (DO NOT PROCEED) Teacher Identifying Information First name: Last name: Sex: O Male O Female 221 The purpose of this tool is to record the specific reading practices that the teacher engages in during the classroom observation. Unlike Stallings or FoI tool, this observation instrument should reflect the entire duration of the lesson and be completed after the lesson is finished. For each “practice,” please record whether you: 1 – do not see the practice or see the opposite of the practice described, 2 - see the practice sometimes or partially correct, 3 - see the practice done very well and consistently where appropriate, or 4 - not applicable (practice is not relevant to the content of the lesson). TEACHER PRACTICE OBSERVED 1 The practice is not done/observed at all or the opposite is done 2 The practice is done sometimes or partially 3 The practice is done very well and consistently where appropriate 4 Not Applicable (practice is not relevant to the subject being taught) 1. Uses a lesson plan or lesson notes 2. Uses a scripted/step-by-step lesson plan 3. Introduces lesson by connecting to what learners have learned previously 4. Introduces lesson with advance organizer (e.g., visual) 5. Actively minimizes classroom time that is off-task 6. Demonstrates effective classroom management (e.g., efficiently manages materials, transition(s) between activities, class start and finish) 7. Uses different resources (e.g., books, tools, manipulatives) and strategies (e.g., audio, visual, kinesthetic) to explain concepts within the same lesson or class period 8. Constructively engages all pupils—not just some—in classroom activities 9. Engages learners in carefully planned cooperative learning strategies (e.g., pupils leading activities, working in groups) 10. Asks probing, open-ended questions that encourage thinking and/or help learners explicate their thinking 11. Provides learners with structured opportunities to apply their understanding and skills to everyday life and problems 12. Provides opportunities for learners to develop higher-order and critical thinking skills (e.g., analyze, evaluate, challenge assumptions, think about things in a new way) 13. Uses teaching and learning materials that are in a language that the pupils appear to understand TEACHER PRACTICE OBSERVED 1 The practice is not done/observed at all or the opposite is done 2 The practice is done sometimes or partially 3 The practice is done very well and consistently where appropriate 4 Not Applicable (practice is not relevant to the subject being taught) 14. Uses relevant teaching and learning materials that are appropriate to the subject, ability level, and culture of pupils 222 15. Assesses pupil learning (checks for understanding) in different ways within a lesson or class period. BIAS or MISTREATMENT 16. Avoids using language that favors one gender over another and/or reinforces gender stereotypes 17. Engages learners of all ability levels 18. Avoids using abusive language 19. Provides constructive, encouraging feedback 20. Does not allow learners to engage in gender bias 21. Does not allow learners to use abusive language 22. Girls have equal access to desks, learning materials, etc. 23. Calls on pupils equitably (girls/boys, front/back, different background) READING PRACTICE 24. Engages learners in reading activities or games 25. Encourages learners to “sound it out” when they don’t know a word 26. Avoids criticizing learners who don’t answer correctly or read poorly 27. Prior to reading a story, teacher asks learners pre-reading questions such as “What do you think the story will be about based on the pictures and/or title of the book?” 28. While teacher or learners read a story, teacher asks learners to make appropriate sounds or act something out, such as the roar a lion makes or the way a frog hops TEACHER PRACTICE OBSERVED 1 The practice is not done/observed at all or the opposite is done 2 The practice is done sometimes or partially 3 The practice is done very well and consistently where appropriate 4 Not Applicable (practice is not relevant to the subject being taught) 29. Applies multiple methods to support comprehension, including games, group work, etc. 223 30. Encourages learners to help each other 31. Has individual learners read aloud (to the class, to a small group, and/or in pairs) 32. Provides instructions on how to blend multiple letter sounds and/or decode syllables and words 33. Teaches learners meanings of new words by explaining the meaning 34. Teaches learners meanings of new words by asking pupils to infer meaning on their own through using context and pictures. 35. Pupils clap the syllables of words 36. Asks learners questions to assess their understanding of text that they read 37. Asks learners questions to assess their understanding of stories they hear 38. Asks learners to recognize letters and say letter names and/or sounds 39. Learners retell a story they or the teacher read 40. Asks learners to recite the alphabet 41. Assigns reading activities for learners to do on their own during school time 42. Provides a variety of methods for learners to establish good writing skills 43. Effectively accommodates multiple languages spoken by pupils (by speaking in multiple languages, or having other pupils or assistants translate, etc.) 1 The practice is not done/observed at all or the opposite is done 2 The practice is done sometimes or partially 3 The practice is done very well and consistently where appropriate 4 Not Applicable (practice is not relevant to the subject being taught) PUPIL BEHAVIOR 44. At least 90 percent of learners are paying attention 45. At least 90 percent of learners are actively engaged in the lesson 224 46. At least 90 percent of learners are actively engaged when working in small groups or pairs 47. Learners appear to understand what the teacher is saying 48. Which language(s) did the teacher use during the lesson? (select all that apply) Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 49. For each language used, indicate approximately what percentage/proportion of instructional time that language was used by the teacher? ____ Ahanta ____ Akuapem Twi ____ Asante Twi ____ Bono ____ Cherepon ____ Dagaare ____ Dagbani ____ Dangme ____ English ____ Ewe ____ Fante ____ Ga ____ Hausa ____ Gonja ____ Gua ____ Gurene ____ Kasem ____ Kusaal ____ Likpakpaln ____ Mampruli ____ Mol ____ Nabt ____ Sissale ____ Sefwi ____ Tale ____ Nzema ____ Waale ____ Decline to answer ____ Don’t know ____ Other (specify): _____________________. 225 50. For each language used, indicate how well the teacher spoke that language: Language Fluently or near fluently Good Somewhat well Not very well Poorly I am not able to judge Ahanta O O O O O O Akuapem Twi O O O O O O Asante Twi O O O O O O Bono O O O O O O Cherepon O O O O O O Dagaare O O O O O O Dagbani O O O O O O Dangme O O O O O O English O O O O O O Ewe O O O O O O Fante O O O O O O Ga O O O O O O Hausa O O O O O O Gonja O O O O O O Gua O O O O O O Gurene O O O O O O Kasem O O O O O O Kasem O O O O O O Likpakpaln O O O O O O Mampruli O O O O O O Mol O O O O O O Nabt O O O O O O Sissale O O O O O O Sefwi O O O O O O Tale O O O O O O Nzema O O O O O O Waale O O O O O O Other (specify): O O O O O O 51. Which language(s) did the pupils use during the lesson? (select all that apply) Ahanta Akuapem Twi Asante Twi Bono Cherepon Dagaare Dagbani Dangme English Ewe Fante Ga Hausa Gonja Gua Gurene Kasem Kusaal Likpakpaln Mampruli Mol Nabt Sissale Sefwi Tale Nzema Waale Decline to answer Don’t know Other (specify): _____________________. 52. At any time during the lesson, did the pupils use pupil workbooks? O Yes O No  skip to 55 226 53. [If yes to 52] What was the primary language of the pupil workbooks? O Akuapim Twi O Asante Twi O Dagaare O Dagbani O Dangme O English O Ewe O Fante O Ga O Gonja O Kasem O Decline to answer O Not applicable O Other (specify): ________________________. 54. [If yes to 52] Were there enough workbooks for each pupil to have his/her own book? O Yes O No 55. Is the classroom clean and neat? O Yes O No O The class was held outside of a classroom environment 56. Is there sufficient work space at a desk or table for all the pupils? O Yes O No 57. Is there enough space in the class for the teacher to move about freely? O Yes O No Lesson end time (HH:MM): : 227 Ghana Early Grade Reading Program Impact Evaluation Midline Stallings Classroom Observation 228 Ghana Early Grade Reading Program Impact Evaluation Midline Early Grade Reading Assessment (EGRA) Sub-test 1. ORAL VOCABULARY  Page 1    Eyinom yɛ nneɛma ahorow bi mfonini. Mepa wo kyɛw, bɔ ade biara din wɔ Borɔfo kasa mu. [point to the bird] Sɛ ebia: Eyi yɛ “bird” mfonini. Sɛ wuhu saa mfonini yi a, ka asɛmfua “bird”. Here are pictures of some objects. Please tell me the name of each object in English. For example, [point to the bird], this is a picture of a bird. When you see this picture, you say the word “bird.” Ma me ne wo nyɛ bi nhwɛ: Bɔ saa ade yi din wɔ Borɔfo kasa mu. [point to the dog] Let’s practice. Tell me the name of this object [point to the dog].  Mo. Eyi yɛ “dog”. Good. That is a dog.  [point to the dog] Eyi yɛ “dog”. Bɔ saa ade yi din wɔ Borɔfo kasa mu. This is a dog. Tell me the name of this object in English. Sɛ wudu mfonini a wunnim ne din so a, kɔ nea edi so no so. Fa wo nsateaa si mfonini a edi kan no so. If you come to a picture that you do not know, go on to the next picture. Put your finger on the first picture. Woayɛ krado? Ma yemfi ase. Ready? Let’s Begin. book frog plane duck trousers monkey banana flower  If a child hesitates for 3 SECONDS, go on to the next word  If the child does not single correct response on the first line (4 say “Thank discontinue this subtask, check the box at the bottom, and go to the next subtask. If the child does not respond in English, stop child and ask him/her to respond in – do this only once, and after that mark any non-English responses as incorrect.  Exercise discontinued because the child had no correct answers in the first Mo! Woayɛ ade. Yɛnkɔ ɔfa a edi so no so. Good effort! Let’s go on to the next section. 229 Sub-test 2. LETTER SOUND IDENTIFICATION  Page 2  60 seconds  Wɔakyerɛw Borɔfo kasa nkyerɛwde wɔ saa kratafa yi so. Mepa wo kyɛw, ka nkyerɛwde yi nnyigyei dodow biara a wubetumi wɔ Borɔfo kasa mu. Ɛnyɛ wɔn din na mmom wɔn nnyigyei. Here is a page full of letters of the ENGLISH alphabet. Please tell me the SOUNDS of as many letters of the alphabet as you can. Not theirnames, but their sounds. [point to the letter T] Nhwɛso. Akyerɛwde yi mu nnyigyei ne /t/. For example, the sound of this letter is /t/. [point to the letter M] Afei yenni dwuma no. Ka nnyigyei a wote fi saa akyerɛwde yi mu. Let’s practice: Tell me the sound of this letter.  Eye, woatwa, ɛyɛ [m] nnyigyei Good, the sound of this letter is/m/.  Nnyigyei no ne /m/. The sound of this letter is/m/. [point to the letter S] Bɔ foforo bio. Ka akyerɛwde yi nnyigyei kyerɛ me. Now let us try another one. Tell me the sound of this letter.  Eye, woatwa, nnyigyei no yɛ /s/. Good, the sound of this letter is/s/.  Nnyigyei a efi akyerɛwde yi mu no ne /s/. The sound ofthisletteris/s/. [pointtofirstletter]Minya ka se‘FiAse’ a,fi asewɔ ha nawie krataa no so de nyinaa. Fa wo nsa si akyerɛwde no so na bɔ nne nnyigyei no dennennen. Wudu akyerɛwde bi so na wunnim ne nnyigyei no a, kɔ akyerɛwde foforo so. Fa wo nsateaa si akyerɛwde a edi kan no so Metumi afi ase? Hyɛ ase. When I say “Begin,” start here and go across the page. Point to each letter and tell me the sound of that letter in a loud voice. Read as quickly and carefully as you can. If you come to a letter you do not know, go on to the next letter. Put your finger on the first letter. Ready? Begin. Start the timer when the child reads the first letter.  If a child hesitates or stops on a letter for 3 SECONDS, point to the next letter and say “Go on” If the child does not respond in English, stop the child and ask him/her to respond in English – do this only once, and after that mark any non-English responses as incorrect.  When the timer reaches 0, say “stop.”  If the child does not provide a single correct response on the first line (10 items), say “Thank you!”, discontinue this subtask, check the box at the bottom, and go on to the next subtask.  ( / ) Mark any incorrect letters with a slash ( Ø ) Circle self-corrections if you already marked the letter incorrect ( ] ) Mark the final letter read with a bracket Examples: t m s 1 2 3 4 5 6 7 8 9 10 e E N T A i E t o r (10) s S L n R o S o s L (20) a A m D t M A e R C (30) I R p w N y E s i U (40) t i k b l g S R E e (50) e n O Z c n a i S s (60) M l d E u d r N A a (70) P W f s J Y T D r I (80) K B h a p G V F T M (90) 230 T X r I Q I f H O E (100)  Time remaining on stopwatch at completion (number of SECONDS)  Exercise discontinued because the child had no correct answers in the firstline 231 Sub-test 3. NON-WORD READING  Page 3  60 seconds  Eyinom yɛ nsɛmfua bi a wɔahyehyɛ no Borɔfo kasa mu. Mepɛ sɛ wokenkan dodow biara a wubetumi. Ɛnsopɛle na mmom kenkan. Here are some made-up words in ENGLISH. I would like you to read as many as you can. Do not spell the words, but read them. [point to the word “ut”] Sɛ ebia saa asɛmfua yi yɛ ‘ut’. For example, this made-up word is: “ut”. [point to the word “dif”] Yɛnsɔ bi nhwɛ: Mesrɛ wo kan asɛmfua yi. Let’s practice: Please read this word.  Mo, Asɛmfua yi yɛ ‘dif’. Good, This made-up word is “dif.”  Saa asɛmfua yi yɛ ‘dif’. This made-up word is “dif.” [point to the word “mab”] Afei san kenkan foforo bio: mesrɛ wo kenkan asɛmfua yi. Now let us try another one. Please read this word.  Mo, asɛmfua yi yɛ “mab.” Good, This made-up word is “mab.”  Asɛmfua yi yɛ “mab.” This made-up word is “mab.” [point to first word] Meka sɛ “Fi Ase” a, fi ase wɔ ha na kenkan kɔ saa ara. Fa wo nsateaa si asɛmfua biara so na ka no den. Bɔ mmɔden kenkan no yiye ne ntɛmntɛm. Wudu asɛmfua bi so na wunnim ne nnyigyei no a, kɔ asɛmfua foforo so. Fa wo nsateaa si asɛmfua a edi kan no so. Metumi afi ase? Hyɛ ase. When I say “Begin,” start here [point to first word] and read across the page [point]. Point to each word and read it in a loud voice. Read as quickly and carefully as you can. If you come to a word you do not know, go on to the next word. Put your finger on the first word. Ready? Begin. Start the timer when the child reads the first word.  If a child hesitates or stops on a letter for 3 SECONDS, point to the next word and say “Go on” If the child does not respond in English, stop the child and ask him/her to respond in English – do this only once, and after that mark any non-English responses as incorrect.  When the timer reaches 0, say “stop.”  If the child does not provide a single correct response on the first line (5 items), say “Thank you!”, discontinue this subtask, check the box at the bottom, and go on to the next subtask.  ( / ) Mark any incorrect words with a slash ( Ø ) Circle self-corrections if you already marked the word incorrect ( ] ) Mark the final word read with a bracket Examples: ut dif mab 1 2 3 4 5 fut lus dit leb gak (5) huz jod kib lek tob (10) nom rop hig reg san (15) tup ral wix nep nad (20) lut yod sim tat sig (25) en mon nup sen kad (30) taw lew paf sal zuv (35) ved kag vom riz gof (40) maz kol ver et beb (45) tib lef yag lim dov (50) 232  Time remaining on stopwatch at completion (number of SECONDS)  Exercise discontinued because the child had no correct answers in the firstline 233 EGRA (English) Assessment (LANGUAGE) Sub-test 4a. ORAL READING PASSAGE  60 seconds Sub-test 4b: READING COMPREHENSION  Page 4 Show the child the sheet in the pupil stimulus booklet as you read the instructions.  If a child hesitates or stops on a letter for 3 SECONDS, say “Go on”  If the child does not provide a single correct word before the word in a box, say “Thank you!”, discontinue this subtask and check the box at the bottom. Do not ask any comprehension questions. If a child says “I don’t know,” mark as incorrect.  X After the child is finished reading, REMOVE the passage from in front of the child. Ask the child only the questions related to the text read. A child must read all the text that corresponds with a given question. If the child does not provide a response to a question after 10 seconds, mark “no response” and continue to the next question. Do not repeat the question.  Abasɛm tiawa bi ni. Mepɛ sɛ wokenkan no dennennen, ne ntɛmntɛm ma me. Wokenkan wie a, mebisabisa nsɛm bi afa neawoakenkannoho.Mekasɛ“FiAse”akenkanabasɛmno sɛnea wubetumi biara. Wudu asɛmfua bi so na wunnim a, kɔ asɛmfua foforo so. Fa wo nsateaa si asɛmfua a edi kan no so. Metumi afi ase? Hyɛ ase. Here is a short story. I want you to read it aloud, quickly but carefully. When you finish, I will ask you some questions about what you have read. When I say “Begin,” read the story as best as you can. If you come to a word you do not know, go on to the next word. Put your finger on the first word. Ready? Begin.  Afei mebisabisa wo nsɛm kakra afa abasɛm a wokenkan no ho. Bɔ mmɔden sɛwubeyiyi nsɛmmisa no ano sɛneawubetumi.Wubetumi de kasa biara a wopɛ ayiyi nsɛmmisa no ano. Now I am going to ask you a few questions about the story you just read. Try to answer the questions as well as you can. You can provide your answers in whichever language you prefer.  ( / ) Mark any incorrect letters with a slash ( Ø ) Circle self-corrections if you already marked the letter incorrect ( ] ) Mark the final letter read with a bracket  () 1 = Correct () 0 = Incorrect () . = No response. Questions [Answers] Mother is sick. There is no doctor in the village to helpher. 13 Where is there no doctor? [in the village] 1 0 . Kofi says that when he grows up he will be a doctor. 25 What does Kofi want to be when he grows up? [a doctor] 1 0 . He will help people who are sick like mother. 35 Why does Kofi want to be a doctor? [to help people / to help people who are sick] 1 0 . Fati also wants to be a doctor. She will teach mothers and children to be healthy. 46 What will Fati teach mothers and children? [to be healthy] 1 0 . Mother smiles. She is happy with both of her children. 56 Why is mother happy with her children? [they want to be doctors / they want to help people ] 1 0 .  Time remaining on stopwatch at completion (number of SECONDS)  Exercise discontinued: the child had no correct answers in the first line 234 Mo! Woayɛ ade. Yɛnkɔ ɔfa a edi so no so. Good effort! Let’s go on to the next section. 235 EGRA (English) Assessment (LANGUAGE) Mo! Woayɛ ade. Yɛnkɔ ɔfa a edi so no so. Good effort! Let’s go on to the next section. Sub-test 5. LISTENING COMPREHENSION  X  X  Merebɛkenkan abasɛm tiawa bi baako pɛ dennennen akyerɛ wo, na mabisabisa wo nsɛm kakra afa ho. Mesrɛ wo tie no yiye na bua nsɛmmisa no sɛnea wubetumi biara. Wubetumi de kasa biara a wopɛ ayiyi nsɛmmisa no ano. Metumi afi ase? Yemfi ase. I am going to read you a short story aloud ONCE and then ask you some questions. Please listen carefully and answer the questions as best as you can. You can answer the questions in whichever language you prefer. Ready? Let’s begin. Remove the pupil stimuli booklet from the child’s view. Do not allow the child to look at the passage or the questions. If a child says “I don’t know,” mark as incorrect.  () 1 = Correct () 0 = Incorrect () . = No response. Grandfather was very sad. He lost his sheep. He could not go to look for them.Issa wentto look forthem. Soon he returned with the sheep.Grandfatheris smiling now. Why was Grandfather sad? [he lost his sheep; he could not go to look for his sheep] 1 0 . Who went to look for the sheep? [Issa] 1 0 . Why is Grandfather smiling now? [Issa returned with his sheep; his sheep are back;Issa found the sheep] 1 0 . 236 237 238 239 240 Sub-test 1. ORAL VOCABULARY Page 1   Eyinom yɛ nneɛma ahorow bi mfonini.Mepa wo kyɛw, bɔ ade biara din wɔ Borɔfo kasa mu. [point to the bird] Sɛ ebia: Eyi yɛ “bird” mfonini. Sɛ wuhu saa mfonini yi a, ka asɛmfua “bird”.Here are pictures of some objects. Please tell me the name of each object in English. For example, [point to the bird], this is a picture of a bird. When you see this picture, you say the word “bird.” Mamenewonyɛbinhwɛ:BɔsaaadeyidinwɔBorɔfokasamu. [point tothedog] Let’s practice. Tell me the name of this object [point to the dog]. Mo.Eyi yɛ “dog”. Good. That is a dog. [pointto the dog]Eyi yɛ “dog”. Bɔ saa ade yi din wɔ Borɔfo kasa mu. Thisis a dog. Tell me the name of this object in English. Sɛwudumfoniniawunnimnedinsoa,kɔneaedisonoso.Fawonsateaasimfoninia edi kan no so.If you come to a picture that you do not know, go on to the next picture. Put your finger on the first picture. Woayɛ krado? Ma yemfi ase.Ready? Let’s Begin. book frog plane duck trousers monkey banana flower If a child hesitates for 3 SECONDS, go on to the next word. If the child does not provide a single correct response on the first line (4 items), say “Thank you!”, discontinue this subtask, check the box at the bottom, and go on to the next subtask. If the child does not respond in English, stop the child and ask him/her to respond in English – do this only once, and after that mark any non-English responses as incorrect.  Exercise discontinued because the child had no correct answers in the firstline Mo! Woayɛ ade. Yɛnkɔ ɔfa a edi so no so.Good effort! Let’s go on to the next section. 241 Sub-test2. LETTER SOUND IDENTIFICATION Page 2 60 seconds  Wɔakyerɛw Borɔfo kasa nkyerɛwde wɔ saa kratafa yi so. Mepa wo kyɛw, ka nkyerɛwdeyinnyigyeidodowbiaraawubetumiwɔBorɔfokasamu. Ɛnyɛwɔn din na mmomwɔn nnyigyei.Here is apage full oflettersofthe ENGLISHalphabet. Please tell me the SOUNDS of as many letters of the alphabet as you can. Not their names, but their sounds. [pointtotheletterT]Nhwɛso.Akyerɛwdeyimunnyigyeine/t/.Forexample,thesoundof this letter is /t/. [pointtotheletterM]Afeiyennidwumano.Kannyigyeiawotefisaaakyerɛwdeyi mu.Let’s practice: Tell me the sound of thisletter.  Eye,woatwa, ɛyɛ [m]nnyigyeiGood,thesoundofthisletteris/m/.  Nnyigyeinone/m/.Thesoundofthisletteris/m/. [pointtotheletter S]Bɔ foforobio.Kaakyerɛwdeyinnyigyeikyerɛme.Nowletustry another one. Tell me the sound of thisletter.  Eye,woatwa,nnyigyeinoyɛ/s/.Good,thesoundofthisletteris/s/.  Nnyigyeiaefiakyerɛwdeyimunone/s/.Thesoundofthisletteris/s/. [pointtofirstletter]Minya ka se ‘FiAse’a,fiase wɔhana wiekrataanosodenyinaa.Fa wonsasiakyerɛwdenosonabɔnnennyigyeinodennennen.Wuduakyerɛwdebiso na wunnim ne nnyigyei no a, kɔ akyerɛwde foforo so.Fa wo nsateaa si akyerɛwde a edi kan no soMetumi afi ase? Hyɛ ase.When Isay “Begin,” start here and go acrossthe page. Point to each letter and tell me the sound of that letter in a loud voice. Read as quickly and carefully as you can. If you come to a letter you do not know, go on to the next letter. Put your finger on the first letter. Ready? Begin. Start the timer when the child reads the first letter. If a child hesitates or stops on a letter for 3 SECONDS, point to the next letter and say “Go on” If the child does not respond in English, stop the child and ask him/her to respond in English – do this only once, and after that mark any non-English responses as incorrect. When the timer reaches 0, say “stop.” If the child does not provide a single correct response on the first line (10 items), say “Thank you!”, discontinue this subtask, check the box at the bottom, and go on to the next subtask.  ( / ) Mark any incorrect letters with a slash ( Ø ) Circle self-corrections if you already marked the letter incorrect ( ] ) Mark the final letter read with a bracket Examples: t m s 1 2 3 4 5 6 7 8 9 10 o T i A E N e E r t (10) o s L o S R L S n s (20) R t M a e D A c m A (30) I y i p R E s N U w (40) e k E t i R g l b S (50) S Z n e c s a i O n (60) N d r M A u E a l d (70) s D f r I J P T Y W (80) B V a F T h p G K M (90) 242 X E f Q O H I r I T (100)  Time remaining on stopwatch at completion (number of SECONDS)  Exercise discontinued because the child had no correct answers in the firstline 243 Sub-test3. NON-WORD READING Page 3 60 seconds  Eyinom yɛ nsɛmfua bi a wɔahyehyɛ no Borɔfo kasa mu. Mepɛ sɛ wokenkan dodow biara a wubetumi. Ɛnsopɛle na mmom kenkan.Here are some made-up words in ENGLISH. I would like you to read as many as you can. Do not spell the words,but read them. [point to the word “ut”]Sɛ ebia saa asɛmfua yi yɛ ‘ut’.For example, this made-up word is: “ut”. [pointto theword“dif”]Yɛnsɔ bi nhwɛ: Mesrɛ wo kan asɛmfua yi.Let’s practice: Please read this word.  Mo, Asɛmfua yi yɛ ‘dif’.Good, This made-up word is “dif.”  Saa asɛmfua yi yɛ ‘dif’.Thismade-upword is “dif.” [pointtotheword“mab”]Afeisankenkanfoforobio:mesrɛwokenkanasɛmfuayi.Now let us try another one. Please read this word.  Mo, asɛmfua yi yɛ “mab.”Good, Thismade-upword is “mab.”  Asɛmfua yi yɛ “mab.” Thismade-up word is “mab.” [point to first word] Meka sɛ “Fi Ase” a, fi ase wɔ ha na kenkan kɔ saa ara.Fa wo nsateaa si asɛmfua biara so na ka no den.Bɔ mmɔden kenkan no yiye ne ntɛmntɛm.Wudu asɛmfua bi so na wunnim ne nnyigyei no a, kɔ asɛmfua foforo so.Fa wo nsateaa si asɛmfua a edi kan no so.Metumi afi ase? Hyɛ ase.When I say “Begin,” start here [point to first word] and read across the page [point]. Point to each word and read it in a loud voice. Read as quickly and carefully as you can. If you come to a word you do not know, go on to the next word. Put your finger on the first word. Ready? Begin. Start the timer when the child reads the first word. If a child hesitates or stops on a letter for 3 SECONDS, point to the next word and say “Go on” If the child does not respond in English, stop the child and ask him/her to respond in English – do this only once, and after that mark any non-English responses as incorrect. When the timer reaches 0, say “stop.” If the child does not provide a single correct response on the first line (5 items), say “Thank you!”, discontinue this subtask, check the box at the bottom, and go on to the next subtask.  ( / ) Mark any incorrect words with a slash ( Ø ) Circle self-corrections if you already marked the word incorrect ( ] ) Mark the final word read with a bracket Examples: ut dif mab 1 2 3 4 5 dit fut lus leb gak (5) huz jod lek tob kib (10) reg san nom rop hig (15) tup nad wix ral nep (20) sim tat yod lut sig (25) nup sen en kad mon (30) taw lew zuv sal paf (35) gof vom riz ved kag (40) beb et maz ver kol (45) lim tib dov yag lef (50) 244  Time remaining on stopwatch at completion (number of SECONDS)  Exercise discontinued because the child had no correct answers in the firstline 245 EGRA (English) Assessment (LANGUAGE) Sub-test 4a. ORAL READING PASSAGE 60 seconds Sub-test 4b: READING COMPREHENSION Page 4 Show the child the sheet in the pupil stimulus booklet as you read the instructions.  If a child hesitates or stops on a letter for 3 SECONDS, say “Go on” If the child does not provide a single correct word before the word in a box, say “Thank you!”, discontinue this subtask and check the box at the bottom. Do not ask any comprehension questions. If a child says “I don’t know,” mark as incorrect. XAfter the child is finished reading, REMOVE the passage from in front of the child. Ask the child only the questions related to the text read. A child must read all the text that corresponds with a given question. If the child does not provide a response to a question after 10 seconds, mark “no response” and continue to the next question. Do not repeat the question. Abasɛm tiawa bi ni. Mepɛ sɛ wokenkan no dennennen, ne ntɛmntɛm ma me.Wokenkan wie a, mebisabisa nsɛm bi afa nea woakenkan no ho.Meka sɛ “Fi Ase” a kenkan abasɛm no sɛnea wubetumi biara.Wudu asɛmfua bi so na wunnim a, kɔ asɛmfua foforo so.Fa wo nsateaa si asɛmfua a edi kan no so.Metumi afi ase? Hyɛ ase.Here is a short story. I want you to read it aloud, quickly but carefully. When you finish, I will ask you some questions about what you have read. When I say “Begin,” read the story as best as you can. If you come to a word you do not know, go on to the next word. Put your finger on the first word. Ready? Begin. Afei mebisabisa wo nsɛm kakra afa abasɛm a wokenkan no ho.Bɔ mmɔdensɛwubeyiyinsɛmmisanoanosɛneawubetumi.Wubetumide kasabiara a wopɛayiyinsɛmmisanoano.NowIamgoing toask youa few questions about the story you just read. Try to answer the questions as well as you can. You can provide your answers in whichever language you prefer.  ( / ) Mark any incorrect letters with a slash ( Ø ) Circle self-corrections if you already marked the letter incorrect ( ] ) Mark the final letter read with a bracket  () 1 = Correct () 0 = Incorrect () . = No response. Questions [Answers] There is no doctor in the village where Ama lives. Father is sick. 13 Who is sick? [Father] 1 0 . Ama says that when she grows up she will be a doctor. 25 What does Ama want to be when she grows up? [a doctor] 1 0 . She will help people who are sick like father. 35 Why does Ama want to be a doctor? [to help people / to help people who are sick] 1 0 . Kojo wants to be a teacher. He will teach boys and girls to be healthy. 46 What will Kojo teach boys and girls? [to be healthy] 1 0 . Father smiles. He is happy with both of his children. 56 Why is father happy with his children? [they want to be doctors / they want to help people ] 1 0 .  Time remaining on stopwatch at completion (number of SECONDS) 246  Exercise discontinued: the child had no correct answers in the first line Mo! Woayɛ ade. Yɛnkɔ ɔfa a edi so no so.Good effort! Let’s go on to the next section. 247 EGRA (English) Assessment (LANGUAGE) Sub-test 5. LISTENING COMPREHENSION X  X Merebɛkenkanabasɛmtiawa bibaakopɛdennennenakyerɛ wo,na mabisabisa wo nsɛm kakra afa ho.Mesrɛ wo tie no yiye na bua nsɛmmisa no sɛnea wubetumi biara.Wubetumi de kasa biara a wopɛ ayiyi nsɛmmisa no ano.Metumi afi ase? Yemfi ase.I am going to read you a short story aloud ONCE and then ask you some questions. Please listen carefully and answer the questions as best as you can. You can answer the questions in whichever language you prefer. Ready? Let’s begin. Remove the pupil stimuli booklet from the child’s view. Do not allow the child to look at the passage or the questions. If a child says “I don’t know,” mark as incorrect.  () 1 = Correct () 0 = Incorrect () . = No response. Issa was very sad. He lost his grandfather’s sheep. He could not go to look for them. Grandfather came to look for them. Soon he returned with the sheep. Issa is smiling now. Why was Issa sad? [he lost his sheep; he could not go to look for his sheep] 1 0 . Who went to look for the sheep? [Grandfather] 1 0 . Why is Issa smiling now? [Grandfather returned with his sheep; his sheep are back; Grandfather found the sheep] 1 0 . Mo! Woayɛ ade. Yɛnkɔ ɔfa a edi so no so. Good effort! Let’s go on to the next section. 248 249 250 251 252 GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PROTOCOL CONSENT FORM – CIRCUIT SUPERVISOR Section A - BACKGROUND INFORMATION Title of Study: Ghana Early Grade Reading Impact Evaluation Principal Investigator: Erika Keaveney, Acting Program Director, Impact Evaluation, Social Impact Inc. Certified Protocol Number: RAMS/ERCP/SS/02/2018 Section B – CONSENT TO PARTICIPATE IN RESEARCH General Information About Research My name is _____________, and I am from Ivy League Consultants (ILC) Africa, an independent data collection firm working with the United States Agency for International Development (USAID)’s mission in Ghana and Social Impact, Inc., a research firm based in the Washington D.C. area of the United States of America. We are conducting a research study to assess the impact of an Early Grade Reading program which supports teachers and schools in Ghanaian language reading instruction across 100 districts of Ghana. The results of this study will be used by the Ghana Education Service and USAID to inform future programs aimed at helping children in Ghana become good readers. This school is among 470 schools across Ghana that has been selected to take part in this study through a process of statistical sampling. The school was visited in 2017 for a baseline assessment and we are now returning to conduct a midline assessment. As Circuit Supervisor of this school, we would like to verbally interview you one-on-one to get some information about this school that will be useful for this study, including the frequency of monitoring activities, your experiences with coaching teachers on reading instruction, teacher receptiveness to feedback, and practices you have observed in reading classes. We will also seek your general feedback on how implementation of the Early Grade Reading program is going at this school. This interview will last approximately 30 minutes. We will also interview you again at the end of the school year in 2019. Benefits/Risks of the Study There are no direct benefits to you for participating in the study, however, information collected in this study may benefit this and other schools in the future by improving early grade reading programming. Additionally, there are no known risks or discomforts associated with participating in this study. Confidentiality If you choose to participate, your responses will be strictly confidential. Your responses will be combined with those from other schools in the study and presented in the form of summary tables. Neither you nor your school will be individually identified or named in any reports. To keep the information you provide safe, each member of the research staff has signed a confidentiality agreement prior to conducting any data collection tasks. Any papers or electronic data with personal identifying information will be stored on password-protected electronic devices or in a locked room and no person outside of the research team will have access to this information. Upon conclusion of the study, all personal identifying information will be destroyed. 253 Compensation There is no compensation provided for participating in this study. Withdrawal from Study Your participation in this study is voluntary and you may withdraw from the study at any time without penalty. Regardless of your decision to participate in the research or not, there will be no negative consequences. Contact for Additional Information If you have any questions regarding this interview or this research project in general, please contact the ILC Africa Program Manager Jennifer Pierre at 055853860 or Erika Keaveney from Social Impact at 0201148919. Your Rights as a Participant This research has been reviewed and approved by the Radiological and Medical Science Research Institute of Atomic Energy Commission. If you have any issues on your rights as a participant, you can contact the Ethical Review Committee office between the hours of 8am-4:30pm through the telephone number or address provided below: Administrator, Radiological and Medical Sciences Research Institute’s Ethical Review Committee Ghana Atomic Energy Commission P. O. Box LG 80 Legon – Accra Tel: +233200402735 or 0303968932 Email: tishjon@yahoo.com / tishjon2@gmail.com Section C - VOLUNTEER AGREEMENT “The above document describing the benefits, risks, and procedures for the research titled “Ghana Early Grade Reading Impact Evaluation” has been read and explained to me. I have been given an opportunity to have any questions about the research answered to my satisfaction. I agree to participate as a volunteer. I will not have waived any of my rights by signing this consent form. Upon signing this consent form, I will receive a copy for my personal records.” STATEMENT OF PERSON OBTAINING INFORMED CONSENT: I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above individual. Name of Person Obtaining Informed Consent Signature of Person Obtaining Informed Consent Date Please provide one signed copy of this form to the Circuit Supervisor and retain one copy for ILC Africa’s records. Circuit Supervisor’s Name Circuit Supervisor’s Signature Date 254 GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PROTOCOL CONSENT FORM – IN LOCO PARENTIS Section A - BACKGROUND INFORMATION Title of Study: Ghana Early Grade Reading Impact Evaluation Principal Investigator: Erika Keaveney, Acting Program Director, Impact Evaluation, Social Impact Inc. Certified Protocol Number: RAMS/ERCP/SS/02/2018 Section B – CONSENT TO PARTICIPATE IN RESEARCH General Information About Research My name is _____________, and I am from Ivy League Consultants (ILC) Africa, an independent data collection firm working with the United States Agency for International Development (USAID)’s mission in Ghana and Social Impact, Inc., a research firm based in the Washington D.C. area of the United States of America. We are conducting a research study to assess the impact of an Early Grade Reading program which supports teachers and schools in Ghanaian language reading instruction across 100 districts of Ghana. The results of this study will be used by the Ghana Education Service and USAID to inform future programs aimed at helping children in Ghana become good readers. This school is among 470 schools across Ghana that has been selected to take part in this study through a process of statistical sampling. The school was visited in 2017 for a baseline assessment and we are now returning to conduct a midline assessment. Should you agree for your school to participate, this will involve a verbal interview with you, an interview with the school’s Curriculum Lead, an interview with one P1 reading teacher, observation of this teacher’s classroom lessons, and brief learning assessments and interviews with a group of P1 pupils. Specifically, ten (10) P1 pupils are to be randomly selected from a P1 classroom, ideally that of the same P1 teacher that was selected last year. Since children in this school are under your care during school hours, we are asking for your consent for their participation, on behalf of the children’s parents. If you agree to allow the children to participate, they will be asked to take Early Grade Reading assessments in the Ghanaian language of instruction as well as English. The learning assessments will provide us with information on their literacy abilities in local language as well as English. In addition, we will ask some questions about attendance, languages spoken, reading habits, and household assets. The assessments should take about 30 minutes per pupil to complete and will take place at school during regular school hours as the school schedule allows. All data collection activities at this school should be completed within approximately 4 hours. We will also return to this school at the end of the school year in 2019 to repeat the same procedures. Benefits/Risks of the Study There are no direct benefits to you for participating in the study, however, information collected in this study may benefit this and other schools in the future by improving early grade reading programming. Additionally, there are no known risks associated with this study, other than time lost from the classroom, which is expected to be no more than 30 minutes per pupil. To ease the disruption of class time that this might cause, the team will try to engage pupils for the assessments at a time convenient with their class schedules. 255 Confidentiality Every effort will be made to keep any information collected about yourself, children, teachers, and this school strictly confidential. To keep information about participants safe, each member of the research staff has signed a confidentiality agreement prior to conducting any data collection tasks. Any papers or electronic data with personal identifying information will be stored on password-protected electronic devices or in a locked room and no person outside of the research team will have access to this information. Upon conclusion of the study, all personal identifying information will be destroyed. Compensation There is no compensation provided for participating in this study. While there is no direct benefit to children in your school, a token of exercise books and pencils will be provided to pupils taking part in the study. Withdrawal from Study You and each of the teachers and pupils involved in this study can choose not to participate at all or to leave the study at any time, without penalty. Regardless of your or any teacher or child’s decision to participate in the research or not, there will be no negative consequences. Contact for Additional Information If you have any questions regarding this interview or this research project in general, please contact the ILC Africa Program Manager Jennifer Pierre at 055853860 or Erika Keaveney from Social Impact at 0201148919. Your Rights as a Participant This research has been reviewed and approved by the Radiological and Medical Science Research Institute of Atomic Energy Commission. If you have any issues on your rights as a participant, you can contact the Ethical Review Committee office between the hours of 8am-4:30pm through the telephone number or address provided below: Administrator, Radiological and Medical Sciences Research Institute’s Ethical Review Committee Ghana Atomic Energy Commission P. O. Box LG 80 Legon – Accra Tel: +233200402735 or 0303968932 Email: tishjon@yahoo.com / tishjon2@gmail.com Section C - VOLUNTEER AGREEMENT “The above document describing the benefits, risks, and procedures for the research titled “Ghana Early Grade Reading Impact Evaluation” has been read and explained to me. I have been given an opportunity to have any questions about the research answered to my satisfaction. I voluntarily agree to allow the children in my school to participate in this study provided they verbally assent to do so. I will not have waived any of the rights of children in my school by signing this consent form. Upon signing this consent form, I will receive a copy of the form for my personal records." STATEMENT OF PERSON OBTAINING INFORMED CONSENT: I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above individual. Name of Person Obtaining Informed Consent Signature of Person Obtaining Informed Consent Date Head Teacher’s Name Head Teacher’s Signature Date 256 Please provide one signed copy of this form to the Head Teacher and retain one copy for ILC Africa’s records. GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PROTOCOL CONSENT FORM – IN LOCO PARENTIS Section A - BACKGROUND INFORMATION Title of Study: Ghana Early Grade Reading Impact Evaluation Principal Investigator: Erika Keaveney, Acting Program Director, Impact Evaluation, Social Impact Inc. Certified Protocol Number: RAMS/ERCP/SS/02/2018 Section B – CONSENT TO PARTICIPATE IN RESEARCH General Information About Research My name is _____________, and I am from Ivy League Consultants (ILC) Africa, an independent data collection firm working with the United States Agency for International Development (USAID)’s mission in Ghana and Social Impact, Inc., a research firm based in the Washington D.C. area of the United States of America. We are conducting a research study to assess the impact of an Early Grade Reading program which supports teachers and schools in Ghanaian language reading instruction across 100 districts of Ghana. The results of this study will be used by the Ghana Education Service and USAID to inform future programs aimed at helping children in Ghana become good readers. This school is among 470 schools across Ghana that has been selected to take part in this study through a process of statistical sampling. The school was visited in 2017 for a baseline assessment and we are now returning to conduct a midline assessment. Should you agree for your school to participate, this will involve a verbal interview with you and brief learning assessments and interviews with a group of P1 pupils. Specifically, ten (10) P1 pupils are to be randomly selected from a P1 classroom, ideally that of the same P1 teacher that was selected last year. Since children in this school are under your care during school hours, we are asking for your consent for their participation, on behalf of the children’s parents. If you agree to allow the children to participate, they will be asked to take Early Grade Reading assessments in the Ghanaian language of instruction as well as English. The learning assessments will provide us with information on their literacy abilities in local language as well as English. In addition, we will ask some questions about attendance, languages spoken, reading habits, and household assets. The assessments should take about 30 minutes per pupil to complete and will take place at school during regular school hours as the school schedule allows. All data collection activities at this school should be completed within approximately 4 hours. We will also return to this school at the end of the school year in 2019 to repeat the same procedures. Benefits/Risks of the Study There are no direct benefits to you for participating in the study, however, information collected in this study may benefit this and other schools in the future by improving early grade reading programming. Additionally, there are no known risks associated with this study, other than time lost from the classroom, which is expected to be no more than 30 minutes per pupil. To ease the disruption of class time that this might cause, the team will try to engage pupils for the assessments at a time convenient with their class schedules. 257 Confidentiality Every effort will be made to keep any information collected about yourself, children, teachers, and this school strictly confidential. To keep information about participants safe, each member of the research staff has signed a confidentiality agreement prior to conducting any data collection tasks. Any papers or electronic data with personal identifying information will be stored on password-protected electronic devices or in a locked room and no person outside of the research team will have access to this information. Upon conclusion of the study, all personal identifying information will be destroyed. Compensation There is no compensation provided for participating in this study. While there is no direct benefit to children in your school, a token of exercise books and pencils will be provided to pupils taking part in the study. Withdrawal from Study You and each of the pupils involved in this study can choose not to participate at all or to leave the study at any time, without penalty. Regardless of your or any child’s decision to participate in the research or not, there will be no negative consequences. Contact for Additional Information If you have any questions regarding this interview or this research project in general, please contact the ILC Africa Program Manager Jennifer Pierre at 055853860 or Erika Keaveney from Social Impact at 0201148919. Your Rights as a Participant This research has been reviewed and approved by the Radiological and Medical Science Research Institute of Atomic Energy Commission. If you have any issues on your rights as a participant, you can contact the Ethical Review Committee office between the hours of 8am-4:30pm through the telephone number or address provided below: Administrator, Radiological and Medical Sciences Research Institute’s Ethical Review Committee Ghana Atomic Energy Commission P. O. Box LG 80 Legon – Accra Tel: +233200402735 or 0303968932 Email: tishjon@yahoo.com / tishjon2@gmail.com Section C - VOLUNTEER AGREEMENT “The above document describing the benefits, risks, and procedures for the research titled “Ghana Early Grade Reading Impact Evaluation” has been read and explained to me. I have been given an opportunity to have any questions about the research answered to my satisfaction. I voluntarily agree to allow the children in my school to participate in this study provided they verbally assent to do so. I will not have waived any of the rights of children in my school by signing this consent form. Upon signing this consent form, I will receive a copy of the form for my personal records." STATEMENT OF PERSON OBTAINING INFORMED CONSENT: I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above individual. Name of Person Obtaining Informed Consent Signature of Person Obtaining Informed Consent Date Head Teacher’s Name Head Teacher’s Signature Date 258 Please provide one signed copy of this form to the Head Teacher and retain one copy for ILC Africa’s records. GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PROTOCOL CONSENT FORM – HEAD TEACHER Section A - BACKGROUND INFORMATION Title of Study: Ghana Early Grade Reading Impact Evaluation Principal Investigator: Erika Keaveney, Acting Program Director, Impact Evaluation, Social Impact Inc. Certified Protocol Number: RAMS/ERCP/SS/02/2018 Section B – CONSENT TO PARTICIPATE IN RESEARCH General Information About Research As previously mentioned, I am ____________ from Ivy League Consultants (ILC) Africa, an independent data collection firm working with the United States Agency for International Development (USAID)’s mission in Ghana and Social Impact. We are conducting a research study to assess the impact of an Early Grade Reading program which supports teachers and schools in Ghanaian language reading instruction across 100 districts of Ghana. The results of this study will be used by the Ghana Education Service and USAID to inform future programs aimed at helping children in Ghana become good readers. This school is among 470 schools across Ghana that has been selected to take part in this study through a process of statistical sampling. As Head Teacher of this school, we would like to verbally interview you one-on-one to get some information about this school that will be useful for this study, including administrative data on pupils and teachers, teacher instructional practices in this school, your experiences with coaching teachers on reading instruction, and your beliefs on current models of instruction for improving early grade reading. This interview will last approximately 60 minutes. We will also return to this school at the end of the school year in 2019 to repeat the same procedures. Benefits/Risks of the Study There are no direct benefits to you for participating in the study, however, information collected in this study may benefit this and other schools in the future by improving early grade reading programming. Additionally, there are no known risks or discomforts associated with participating in this study. Confidentiality If you choose to participate, your responses will be strictly confidential. Your responses will be combined with those from other schools in the study and presented in the form of summary tables. Neither you nor your school will be individually identified or named in any reports. To keep the information you provide safe, each member of the research staff has signed a confidentiality agreement prior to conducting any data collection tasks. Any papers or electronic data with personal identifying information will be stored on password-protected electronic devices or in a locked room and no person outside of the research team will have access to this information. Upon conclusion of the study, all personal identifying information will be destroyed. Compensation 259 There is no compensation provided for participating in this study. Withdrawal from Study Your participation in this study is voluntary and you may withdraw from the study at any time without penalty. Regardless of your decision to participate in the research or not, there will be no negative consequences. Contact for Additional Information If you have any questions regarding this interview or this research project in general, please contact the ILC Africa Program Manager Jennifer Pierre at 055853860 or Erika Keaveney from Social Impact at 0201148919. Your Rights as a Participant This research has been reviewed and approved by the Radiological and Medical Science Research Institute of Atomic Energy Commission. If you have any issues on your rights as a participant, you can contact the Ethical Review Committee office between the hours of 8am-4:30pm through the telephone number or address provided below: Administrator, Radiological and Medical Sciences Research Institute’s Ethical Review Committee Ghana Atomic Energy Commission P. O. Box LG 80 Legon – Accra Tel: +233200402735 or 0303968932 Email: tishjon@yahoo.com / tishjon2@gmail.com Section C - VOLUNTEER AGREEMENT “The above document describing the benefits, risks, and procedures for the research titled “Ghana Early Grade Reading Impact Evaluation” has been read and explained to me. I have been given an opportunity to have any questions about the research answered to my satisfaction. I agree to participate as a volunteer. I will not have waived any of my rights by signing this consent form. Upon signing this consent form, I will receive a copy for my personal records.” STATEMENT OF PERSON OBTAINING INFORMED CONSENT: I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above individual. Name of Person Obtaining Informed Consent Signature of Person Obtaining Informed Consent Date Please provide one signed copy of this form to the Head Teacher and retain one copy for ILC Africa’s records. Head Teacher’s Name Head Teacher’s Signature Date 260 GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PROTOCOL CONSENT FORM – CURRICULUM LEAD Section A - BACKGROUND INFORMATION Title of Study: Ghana Early Grade Reading Impact Evaluation Principal Investigator: Erika Keaveney, Acting Program Director, Impact Evaluation, Social Impact Inc. Certified Protocol Number: RAMS/ERCP/SS/02/2018 Section B – CONSENT TO PARTICIPATE IN RESEARCH General Information About Research My name is _____________, and I am from Ivy League Consultants (ILC) Africa, an independent data collection firm working with the United States Agency for International Development (USAID)’s mission in Ghana and Social Impact, Inc., a research firm based in the Washington D.C. area of the United States of America. We are conducting a research study to assess the impact of an Early Grade Reading program which supports teachers and schools in Ghanaian language reading instruction across 100 districts of Ghana. The results of this study will be used by the Ghana Education Service and USAID to inform future programs aimed at helping children in Ghana become good readers. This school is among 470 schools across Ghana that has been selected to take part in this study through a process of statistical sampling. The school was visited in 2017 for a baseline assessment and we are now returning to conduct a midline assessment. Within each sampled school, we are interviewing the Curriculum Lead to understand how implementation of the Early Grade Reading program is going. This interview is expected to last around 30 minutes and will cover topics related to early grade reading instruction in Ghanaian language, including availability of resources and materials, access to and quality of in-service training, and provision of teacher coaching and monitoring. In addition, we will ask some questions about your beliefs on effective instructional methods for early grade reading. We will also return to this school at the end of school year in 2019 to repeat the same procedure. Benefits/Risks of the Study There are no direct benefits to you for participating in the study, however, information collected in this study may benefit this and other schools in the future by improving early grade reading programming. Additionally, there are no known risks or discomforts associated with participating in this study. Confidentiality If you choose to participate, your responses will be strictly confidential. Your responses will be combined with those from other schools in the study and presented in the form of summary tables. Neither you nor your school will be individually identified or named in any reports. To keep the information you provide safe, each member of the research staff has signed a confidentiality agreement prior to conducting any data collection tasks. Any papers or electronic data with personal identifying information will be stored on password-protected electronic devices or in a locked room and no person outside of the research team will have access to this information. Upon conclusion of the study, all personal identifying information will be destroyed. 261 Compensation There is no compensation provided for participating in this study. Withdrawal from Study Your participation in this study is voluntary and you may withdraw from the study at any time without penalty. Regardless of your decision to participate in the research or not, there will be no negative consequences. Contact for Additional Information If you have any questions regarding this interview or this research project in general, please contact the ILC Africa Program Manager Jennifer Pierre at 055853860 or Erika Keaveney from Social Impact at 0201148919. Your Rights as a Participant This research has been reviewed and approved by the Radiological and Medical Science Research Institute of Atomic Energy Commission. If you have any issues on your rights as a participant, you can contact the Ethical Review Committee office between the hours of 8am-4:30pm through the telephone number or address provided below: Administrator, Radiological and Medical Sciences Research Institute’s Ethical Review Committee Ghana Atomic Energy Commission P. O. Box LG 80 Legon – Accra Tel: +233200402735 or 0303968932 Email: tishjon@yahoo.com / tishjon2@gmail.com Section C - VOLUNTEER AGREEMENT “The above document describing the benefits, risks, and procedures for the research titled “Ghana Early Grade Reading Impact Evaluation” has been read and explained to me. I have been given an opportunity to have any questions about the research answered to my satisfaction. I agree to participate as a volunteer. I will not have waived any of my rights by signing this consent form. Upon signing this consent form, I will receive a copy for my personal records.” STATEMENT OF PERSON OBTAINING INFORMED CONSENT: I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above individual. Name of Person Obtaining Informed Consent Signature of Person Obtaining Informed Consent Date Please provide one signed copy of this form to the Curriculum Lead and retain one copy for ILC Africa’s records. Curriculum Lead’s Name Curriculum Lead’s Signature Date 262 GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PROTOCOL CONSENT FORM – P1 TEACHER Section A - BACKGROUND INFORMATION Title of Study: Ghana Early Grade Reading Impact Evaluation Principal Investigator: Erika Keaveney, Acting Program Director, Impact Evaluation, Social Impact Inc. Certified Protocol Number: RAMS/ERCP/SS/02/2018 Section B – CONSENT TO PARTICIPATE IN RESEARCH General Information About Research My name is _____________, and I am from Ivy League Consultants (ILC) Africa, an independent data collection firm working with the United States Agency for International Development (USAID)’s mission in Ghana and Social Impact, Inc., a research firm based in the Washington D.C. area of the United States of America. We are conducting a research study to assess the impact of an Early Grade Reading program which supports teachers and schools in Ghanaian language reading instruction across 100 districts of Ghana. The results of this study will be used by the Ghana Education Service and USAID to inform future programs aimed at helping children in Ghana become good readers. This school is among 470 schools across Ghana that has been selected to take part in this study through a process of statistical sampling. The school was visited in 2017 for a baseline assessment and we are now returning to conduct a midline assessment. Within this school, you and your reading class have been selected to be included in the study. This will involve observing a reading lesson followed by a 45-60 minute verbal, one-on-one interview with you. The purpose of the classroom observation is to document what teachers and pupils are doing during reading lessons as well as learn about teacher approaches to teaching reading. Please note that you are not being assessed or evaluated on your teaching performance and all the data collected will be used for statistical and research purposes only. As such, your reading lesson should proceed as if today were an ordinary day and you were not being observed. The 45-60 minute one-on-one interview will help us learn how teachers plan reading lessons, what materials and textbooks they use, how they go about assessing learners, and teacher beliefs on effective methods for teaching reading. We will also return to this school at the end of school year in 2019 to repeat the same procedures. Benefits/Risks of the Study There are no direct benefits to you for participating in the study, however, information collected in this study may benefit this and other schools in the future by improving early grade reading programming. Additionally, there are no known risks or discomforts associated with participating in this study. Confidentiality If you choose to participate, your responses will be strictly confidential. Your responses will be combined with those from other schools in the study and presented in the form of summary tables. Neither you nor your school will be individually identified or named in any reports. To keep the information you provide safe, each member of the research staff has signed a confidentiality agreement prior to conducting any data collection 263 tasks. Any papers or electronic data with personal identifying information will be stored on password-protected electronic devices or in a locked room and no person outside of the research team will have access to this information. Upon conclusion of the study, all personal identifying information will be destroyed. Compensation There is no compensation provided for participating in this study. Withdrawal from Study Your participation in this study is voluntary and you may withdraw from the study at any time without penalty. Regardless of your decision to participate in the research or not, there will be no negative consequences. Contact for Additional Information If you have any questions regarding this interview or this research project in general, please contact the ILC Africa Program Manager Jennifer Pierre at 055853860 or Erika Keaveney from Social Impact at 0201148919. Your Rights as a Participant This research has been reviewed and approved by the Radiological and Medical Science Research Institute of Atomic Energy Commission. If you have any issues on your rights as a participant, you can contact the Ethical Review Committee office between the hours of 8am-4:30pm through the telephone number or address provided below: Administrator, Radiological and Medical Sciences Research Institute’s Ethical Review Committee Ghana Atomic Energy Commission P. O. Box LG 80 Legon – Accra Tel: +233200402735 or 0303968932 Email: tishjon@yahoo.com / tishjon2@gmail.com Section C - VOLUNTEER AGREEMENT “The above document describing the benefits, risks, and procedures for the research titled “Ghana Early Grade Reading Impact Evaluation” has been read and explained to me. I have been given an opportunity to have any questions about the research answered to my satisfaction. I agree to participate as a volunteer. I will not have waived any of my rights by signing this consent form. Upon signing this consent form, I will receive a copy for my personal records.” STATEMENT OF PERSON OBTAINING INFORMED CONSENT: I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above individual. Name of Person Obtaining Informed Consent Signature of Person Obtaining Informed Consent Date Please provide one signed copy of this form to the Teacher and retain one copy for ILC Africa’s records. Teacher’s Name Teacher’s Signature Date 264 GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PUPIL ASSENT SCRIPT Section A- BACKGROUND INFORMATION Title of Study: h Ghana Early Grade Reading Impact Evaluation Principal Investigator: Erika Keaveney, Acting Program Director, Impact Evaluation, Social Impact Inc. Certified Protocol Number RAMS/ERCP/SS/02/2018 My name is ___________. I am working with a study for the Ghana Education Service. We are trying to understand how children learn to read. Yours is one of the schools we have chosen to help us. We would like your help in this process too. However, you do not have to participate in the study if you do not want to. I will ask you some questions and show you some letters, words, and short stories that I want you to read aloud in [GHANAIAN LANGUAGE OF INSTRUCTION]. I will also ask questions in English. This is NOT a test and it will not affect your grades in school. Nobody at your school will know how you do. I also have some questions about your reading classes. I will also ask some questions about which languages you use and some things that your family has at home. This should take 30 minutes or less. We will NEVER share your name or your answers with anyone who is not participating in the study. I repeat, you do not need to participate in this study if you do not want to. If there are any questions you do not want to answer after we have already started, you can choose not to answer them. Can we start? 265 GHANA ATOMIC ENERGY COMMISSION RADIOLOGICAL AND MEDICAL SCIENCES RESEARCH INSTITUTE (RAMSRI) Ethical Review Committee (ERC) PROTOCOL CONSENT FORM – PARENT INFORMATION SHEET Today’s date: _______________________ Dear Parent or Guardian: The Ministry of Education, Ghana Education Service, and United States Agency for International Development (USAID)/Ghana are jointly implementing a Partnership for Education project aimed at improving reading in primary schools in Ghana. As part of the project, USAID/Ghana has partnered with Social Impact to conduct an evaluation to assess the impact of a program which supports teachers and schools in mother tongue, phonics-based reading instruction across 100 districts of Ghana. The results of this study will be used by the Ghana Education Service and USAID to inform future programs aimed at helping children in Ghana become good readers. Your child was randomly selected among other children in their school to participate in this study. Our assessment team led by ILC Africa, a local data collection firm, administered an Early Grade Reading Assessment and a pupil questionnaire to your child. The assessment is NOT a test and it will not affect your child’s grades in school. Nobody at the school will have access to the assessment scores. The assessment scores will be combined with other children’s scores and reported as summary in the report. Since children in school are under the care of the Head Teacher during school hours, consent to assess and interview your child was obtained from the Head Teacher of the school. This information sheet is intended to provide you with information about your child’s participation in the study. You are assured that there are no known risks associated with participating in this study, other than time lost from the classroom, which is anticipated to be no more than 10-30 minutes per pupil. To ease the disruption of class time that this might cause, the team worked to engage pupils for the assessments at times convenient with their class schedules. There are no direct benefits to you or your child for participating in the study, however, information collected in this study may help the GES to improve early grade reading instruction in primary schools in Ghana. While there is no direct benefit to your child, a token of exercise books and pencils was provided to pupils taking part in the study. Confidentiality: Any information collected about your child will be keep strictly confidential and will not appear in any part of the study report nor will it be shared with anyone at his or her school. To ensure confidentially, each member of the research staff has signed a confidentiality agreement prior to conducting any data collection tasks. Any papers or electronic data with personal identifying information will be stored on password-protected devices or in a locked room and no person outside of the research team will have access to this information. Upon conclusion of the study, all information that could be used to potentially identify your child will be destroyed. Voluntary Participation: Your child was given the option to choose not to participate at all or to leave the study at any time, without penalty. Similarly, you as his/her parent or guardian have the option to withdraw your child from the study. If you wish to do so, please contact the ILC Africa Program Manager at the number below and your child’s data will be deleted from the study. 266 Contacts for Additional Information: If you have any questions regarding this interview or this research project in general, please contact the ILC Africa Program Manager Jennifer Pierre at 0558339042 or Erika Keaveney from Social Impact at 0201148919. Rights as a Participant: This research has been reviewed and approved by the Radiological and Medical Sciences Research Institute (RAMSRI-ERC). If you have any questions about your child’s rights as a research participant you can contact the ERC office between the hours of 8:30 am-4:30pm at email addresses: ramsrierc@yahoo.com or the ERC Administrator at tishjon@yahoo.com or at telephone numbers: 0303968932 or 0200402735. 267 ANNEX VII. STATEMENT OF WORK A third early grade reading evaluation is expected to emerge from the USAID Partnership for Education: Learning activity. The Contractor will work with MOE/GES, USAID and the Learning implementing partners to finalize this impact evaluation. (a) ACTIVITIES • Develop an Early Grade Reading Evaluation Design Report using format to be provided by USAID. • In close collaboration and coordination with the Learning contractor and the MOE/GES, develop or build on proposed or existing statistical tools (such as design of forms, questionnaires, surveys and/or census) to gather baseline data on relevant indicators, particularly impact indicators at the relevant levels and conduct subsequent data collection. • Design the evaluation approach, including collection of baseline data, mid-term and final data collection, analysis and dissemination of results. • Complete impact evaluation, including developing testable hypotheses and samples to show whether or not USAID activities have had an impact on literacy, teaching and learning. (b) EXPECTED OUTPUTS • Early Grade Reading impact evaluation Design Report developed using format to be provided by USAID. • Entrance briefing(s) conducted. • Exit briefing(s) conducted. (c) DELIVERABLES • Early Grade Reading impact evaluation Design Report. • Draft work plan and data collection protocols. • Final work plan and data collection protocols. • Evaluation data sets. • Draft Early Grade Reading impact evaluation report. • Final Early Grade Reading impact evaluation report. • PowerPoint presentation for Early Grade Reading impact evaluation to USAID, MOE/GES and relevant development partners.” 268 ANNEX VIII. DISCLOSURE OF CONFLICT OF INTEREST FORMS 269 270 271 272 273 274 275 276 ANNEX IX. EVALUATION TEAM Erika Keaveney is a Senior Program Manager for IE at SI, and was responsible for leading the evaluation design, data collection, and fieldwork for the EGRA, which included training field supervisors. Ms. Keaveney has over eight years of experience in international development research, evaluation and project management and currently manages an $8 million portfolio of USAID-supported IEs in Ghana and Malawi, focusing on early grade literacy and numeracy. Ms. Keaveney serves as a Board Member of SI’s IRB. She holds an MA in International Comparative Education from Stanford University and a BA in Political Science from the University of California-San Diego. Dr. Andrew Epstein serves as Senior Education and Evaluation Technical Specialist at SI and was a lead researcher for the EGRA, supporting the EGRA design, data collection, and fieldwork trainings. At SI, Dr. Epstein implements impact and PEs of international development and humanitarian aid projects across multiple sectors, focusing specifically on the areas of education and youth. He has designed IEs of three educational pilot projects in Ghana in the areas of teacher retention, LOI policy, and mathematics education. Dr. Epstein has a Ph.D. from the University of Wisconsin-Madison where he focused on International and Comparative Education and an M.Ed. in Educational Leadership and Policy Studies from the University of Washington. Data collection for the EGRA was supported by members of the ILC Africa team, led by Jennifer Pierre. Ms. Pierre has over ten years of experience in project monitoring and evaluation services, IE, and performance management frameworks, as well as experience supporting assessments in Ghana for organizations including USAID and CARE. She holds a Masters in Science, Population and International Health/Management Studies from the Harvard School of Public Health. The evaluation team was also supported by Mohammed Dawuda, Senior Monitoring and Evaluation Specialist for Evaluating Systems in Ghana, who trained field supervisors on data collection procedures and monitored data collection quality and progress. SI’s Sr. Technical Advisor, Mike Duthie provided oversight over and support to the design of the project, tool development and report writing. Project Assistant Miguel Albornoz provided data analysis and report preparation. Project Assistant Tommie Thompson provided additional technical support to the team through data analysis and cleanup, and development of the draft baseline report. 277 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523