JULY 2017 This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. PERFORMANCE & IMPACT EVALUATION (P&IE) OF THE USAID/UGANDA SCHOOL HEALTH AND READING PROGRAM: RESULT 1 SCHOOL LEVEL INTERVENTIONS Impact Evaluation – Final Report Final Report | i PERFORMANCE & IMPACT EVALUATION (P&IE) OF THE USAID/UGANDA SCHOOL HEALTH AND READING PROGRAM: RESULT 1 SCHOOL LEVEL INTERVENTIONS IMPACT EVALUATION FINAL REPORT July 2017 PN 7384; USAID Contract N0: AID-617-C-12-00006 PRESENTED TO: USAID/Uganda PRESENTED BY: NORC at the University of Chicago 4350 East-West Highway, 8th Floor Bethesda, MD 20814 Telephone: (301) 634-9413 Fax: (301) 634-9301 This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | ii CONTENTS Table of Figures.........................................................................................................................iv Table of Tables ...........................................................................................................................v Acronyms...................................................................................................................................vi Executive Summary ................................................................................................................... 8 Project Background .................................................................................................................8 Evaluation design ....................................................................................................................9 Findings and Conclusions........................................................................................................9 Recommendations.................................................................................................................11 A. Introduction ...................................................................................................................13 A.1 Project Description ..........................................................................................................13 A.2 Evaluation Design ...........................................................................................................16 A.3 Assignment of Schools to Treatment and Control Groups and Estimation Strategy.........17 A.4 Impact Indicators And Data Instruments..........................................................................19 A.5 collection of reading assessment data.............................................................................21 A.6 Assessor training.............................................................................................................21 A.7 FOCUS GROUP DISCUSSION METHODOLOGY..........................................................22 B. Balance at Baseline.......................................................................................................22 C. Impact Analysis: Quantitative Findings ..........................................................................23 C.1 Cluster 1 Schools............................................................................................................24 C.2 Cluster 2 Schools............................................................................................................28 C.3 Cluster 3 Schools............................................................................................................32 C.4 Impact on Boys vs. Girls..................................................................................................36 C.5 Factors that MAY BE hinderING SHRP Impact on reading performance.........................40 D. Limitations .....................................................................................................................49 E. Conclusions...................................................................................................................49 F. recommendations..........................................................................................................51 Annex A. Sample Size ..............................................................................................................53 Cluster 1................................................................................................................................54 Cluster 2................................................................................................................................56 Cluster 3................................................................................................................................57 Annex B. Sample Balance at Baseline ......................................................................................58 Annex C. EGRA Subtasks, Distributional Graphs by Language Group, baseline and endline ...65 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | iii Annex D. Sample Balance at endline ......................................................................................155 Annex E. Interactions Between Student Absenteeism and SHRP Treatment ..........................168 Annex F. Exploration of Possible Causes of Underperformance of SHRP (Full Report)..........171 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | iv TABLE OF FIGURES Figure 1: Assignment to Treatment and Control within Districts ...............................................................18 Figure 2: Cluster 1, Letter Sound Scores at baseline and after four years of program, by language...........................................................................................................................................................................25 Figure 3: Cluster 1, SHRP estimated effects on letter sound identification since baseline, by language...........................................................................................................................................................................26 Figure 4: Cluster 1, Oral reading fluency at baseline and after four years of program, by language. .........................................................................................................................................................................27 Figure 5: Cluster 2, Letter Sound Scores at baseline and after two years of intervention, by language...........................................................................................................................................................................29 Figure 6: Cluster2, SHRP estimated effects on letter sound identification since baseline, by language...........................................................................................................................................................................30 Figure 7: Cluster 2, Oral reading fluency at baseline and after three years of intervention, by language...........................................................................................................................................................................31 Figure 8: Cluster 2, Zero Scores on oral reading fluency at baseline and after three years of intervention, by language ..........................................................................................................................................32 Figure 9: Cluster 3, Letter Sound Scores at baseline and after one year of intervention, by language...........................................................................................................................................................................34 Figure 10: Cluster 3, SHRP estimated effects on letter sound identification since baseline, by language...........................................................................................................................................................................34 Figure 11: Cluster 3, Oral reading fluency at baseline and after two years of intervention, by language...........................................................................................................................................................................35 Figure 12: Cluster 1, Letter sound score and oral reading fluency, .........................................................37 Figure 13: Cluster 2, Letter sound score and oral reading fluency, .........................................................38 Figure 14: Cluster 3, Letter sound score and oral reading fluency, .........................................................39 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | v TABLE OF TABLES Table 1: Oral Reading Fluency (number of words per minute) Overview by Cluster and Language.........................................................................................................................................................................10 Table 1: SHRP Clusters, Languages, Regions and Districts......................................................................16 Table 2: Early Grade Literacy Skills, EGRA Subtasks at Endline.............................................................20 Table 3: Data Collection Rounds by Cluster......................................................................................................21 Table 4: Cluster 1 Estimated SHRP School Level Effects after 4 years of program.........................24 Table 5: Cluster 2 Estimated SHRP School Level Effects after three years of program.................28 Table 6: Cluster 3 SHRP School Level Effect...................................................................................................33 Table 7: Learner Absenteeism in the Previous Week, ..................................................................................43 Table 8: Estimated Teacher Absenteeism in the Previous Week,............................................................43 Table 9: Cluster 1, Estimated Effects of SHRP on Oral Reading Fluency: Interaction with Learner Absenteeism..................................................................................................................................................44 Table 10: Cluster 1, Estimated Effects of SHRP on Oral Reading Fluency: Interaction with Teacher Absenteeism.................................................................................................................................................45 Table 11: Reading at Home and Bringing Reading Books from School to Home, at Endline and by Language...................................................................................................................................................................46 Table 12: Cluster 1, Estimated Effects of SHRP on Oral Reading Fluency: Interaction with Reading at Home with Someone............................................................................................................................48 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | vi ACRONYMS AIDS Acquired Immune Deficiency Syndrome CAO Chief Administrative Officer CAQDAS Computer-Assisted Qualitative Data Analysis Software CC Coordinating Center CCT Coordinating Center Tutors CDO Community Development Officer CSR Center for Social Research DEO District Education Officer DIS District Inspector of schools DOS Director of Studies DQA Data Quality Assurance EGR Early Grade Reading EGRA Early Grade Reading Assessment FA Field Assistants. FGD Focus Group Discussion HIV Human Immunodeficiency Virus IE Impact Evaluation IP Implementing Partner IRB Institutional Review Board IRR Inter-rater Reliability Test KIIs Key Informant Interviews LLB Local Language Boards MoES Ministry of Education and Sports NCDC National Curriculum Development Centre P1, P2, P3, P4 Primary 1, 2, 3 or 4 P&IE Performance and Impact Evaluation PE Performance Evaluation PTA Parent Teachers Association SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | vii RCT Randomized Controlled Trial RTI Research Triangle Institute SHRP School Health and Reading Program SST Social Studies TOT Training of Trainers UNEB Uganda National Examination Board USAID United States Agency for International Development VHTs Village Health Teams VSO Voluntary Services Overseas SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 8 EXECUTIVE SUMMARY NORC at the University of Chicago conducted the Impact Evaluation of USAID/Uganda's School Health and Reading Program (SHRP), implemented by RTI and partners. Implementation of SHRP began in 2012 and has reached 31 districts in Uganda to date. The program consists of two sets of activities: Result 1 activities pertaining to Improved Early Grade Reading and Transition to English; and Result 2 activities pertaining to Improved HIV/AIDS Knowledge, Attitudes and Practice. This reports presents the findings of the impact evaluation of SHRP’s Result 1 activities. The implementation of SHRP Result 1 activities started in 2013 in Cluster 1 schools, in a group of districts that include four local languages: Luganda, Ateso, Runyankore/Rukiga, and Leblango. The next year, the program extended to Cluster 2 schools, which included districts covering four other local languages: Runyoro/Rutoro, Acholi, Lugbarati and Lamasaaba. Finally, in 2015 the program was rolled out to Cluster 3 schools in districts that included another four local languages: Lugwere, Nkarimojong, Lukhonzo and Lusoga1 . This evaluation, which commenced in October 2012 and wrapped up in 2017, sought to measure the impact of SHRP on reading and literacy skills among early grade learners. Specifically, it answers the following question: What is the causal impact of SHRP’s early grade reading intervention package on the literacy skills of learners in clusters 1, 2 and 3, at the end of P2, P3 and P4 respectively as measured by the Early Grade Reading Assessment (EGRA) in English and mother tongues? In addition, the evaluation used qualitative methods – two performance evaluations, data collected through continuous evaluations, and a final set of focus group discussions with teachers - to add depth to the quantitative findings, wherever possible. PROJECT BACKGROUND The USAID/Uganda School Health and Reading Program (SHRP) was originally planned as a 4 year intervention aimed at improving the early grade reading and literacy scores of students in 31 districts across Uganda. The project, funded by USAID and implemented by Research Triangle Institute (RTI) provided 3 levels of intervention:: 1. National level: Systems and pedagogical and language frameworks aim to strengthen and support mother-tongue based early grade reading and transition to English. The intent is to help strengthen policies related to reading; increase advocacy for reading at multiple levels (e.g. student, teacher, school, district, and national); and generate and use data for programmatic decision making. 2. District level: Training for district education officials to raise awareness and build support for the new curriculum. 3. School level: Teacher training in early grade literacy instruction using students’ mother￾tongue in P1-P3 and with a transition to English in P4; distribution of textbooks and 1 At the time of submission of this report, the Result 1 team had collected data for Round 1 (Feb-Mar 2013), Round 2 (Oct 2013), Round 3 (Oct 2014), Round 4 (Oct 2015) and Round 5 (Oct 2016) for Cluster 1; Round 1 (Feb 2014), Round 2 (Oct 2014), Round 3 (Oct 2015) and Round 4 (Oct 2016) for Cluster2; Round 1 (Feb 2015), Round 2 (2015) and Round 3 (Oct 2016) for Cluster 3. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 9 instructional materials and teacher guides; support supervision for teachers; leadership training for head teachers; and reading competitions and community awareness activities (reading awareness days, literacy week, etc.). The hypothesis is that together, these interventions will improve the instruction and learning environment of students and in turn lead to better literacy skills among early grade learners. NORC’s evaluation tests this hypothesis. EVALUATION DESIGN To measure the impact of the SHRP Result 1 interventions at the school level, NORC used a randomized controlled trial (RCT) design, which relies on random assignment of schools to treatment and control groups within SHRP intervention districts. This experimental design allows us to isolate the effects of school-level interventions. The evaluation also included several qualitative research activities to complement the impact evaluation. These include (1) a set of performance evaluations that were conducted in 2014 and 2016, that is, in years 2 and 4 of the project, to assess the implementation of activities and shed light on the findings of the impact evaluation; (2), a “continuous evaluation” exercise that accompanied implementation of the project for most of the project’s duration, providing regular feedback to the IP based on regular observation of activities; and a (3) final targeted qualitative research component using focus group discussions with teachers to delve more deeply into some of the IE’s findings. Key outcomes for this evaluation comprised of students’ scores on five main reading skill areas including alphabetics (letter knowledge and phonemic awareness), fluency, vocabulary, and comprehension and phonics. The Early Grade Reading Assessment, (EGRA) a brief oral reading assessment that tests these skills and is widely used by USAID to measure student reading outcomes, was used to measure the program’s impacts on literacy. Because of SHRP’s focus on mother tongue-based education with transition of English over the course of four years, all literacy skills were assessed in mother tongue and English. In addition to the EGRA, the evaluation used data collected through learner context interviews, teacher and head teacher surveys, and school inventories FINDINGS AND CONCLUSIONS This evaluation measured the impact of SHRP on the three language clusters separately. After four years of SHRP interventions, most languages subgroups in Cluster 1 started to show more positive impacts. SHRP effects are still small in absolute values but effect sizes are larger than in previous years and, hence, more encouraging. The one exception in Cluster 1 languages is Leblango, where SHRP only had a positive impact on letter reading sound identification. In general, findings for Cluster 2, which received SHRP interventions for three years, show lack of or very modest impacts. Runyoro/Rutooro is the only language in this cluster that shows show larger and more consistent positive effects of SHRP. Cluster 3, after two years of interventions, shows positive effects on the letter sound identification but very little impact on other subtasks. Findings for the Ngakarimojong subgroup are better but still small. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 10 There is no evidence that SHRP had any impact on the listening comprehension or on English receptive vocabulary for any cluster/language with the exception of Acoli where the impact on listening comprehension is negative. The fact that we do not find any effect in this type of subtasks suggests that the treatment did not generate development of language skills or English vocabulary above and beyond the regular development in control schools. Overall, the evaluation findings point to heterogeneous and often small impacts, with absolute reading skills remaining low across most languages and reading skills. However, we do see more promising and consistent impacts on one specific reading skill, Oral Reading Fluency (ORF). As Table 1 shows, for ORF, where impact is observed, it is always positive, showing that children from treatment schools show improvement linked to SHRP activities. Cohort 1 students showed positive impact for ORF in all language groups except Leblango, where no impact was detected. Each of the other languages groups, including English, show positive impact of between 4.5 and 6.4 words per minute (wpm). Similarly, Cluster 2 students showed positive impacts in all of the language groups except for Lugbarati, where no impact was detected. Each of the other language groups, including English, observed an impact of between 2.4 and 7.2 wpm. Cluster 3 students showed the least impact in ORF, with only Nagkarimonjong and Lusoga showing small positive impacts of .4 and 2.6 wpm respectively. Looking at the results across clusters, Cluster 1 shows the most consistent and positive impact in ORF, although the Runyoro/Rutooro students in Cluster 2 showed the highest impact of 7.2 wpm. Oral Reading Fluency (number of words per minute) Overview by Cluster and Language Cluster/Language Average in Treatment Schools Statistically Significant Improvement Over Control Schools Effect Size Cluster 1 (end of P4) Luganda 20 4.50 0.20 Leblango 14.9 no impact no impact Ateso 12.8 6.20 0.44 Runyankore/Rukiga 26.6 6.40 0.44 English 24.6 5.00 0.28 Cluster 2 (end of P3) Runyoro/Rutooro 14.2 7.20 0.50 Acoli 3.4 2.40 0.32 Lugbarati 6.6 no impact no impact Lumasaaba 4.1 2.40 0.36 English 7.1 3.40 0.28 Cluster 3 (end of P2) Lugwere 0.44 no impact no impact Nagkarimojong 3.4 2.60 0.30 Lhukonzo 1.8 no impact no impact Lusoga 0.45 0.40 0.20 English not tested not tested not tested SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 11 Despite the positive impacts in ORF, the fact still remains that SHRP’s focused interventions over multiple years did little to move the needle on Ugandan students’ overall reading abilities. Recognizing this, NORC supplemented the impact evaluation with qualitative research to try and better understand the lower than expected overall impacts of SHRP. During the midterm and final performance evaluations conducted by the NORC team, four potential explanations were identified through key informant interviews and FGDs: 1. Teacher trainings fell short of fully training and/or motivating teachers to implement SHRP techniques in their classrooms; 2. Teacher support supervision did not provide sufficient support to ensure that teachers were able to implement technical effectively in the classroom; 3. Community mobilization efforts were insufficient to complement classroom efforts at home; and 4. High absenteeism among learners and teachers means that they do not meet regularly enough for training, coaching, and learning materials to take effect. In Year 5 of the P&IE projects, NORC further explored these posited reasons for low reading performance among students in SHRP schools through a series of focus group discussions (FGDs) with teachers who had benefited from SHRP trainings and instructional materials. These FGDs provided important insight into possible limitations of the SHRP interventions that should be considered in future efforts to replicate the program. Overall, while teachers believe the content of SHRP training was comprehensive, they stated that the days allotted for training were insufficient; as a result, they claimed that they did not acquire full and sufficient understanding of the techniques being taught. They also pointed to deficiencies in the instructional materials as a challenge in classroom implementation of teaching techniques. The FGDs also pointed to teacher and student absenteeism as a main reason for SHRP’s disappointing impacts. Main reasons for teacher absenteeism, as revealed by the teachers themselves, were sickness, long distances to schools, weather, lack of motivation, low salaries and delayed paydays, and family responsibilities. Teachers also pointed to student absenteeism as an impediment to improving students’ reading skills. One of the reasons for student absenteeism, according to teachers, was parentss’ clear preference for English language instruction, and the belief that mother tongue-based education, promoted through SHRP, is not useful. As a result, parents tend to put children in private schools or pull them out of school all together. These FGD response provided valuable insight that may help understand the quantitative results. RECOMMENDATIONS A number of recommendations stem from the FGDs with teachers. We present a few below: Redesign the training course to allow for more exposure and understanding of the methods and content. Most teachers felt that they could not effectively apply the new methods and content to the children in class because they were not fully equipped to apply the new methods and content due to a short period of exposure to the course. The training course for the teachers should be revised to increase the frequency and length of training, with a systematic follow up plan be put in place for conducting refresher courses. Revise text books. Teachers pointed to missing content and existing errors in the SHRP books as an issue that impeded their ability to teach. This should be addressed, perhaps through a participatory approach with a revision team of teachers. In addition, the IP could identify and recommend particular textbooks for cross referencing to support the existing SHRP textbooks. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 12 Explore ways to motivate teachers through non-monetary incentives. Teachers noted that they are not recognized for their contribution to learning and reading which has led to low teacher morale. Teachers should be motivated through non-monetary incentives like provision of certificates of attendance and recognition on open days. Improve teacher support supervision: All teachers had received some form of support supervision but claimed it was inadequate to enable them to apply the taught content and new methods to the classrooms. A comprehensive and coordinated systematic support supervision plan should be developed and communicated to the schools to support the SHRP program. It may also be useful to consider adding peer-to-peer support Efforts to improve school attendance among pupils. Although some reasons behind learners’ absenteeism –such as bad health and child labor – cannot be addressed by the program, there is some room to improve attendance. Efforts should be directed to raise awareness about the importance of attending school every day even in the early grades, starting on the first day and attending through the last day of each term. Non-monetary incentives could be used to recognize learners and families that have started the year on time, or show high attendance. Measures to reduce corporal punishment. Children exposed to violence suffer multiple negative consequences and are also less likely to attend school consistently. Corporal punishment, which is common in Ugandan schools, needs to be addressed. Fortunately, the USAID Literacy Achievement and Retention Activity (LARA) is already working to this end with an intervention that includes teacher, parent and community trainings to improve retention of primary grade students by fostering safe school environments Work to change the negative parental perception towards SHRP. Teachers from the study districts noted that parents’ attitude towards SHRP, as a program taught in the vernacular, is negative and that is affecting attendance. Efforts to build community support for the program should be stepped up to target this negative perception. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 13 A. INTRODUCTION NORC at the University of Chicago conducted the Impact Evaluation of USAID/Uganda's School Health and Reading Program (SHRP), implemented by RTI and partners. RTI (referred to as the implementing Partner or IP in the rest of the document) has been implementing USAID's School Health and Reading Program (SHRP) in 31 districts in Uganda since 2012. The program consists of two sets of activities: Result 1 activities pertaining to Improved Early Grade Reading and Transition to English; and Result 2 activities pertaining to Improved HIV/AIDS Knowledge, Attitudes and Practice. This reports presents the findings of the impact evaluation of SHRP’s Result 1 activities. The implementation of SHRP Result 1 activities started in Cluster 1 schools, in a group of districts that include four local languages: Luganda, Ateso, Runyankore/Rukiga, and Leblango. The next year, the program extended to Cluster 2 schools, which included districts covering four other local languages: Runyoro/Rutoro, Acholi, Lugbarati and Lamasaaba. Finally, in 2015 the program was rolled out to Cluster 3 schools in districts that included another four local languages: Lugwere, Nkarimojong, Lukhonzo and Lusoga2 . This evaluation specifically answers the following question: What is the causal impact of SHRP’s early grade reading intervention package on the literacy skills of learners in clusters 1, 2 and 3, at the end of P2, P3 and P4 respectively as measured by the Early Grade Reading Assessment (EGRA) in English and mother tongues? In addition, the evaluation used qualitative methods – two performance evaluations, data collected through continuous evaluations, and a final set of focus group discussions with teachers - to add depth to the quantitative findings, wherever possible. A.1 PROJECT DESCRIPTION For its Improved Early Grade Reading and Transition to English (Result 1) interventions, SHRP focuses on the nexus of language, pedagogy, and instructional materials to significantly improve students’ early grade reading and literacy scores (RTI International, 2012, p. 1). The program has aimed to build institutional capacity, support policy development, and help institutionalize training, support structures, and policies necessary for sustainability. The intervention works at multiple levels and consists of: 4. At the national level, systems and pedagogical and language frameworks aim to strengthen and support mother-tongue based early grade reading and transition to English. The intent is to support the strengthening of policies related to reading; increase advocacy for reading at multiple levels (e.g. student, teacher, school, district, and national); and generate and use data for programmatic decision making. 5. At the district level, training for district education officials to raise awareness and build district-level support for the new curriculum. 2 At the time of submission of this report, the Result 1 team had collected data for Round 1 (Feb-Mar 2013), Round 2 (Oct 2013), Round 3 (Oct 2014), Round 4 (Oct 2015) and Round 5 (Oct 2016) for Cluster 1; Round 1 (Feb 2014), Round 2 (Oct 2014), Round 3 (Oct 2015) and Round 4 (Oct 2016) for Cluster2; Round 1 (Feb 2015), Round 2 (2015) and Round 3 (Oct 2016) for Cluster 3. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 14 6. At the school level, the activity is comprised of teacher training in early grade literacy instruction using students’ mother-tongue in P1-P3 and with a transition to English in P4; distribution of textbooks and instructional materials and teacher guides in local languages and English; support supervision provided to teachers; trainings in leadership for head teachers; and reading competitions and community awareness activities (reading awareness days, literacy week, etc.). The hypothesis is that together, these interventions will improve the instruction and learning environment of students and in turn lead to better literacy skills among early grade learners. Teacher trainings under Result 1 are implemented mostly3 through Coordinating Center Tutors (CCTs), who are school support workers in charge of monitoring education quality within their Coordinating Centers (CC). Under the district education structure, selected CCTs are responsible for a certain number of schools within a district (one district typically has multiple CCTs). The CCTs selected for the intervention received training directly from SHRP and, in turn, delivered teacher training and program support in their schools, thus following a Training of Trainers (TOT) model. Teacher trainings at the school level focus on pedagogy with an emphasis on using structured lesson plans and learner books. These lesson plans provide teachers with a practical step-by￾step process for implementing the transitional bilingual approach mandated by the Ugandan Early Grade Reading (EGR) policy. In addition, SHRP developed materials to support EGR. These materials were adapted in order to take into account the different needs of learners at different stages of cognitive and academic development, and the linguistic characteristics of the different local languages, rather than being translated directly from one language to another. Furthermore, in order to develop these materials, SHRP worked with MoES and Local Language Boards (LLBs) to standardize orthographies of the target languages. All materials follow the same general pedagogical framework to facilitate guidelines for textbook development and teacher training. Teachers received these instructional materials in their local language of instruction and English. Teachers also received ongoing monitoring support supervision from SHRP staff that observed classes and provided constructive feedback. Additionally, at the district level, workshops and training for district education officials served as a forum for raising awareness and building district-level support for the new curriculum. Communication campaigns directed at Members of Parliament and district leaders were also organized. Finally, at the national level, SHRP worked with the MoES and the Sector Policy Management Working Group to develop a Uganda-specific reading strategy, which included policies in the areas of LLB development, textbook development, printing as well as Special Needs. In particular, SHRP worked to develop a national literacy strategy, national reading standards and benchmarks, as well as to harmonize reading assessment efforts with the Uganda National Examination Board (UNEB). SHRP also worked to strengthen LLBs and to raise awareness about special needs education. SHRP assisted the MoES in advocating for reading outside the classroom. Together with the MoES, SHRP aimed to raise awareness of local language development, reading instructions and special needs learners by using national communication campaigns through mass media and mobilizing local communities. 3 Pre-service tutors, district inspectors and writers are also teacher trainers. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 15 In total, SHRP worked with 12 local languages, developing instructional materials for each language and providing training and in-service support to teachers in geographic areas where these 12 languages are spoken and used for mother-tongue based instruction in the early grades. The intervention was implemented following a pipeline roll-out, with the intervention targeting districts associated with the first four local languages starting in Year 1 (Cluster 1), then rolling out to districts associated with an additional four local languages starting Year 2 (Cluster 2), and finally to districts associated with the final four local languages starting in Year 3 (Cluster 3). As such, the original plan was for Cluster 1 students to receive the intervention for a total of 4 years, Cluster 2 students for 3 years and Cluster 3 students for 2 years. However, SHRP was extended for two additional in all clusters. SHRP is targeting the following local languages and associated districts: SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 16 Table 1: SHRP Clusters, Languages, Regions and Districts Cluster Local Language Region Districts 1 Luganda Central Wakiso, Gomba Runyankore/Rukiga South West Kiruhura, Bushenyi, Kabale Ateso Eastern Kumi, Katakwi, Serere Leblango Northern Apac, Lira, Kole 2 Runyoro/Rutoro Mid-Western Masindi, Kyenjojo, Kbarole Acholi Mid-Northern Gulu, Pader, Kitgum Lugbarati West Nile Arua Lumasaaba Mid-Eastern Mbale, Sironko, Manafwa 3 Lugwere Mid-Eastern Budaka, Pallisa, Kibuku Nkarimojong North East Nakapiripirit, Napak, Moroto, Kaabong Lukhonzo Mid-Eastern Kasese Lusoga East Central Iganga and Kamuli These 31 project districts were chosen by the IP and MoES; the selection was not part of the evaluation design. This report focuses on the impact of SHRP after four years of intervention. The program started in early 2013 in Cluster 1, which has been exposed to SHRP Result 1 for 4 years. The following year, in 2014, SHRP expanded to Cluster 2 and, therefore, learners in this group have received the intervention for three years. Finally, Cluster 3 started receiving SHRP interventions in 2015, and has completed two years of the intervention. In all cases, the program started at the beginning of Primary 1. A.2 EVALUATION DESIGN To assess the impact of the School Health and Reading Program Result 1 interventions at the school level, NORC used an experimental (randomized controlled trial, or RCT) design, which relies on random assignment of schools to treatment and control groups within SHRP intervention districts. This experimental design allowed us to isolate the effects of school-level interventions. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 17 The evaluation design also included several qualitative research activities to complement the impact evaluation. These include (1) a set of performance evaluations that were conducted in 2014 and 2016, that is, in years 2 and 4 of the project, to assess the implementation of activities and shed light on the findings of the impact evaluation; (2), a “continuous evaluation” exercise that accompanied implementation of the project for most of the project’s duration, providing regular feedback to the IP based on regular observation of activities; and a final targeted qualitative research component using focus group discussions with teachers to delve more deeply into some of the IE’s findings. A.3 ASSIGNMENT OF SCHOOLS TO TREATMENT AND CONTROL GROUPS AND ESTIMATION STRATEGY As described previously, CCTs are responsible for training teachers and providing follow-up support and assistance in implementing the Result 1 interventions. Towards this end, SHRP conducted training workshops (training of trainer workshops) for CCTs in different regions. Since each CCT is responsible for several schools, randomizing at the school level would imply that a CCT would have to treat schools under his or her jurisdiction differently if some were designated as treatment schools and others as controls. Therefore, randomization at the school level had a high risk of 'contamination' between treatment and control groups. To avoid this potential problem, we opted to randomize at the CCT level, assigning the entire cluster of schools under a CCT to either the treatment or the control group. Figure 1 illustrates the evaluation design, i.e. random assignment into the treatment or control group within treatment districts. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 18 Figure 1: Assignment to Treatment and Control within Districts The difference in outcome indicators between treatment schools (green) and control schools (orange) will show the effect of the school-level intervention. We estimate the impact of the intervention for each language on a number of literacy outcomes using a difference in differences model. To evaluate the impact of the intervention on literacy score, Y, for example, for students in Cluster 1 within a specific language group, we use the scores collected at the beginning of the school year 2013 (baseline) when they entered P1 and from the end of their fourth year of school when they finished P4 (endline), and regress the individual test score Y of student i on the treatment status T of the school s in district d, a dummy variable P indicating period (i.e. 0 or 1 respectively for baseline and endline) and their interaction T*P: 𝑌𝑖𝑠𝑑𝑝 = 𝛽0 + 𝛽1 𝑇𝑠𝑑 + 𝛽2 𝑃 + 𝛽3 𝑇𝑠𝑑 ∗ 𝑃 + 𝛽4 𝐷𝑑 + 𝛽5 𝑋𝑖 + 𝛽6 𝑍𝑠 + 𝜀𝑖𝑠𝑑 (1) where Xi are individual characteristics of the student i, such as age and sex, Zs are school characteristics such as language of instruction, Dd is a vector of dummy variables indicating the districts. As we mentioned Tsd is a dummy equal to 1 if school s in district d received the intervention and 0 otherwise, P indicates the period –baseline or endline, therefore, our coefficient of interest is 𝛽3, which shows the differential effect of the treatment at the endline. SHRP Districts District Level CCT Level CCT Randomization School School School School School School School School School School School School School School School School Treatment CCTs Control CCTs School Level Random Selection of Schools for Data Collection SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 19 A.4 IMPACT INDICATORS AND DATA INSTRUMENTS Literacy is comprised of multiple skills, both receptive and productive. Successful readers must be able to identify letters and their corresponding sounds, segment and blend those sounds to form words and sentences, master appropriate vocabulary, and make meaning from text, among other skills. They must also be able to demonstrate their understanding and engagement with text through writing. To assess the effectiveness of the SHRP in reaching its goal to improve early grade reading and transition to English, specific key literacy skills were assessed. The consensus among the reading research community in the United States is that effective reading instruction attends to at least five main reading skill areas including alphabetics (letter knowledge and phonemic awareness), fluency, vocabulary, and comprehension and phonics (National Reading Panel, 2000; Snow, Burns & Griffin, 1998).4 Based on this research, the Early Grade Reading Assessment, (EGRA) a brief oral reading assessment that tests these skills, was used to measure program impacts on literacy (RTI International, 2007). EGRA is comprised of multiple sub-tests that focus on the five main reading skill areas outlined above. Within each of these five areas, there are multiple sub-tests that can be selected for inclusion, based on local needs and the goals of the assessment system. No clear benchmarks for the EGRA tool have been established in Uganda. That is, the EGRA tool provides a snapshot of early literacy skills but does not provide guidelines for interpreting which children can be considered “readers” or what level of performance should be expected on each sub-test. A notable component of SHRP is its transitional bilingual design. That is, literacy instruction begins in one of four mother tongue languages, with English introduced as a subject area nearly simultaneously (within 4-8 weeks after mother tongue instruction has begun). The language of instruction will then increasingly transition from mother tongue to English over the course of four years. Because of this transitional bilingual design, the impact evaluation necessarily requires a heteroglossic5 approach to assessment. Therefore, early literacy skills are assessed in mother tongue and English. All the EGRA tools used for this project were created and pre-tested by RTI. The subtasks selected to be included were modified sometimes between clusters and over time. In particular, for cluster 3, the IP decided to modify the assessment and most subtasks in English were dropped, leaving letter sound knowledge and vocabulary as the only subtasks in English. Non￾word decoding was also dropped in the mother-tongue. Knowledge of print orientation and letter writing were introduced for October 2015 data collection but eliminated for the endline. Cluster 1 and cluster 2 discontinued some subtasks for the endline, such as segmenting in the local language and segmenting in English, respectively. The detail of every sub-task included in the endline tool for each cluster can be found in Table 2. 4 These five skills are not meant to be all inclusive; however, considerable empirical research has been conducted in these skill areas that has indicated they are important predictors of reading. 5 A “heteroglossic” approach conceptualizes literacy learning in both mother tongue and English as interconnected, co-existing, and mutually reinforcing. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 20 Table 2: Early Grade Literacy Skills, EGRA Subtasks at Endline Early Literacy Skill Sub-test Measurement C1 C2 C3 Englis h Mother￾Tongue Englis h Mother￾Tongue English Mother￾Tongue Alphabetic Knowledge Letter Sound Knowledge No. of letter sounds correctly identified in 60 seconds X X X X X X Phonemic Awareness Segmenting No. of words correctly segmented out of 10 words X -- -- X -- X Phonics/Alpha￾betic Principles Nonword decoding No. of nonwords correctly decoded in 60 seconds X X X X -- -- Fluency Oral passage reading No. of words in a passage read fluently (with accuracy) in 60 seconds X X X X -- X Reading Comprehension Oral recall No. of questions about a reading passage (read by student) answered correctly X X X X -- X Listening Comprehension Oral recall No. of questions about a passage read aloud (by facilitator) answered correctly -- X -- X -- X Receptive Vocabulary Oral identificatio n of common objects No. of common objects correctly identified X -- X -- X -- In addition to the EGRA, RTI, with feedback from NORC, designed additional tools. The following data collection instruments were developed and administered:  Learner Context interview: to determine learner attendance to pre-school, socio￾economic status and home literacy environment  Teacher interview: to determine language of instruction in the school, teacher qualifications (including attendance to SHRP trainings) and the amount of support received from head teachers and CCTs  Head Teacher interview: to obtain school enrollment information and determine participation in SHRP Training and amount of support provided to teachers  School inventory: to determine quality of school environment (access to electricity, water, functioning toilets/latrines, availability of a school library) SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 21 A.5 COLLECTION OF READING ASSESSMENT DATA The first EGRA data collection was conducted for Cluster 1 at the beginning of the 2013 school year by the IP and its local partner, the Center for Social Research (CSR). Follow-up data collections for Cluster 1 took place every year in October, resulting in 5 rounds of data collection by October 2016. Table 3 shows the data collection details for the 3 clusters. Over time, Clusters 2 and 3 were also surveyed. Cluster 2 was initially assessed in February 2014 while the baseline for Cluster 3 took place in February 2015. Table 3: Data Collection Rounds by Cluster Cluster 1 Cluster 2 Cluster 3 February 2013 Baseline (P1) October 2013 Follow-Up1 (P1) February 2014 Baseline (P1) October 2014 Follow-Up 2 (P2) Follow-Up1 (P1) February 2015 Baseline (P1) October 2015 Follow Up-3 (P3) Follow-Up 2 (P2) Follow-Up 1 (P1) October 2016 Follow Up-4 (P4) Follow-Up 3 (P3) Follow-Up 2 (P2) Data collection was conducted using tablets in Tangerine software and consisted of EGRAs in English and 12 local languages (each student was assessed in his/her local language and in English), followed by a learner context interview. At each school, field teams also administered a Teacher and Head Teacher interview and a school inventory. Details of the sample size calculation, the sample plan for each cluster and the analytical samples included in the impact analyses can be found in Annex A. RTI and NORC had ethics clearance from USA and Ugandan IRBs to conduct this research and all data collections associated with it. A.6 ASSESSOR TRAINING In preparation for each round of data collection, RTI conducted a training program which usually lasted 7 days. Assessors were selected based on inter-rater reliability tests (IRR) that were given throughout the week as well as interpersonal and leadership skills. Technical training, which was undertaken by School Health and Reading Program staff (RTI and CSR), included hands-on practice, where assessors spent one day in a school administering the tool to learners and teachers. Trainees were first trained to administer the tools on paper and then introduced to electronic data collection on Nexus tablets so that they could be prepared for both circumstances. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 22 Four Data Quality Assurance (DQA) Officers, who also acted as assessor trainers, were given an extra day of training on the classroom observation instrument. This included going to a school, observing a class, and comparing findings among one another. In its capacity as the data quality assurance team, NORC and its subcontractor School-to￾School International attended all enumerator trainings. During these trainings, we provided real time feedback intended to improve the quality of training activities. NORC also produced reports describing positive and negative aspects of training activities, as well as laying out recommendations for improving future trainings (for details please see Performance & Impact Evaluation (P&IE) Semi-Annual Reports, April 2014, April 2015, April 2016, and May 2017). A.7 FOCUS GROUP DISCUSSION METHODOLOGY NORC conducted a limited but in-depth qualitative study to explore specific topics of interest that might shed light on the findings of the impact evaluation. These topics included teacher trainings, teacher support supervision, community mobilization, and absenteeism among learners and teachers. The study began with a review of both quantitative and qualitative data on project implementation collected throughout the evaluation period (continuous evaluation data) collected data, followed by a set of six Focus Group Discussions (FGDs) with teachers from schools that received SHRP treatment in three different districts. The districts, purposefully selected to represent the three different SHRP clusters, different geographical regions, and different languages, were Kiruhura, Pader, and Kamuli. Two Coordinating Centers (CCs) were randomly selected from each district, and where possible four schools were then randomly selected from each CC. Finally, two teachers were selected purposively from each school, to ensure both male and female teachers who participated in the SHRP training and are implementing the interventions. Altogether, the six FGDs (two per district) included 40 teachers, 17 female and 23 male. The FGDs were conducted using prepared guides, focusing on the four topics of interest, as well as teacher experiences in SHRP training workshops and the classroom; specifically, the positive and negative aspects of the training, as it pertains to implementing a new curriculum and teaching methods in the classroom, barriers and successes to uptake in the classroom, and parents’ involvement and awareness about the importance of reading. More detailed information about the methodology can be found in the full report on the FGDs, which is included in Annex F. B. BALANCE AT BASELINE In order to explore comparability of pre-treatment characteristics of treatment and control groups, we conducted mean equality tests to assess the sample balance at baseline. We compared socio-demographic characteristics of the students and pre-treatment EGRA scores between treatment and control schools in treatment district. Because comparisons between languages cannot be made given that no benchmarking and psychometric analysis has been performed to assess comparability between languages, mean equality tests of local language literacy scores are conducted within language subgroup. On the other hand, tests for English literacy scores are conducted on the entire sample. Overall, the samples at baseline for each cluster are well-balanced between treatment and control schools indicating that the randomization generated equivalent groups in terms of observable characteristics. Annex B shows that for each cluster the treatment and control SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 23 groups are very similar in terms of demographics and other socio-economic status (SES) characteristics at baseline. Almost all the variables included in the means equality tests are not statistically significantly different between treatment and control groups. We also compare the basic literacy subtasks scores in English and in local language. We focus on the 3 most basic reading subtasks given that at baseline most of the students are unable to complete any of the more complex items and, therefore, those scores are noisy and uninformative. There are some small differences between treatment and control groups in some of the scores for some languages. It is not unusual that among so many variables we would find that a few of the tests indicate a difference. The sample is overall well balanced and shows a proper randomization exercise. Although socio-demographic variables and test-scores in general show no statistically significant differences between the treatment and control groups, we will include them as covariates in our analysis of the impact of SHRP on the outcomes of interest to add precision to our impact estimates. C. IMPACT ANALYSIS: QUANTITATIVE FINDINGS This section summarizes the results of the impact analysis for each of the clusters. We begin by showing the effects of the intervention on Cluster 1 students who have been exposed to the SHRP Result 1 interventions for the longest period of time (from P1 to the end of P4). We then present impact estimations for Cluster 2 students who received 3 years of intervention, and end with impact estimations for Cluster 3 students who were exposed to two years of intervention. Our analysis is conducted for each of the outcome scores on subtasks conducted in the local language for each of the four local languages included in each cluster. On the other hand, for outcome scores on English subtasks, our analysis is applied to the entire sample (pooled sample from all four language subgroups) in the cluster. It is important to keep in mind that this evaluation does not allow for comparison of progress and impact between languages as the EGRA tools were developed independently for each local language and no psychometric analysis has been conducted to determine whether EGRA scores in one language can be compared to EGRA scores in another language. Therefore, while we can determine whether SHRP has had an impact for a particular language subgroup vs. another language subgroup, we cannot comment on the relative magnitude of these impacts between language subgroups. We study the effect of the school-level intervention following the model in equation (1) in Section B1, using the treatment and control school sample. We present findings for scores in local language within each language subgroup and in English for the pooled sample. In Tables 4 to 6 below, we report the average treatment effect on each outcome of interest6 and for each language subgroup in each of the three clusters. The regression models include individual controls (sex and age of the student, a dummy for whether age is missing, household asset index, dummies for living with both parents, having someone read to the student at home, and language spoken at home), and school fixed effects7 . For the regressions on English scores using the pooled sample, we include dummies for local language subgroups. All standard errors 6 We use October 2015 outcomes that have been equated to the baseline measures whenever available. 7 The results are very similar with and without school fixed effects. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 24 are robust standard errors and allow for correlation in the unobservables between learners in the same class. All regressions are Ordinary Least Squares (OLS) regressions. C.1 CLUSTER 1 SCHOOLS Each cell in Table 4 shows the average effect of the treatment (SHRP intervention) on an EGRA subtask for each language in cluster 1 as estimated using equation (1). Table 4: Cluster 1 Estimated SHRP School Level Effects after 4 years of program (1) (2) (3) (4) (5) VARIABLES English Luganda Leblango Ateso Runyankore/ Rukiga Letter Sound Score 4.225*** 6.997*** 6.270*** 2.111 1.671 (0.871) (1.864) (1.395) (1.439) (1.675) % Letter Sound Zero Score -0.156*** -0.109 -0.328*** -0.071 -0.089 (0.040) (0.075) (0.070) (0.057) (0.094) Segmenting -0.280 (0.178) Non-words 2.019+ 2.609+ 0.406 3.768** 4.041+ (1.047) (1.514) (0.823) (1.163) (1.986) Fluency 5.038+ 4.495+ 0.444 6.250*** 6.327* (2.950) (2.366) (1.953) (1.534) (2.607) Reading Comprehension 0.180 0.435+ 0.094 0.455** 0.591* (0.136) (0.254) (0.201) (0.123) (0.248) Listening Comprehension 0.227 -0.144 0.140 0.070 (0.172) (0.136) (0.097) (0.128) English Receptive Vocabulary 0.147 (0.269) Robust standard errors in parentheses. *** p<0.001, ** p<0.01, * p<0.05, + p<0.1. All regressions include school fixed effects, individual controls and a constant term. Individual controls include age and sex of learner, dummy for age missing, score on household assets index, dummies for student living with both parents, someone at home reading to the student and language spoken at home. Regressions for English language include control for local language The first row in Table 4 shows that, after four years, the intervention had a positive impact on letter sound scores in Luganda, Leblango and English. The estimates for Runyankore/Rukiga and Ateso are not statistically significantly different from zero. The largest effect is for the Luganda subgroup where the treatment students, on average, were able to recognize almost 7 letters sounds –an effect size of 0.53 pooled standard deviation- more than control students due to the program. The program effect among the Leblango subgroup is also positive; treatment students were able to correctly identify over 6 additional letter sounds (0.50 standard deviation) compared to control students as a result of the program. The impact of the intervention on the English version of this subtask is approximately 4.2 letter sounds; that is, on average, treatment students were able to recognize 4.2 more English letter sounds (0.45 standard deviation) than their counterparts in control schools. Most of this effect comes from a differential impact among the students in the Luganda sub-group. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 25 Although the raw data cannot reflect the complexity of the econometric analysis shown above, they are useful for visualizing the level of knowledge of the students in the different subgroups at baseline and after four years of intervention. Figure 2 shows the average number of correctly identified letter sounds by treatment and control groups by language at the beginning of P1 (baseline) and at the end of P4 (endline). Figure 2: Cluster 1, Letter Sound Scores at baseline and after four years of program, by language Error bars indicate 95% confidence interval As we show in Table 4, the program helped students achieve higher scores in Leblango and Luganda. However, it is important to note that the absolute levels of competence remain modest. The Luganda subgroup scores the highest with 18.1 letter sounds identified correctly, one more letter sound than the previous year. Figure 3, shows the difference in letter sounds correctly identified between treatment and control learners at the end of each grade since the start of the program when learners were just entering P1. While the effect for the Luganda speaking learners is similar to what we found in previous years (end of P2 and end of P3), the effects for Leblango, in particular, but also in English have increased between end of P3 and end of P4. In Leblango, for example, the effect went from 2.5 letter sounds at the end of P3 to 6.3 at the end of P4. Although the effect size of the improvement during the last year is large in Leblango, the absolute scores achieved are low, at 10.52 letters, for students who have completed P4. In Luganda, the difference between treatment and control learners has been around the same values since the end of P2, when it was 6.7. It dropped to 5 letter sounds at the end of P3 and returned to around 7 at the end of P4. 10.52 .44 10.52 .44 10.52 .44 3.39 10.51 3.39 .69 4.49 .69 4.49 .69 .44 10.52 .44 .69 4.49 .69 4.49 .69 4.49 .44 10.52 .44 .69 4.49 .69 4.49 .69 4.49 10.51 3.39 10.51 3.08 11.87 3.08 10.51 3.39 3.08 11.87 3.08 11.87 3.39 10.51 3.39 18.07 1.9 18.07 1.9 18.07 1.85 1.9 11.04 1.85 11.04 1.85 18.07 1.9 18.07 1.9 18.07 1.9 11.04 1.85 11.04 1.85 18.07 1.9 18.07 1.9 11.04 1.85 1.9 18.07 1.9 18.07 1.9 18.07 1.85 11.04 3.39 10.51 3.39 11.87 3.08 11.87 3.08 11.87 3.39 10.51 3.39 11.87 3.08 11.87 3.08 11.87 3.08 10.51 3.39 10.51 3.39 10.51 3.39 3.08 11.87 3.08 3.39 10.51 3.39 11.87 3.08 10.52 .44 10.52 .44 10.52 .44 .69 4.49 .69 4.49 .69 4.49 .69 4.49 .69 4.49 .69 4.49 .69 4.49 .69 1.73 15.36 1.73 3.39 10.51 3.39 10.52 10.51 .44 10.52 .69 4.49 15.36 1.73 15.36 1.73 15.36 1.73 15.36 1.73 15.36 1.73 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 1.73 15.36 1.73 15.36 1.84 1.73 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 13.52 1.84 11.87 3.08 11.87 1.9 18.07 1.85 11.04 1.85 11.04 18.07 1.9 3.08 11.87 3.08 10.52 .44 10.52 .44 10.52 1.73 15.36 1.73 15.36 1.73 15.36 1.73 15.36 1.73 13.52 1.84 13.52 1.84 13.52 1.84 0 5 10 15 20 Number of correct letter sounds Control Treatment Control Treatment Control Treatment Control Treatment Luganda Leblango Ateso RR Baseline Endline Letter Sound Identification SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 26 For the two languages where we find no significant effect of the program in letter sounds - Runyankore/Rukiga and Ateso- we observe that, at the end of P4 the difference between treatment and control learners remain very similar to the previous year’s, around 1.8 and around 2 respectively. Learners in the treatment group were able to correctly recognize, on average, 15.4 letter sounds per minute in Ateso and 11.9 in the case of Runyankore/Rukiga learners. Figure 3: Cluster 1, SHRP estimated effects on letter sound identification since baseline, by language *** p<0.001, ** p<0.01, * p<0.05, + p<0.1Estimates from OLS regressions including school fixed effects, individual controls and a constant term. Individual controls include age and sex of learner, dummy for age missing, score on household assets index, dummies for student living with both parents, someone at home reading to the student and language spoken at home. Regressions for English language include control for local language The second row in Table 4 shows the average treatment effect on the percentage of students who could not correctly identify any letters in the letter sound knowledge subtask (percentage of zero scores). At the end of P4, we would expect to find no zero scores anymore; however, there are still some learners that completely fail the subtask (See Annex C for complete score distribution for all subtaks and languages). In general, the treatment learners perform better than the controls but the significant reductions due to the program are only present in Leblango and in English, where the effect size is 0.64 and 0.32 standard deviations respectively. The lack of a clear impact in the reduction of zero scores in this task while we see some positive impact in the number of correctly identified letter sounds suggests that the improvements do not occurred among the worst performers. The word segmenting task at endline was only assessed in English and we find no impact of the treatment. In terms of the impact of the intervention on higher level literacy skills, namely non-word decoding, oral reading fluency, reading comprehension and English receptive vocabulary, we find a few positive and statistically significant effects. We estimate a positive and significant effect on non-word decoding for the Ateso group of a 3.8 words (0.35 standard deviations). The 0.0 0.2 0.9 2.0 1.7 2.5* 6.7** 5.0+ 7.0** 1.1 2.2** 2.5* 6.3** 0.3 1.7 1.8 2.1 1.3 2.3** 2.4* 4.2** 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 Baseline At P1 At P2 At P3 At P4 Difference in Letter Sound Identification Runyankore/Rukiga Luganda Lango Ateso English SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 27 effect in English is 2 words (0.17 standard deviations), in Runyankore/Rukiga is 4 words (0.33 standard deviations), and in Luganda 2.6 words (0.21 standard deviations). These estimates are borderline significant (only significant at 10%) while in Leblango the difference is not statistically significant. The effects of the program on oral reading fluency are positive for all languages with the exception of Leblango. Ateso and Runyankore/Rukiga treatment learners read around 6 words per minute more than learners in the control group due to SHRP; these are effects of around 0.44 standard deviations. In Luganda the advantage is of 4.5 words and in English it is 5 words and these effects sizes are 0.2 and 0.28 standard deviations respectively. It is interesting to note that although Leblango learners show a positive impact of SHRP on alphabetic skills such as letter sound identification, that impact has not been translated into higher level literacy skills such as non-word decoding and oral reading fluency. Figure 4 shows the average number of words that students in each sub-group were able to read correctly in a reading passage in 60 seconds at baseline and after four years of intervention. Although all language groups -treatment and control- have made some improvements from the previous year, the absolute number of words is low for fourth grades, particularly in the case of Ateso learners. Figure 4: Cluster 1, Oral reading fluency at baseline and after four years of program, by language. Error bars indicate 95% confidence interval As we noted in previous reports, Runyankore/Rukiga, Luganda, and now also Leblango learners seem to be able to read on average more words from the passage in one minute than the 14.86 .04 14.86 .04 14.86 .04 .3 20.26 .03 .3 14.22 .03 14.22 .03 .04 14.86 .03 .04 14.22 .03 14.22 .03 14.22 .04 14.86 .03 .04 14.22 .03 14.22 .03 14.22 20.26 .3 20.26 .53 26.63 .53 20.26 .3 .53 26.63 .53 26.63 .3 20.26 .3 20.03 .17 20.03 .17 20.03 .14 .17 15.82 .14 15.82 .14 20.03 .17 20.03 .17 20.03 .17 15.82 .14 15.82 .14 20.03 .17 20.03 .17 15.82 .14 .17 20.03 .17 20.03 .17 20.03 .14 15.82 .3 20.26 .3 26.63 .53 26.63 .53 26.63 .3 20.26 .3 26.63 .53 26.63 .53 26.63 .53 20.26 .3 20.26 .3 20.26 .3 .53 26.63 .3 .53 20.26 .3 26.63 .53 14.86 .04 14.86 .04 14.86 .03 .04 14.22 .03 14.22 .03 14.22 .03 14.22 .03 14.22 .03 14.22 .03 14.22 .03 .01 12.76 .01 .3 20.26 .3 20.26 14.86 .04 14.86 .03 14.22 12.76 .01 12.76 .01 12.76 .01 12.76 .01 12.76 .01 6.18 .05 6.18 .05 6.18 .05 6.18 .05 6.18 .05 6.18 .05 .01 12.76 .01 12.76 .05 .01 6.18 .05 6.18 .05 6.18 .05 6.18 .05 6.18 .05 6.18 .05 6.18 .05 6.18 .05 26.63 .53 26.63 .17 20.03 .14 15.82 .14 15.82 20.03 .17 .53 26.63 .53 14.86 .04 14.86 .04 14.86 .01 12.76 .01 12.76 .01 12.76 .01 12.76 .01 6.18 .05 6.18 .05 6.18 .05 0 10 20 30 Fluency - Number of correct words Control Treatment Control Treatment Control Treatment Control Treatment Luganda Leblango Ateso RR Baseline Endline Correct Words SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 28 number of letter sounds that they are able to identify correctly in the same amount of time. For example, the treatment group in Runyankore/Rukiga reads an average of 26.6 words while they only identify an average of 11.6 letter sounds correctly. In theory, skills related to alphabetics (such as letter sound knowledge and phonemic awareness) are foundational for decoding words and reading8 ; one therefore would expect to first see a better command in these basic skills. The reason behind this phenomenon is not clear. It could be the case that learners know the sounds of the letters but are less precise in their pronunciation than the assessors’ standards of correctness. Alternatively, it could be that they are learning to read despite not mastering the phonics technique. We found some statistically significant impacts in reading comprehension. The Luganda treatment group scored on average 0.43 questions -0.24 standard deviations- more than the control group and this effect is statistically significant at 10%. In Ateso and Runyankore/Rukiga, the effects are 0.45 and 0.59 respectively (0.4 standard deviations in both cases) and statistically significant at conventional levels. The Leblango subgroup shows zero effect and there are no effects in English reading comprehension either. Finally, we do not find differences between treatment and control groups in the listening comprehension or in the English vocabulary subtasks. C.2 CLUSTER 2 SCHOOLS In Table 5 we present the estimated effects of SHRP on each of the EGRA subtasks for each of the languages in Cluster 2. Table 5: Cluster 2 Estimated SHRP School Level Effects after three years of program (1) (2) (3) (4) (5) VARIABLES English Runyoro/ Rutooro Acoli Lugbarati Lumasaaba Letter Sound Score 2.368* 2.607 2.626 -1.732 5.611** (0.966) (2.079) (1.680) (1.530) (1.997) % Letter Sound Zero Score -0.080+ -0.018 -0.069 0.115* -0.110** (0.041) (0.069) (0.071) (0.042) (0.039) Segmenting 0.423 0.496 0.240 0.491 (0.464) (0.387) (0.626) (0.415) Non-words 1.771** 5.200*** 0.977+ 0.397 1.620+ (0.609) (1.171) (0.566) (0.982) (0.848) Fluency 3.367** 7.247*** 2.439+ 0.695 2.441* (1.139) (1.698) (1.353) (1.463) (1.041) Reading Comprehension 0.077* 0.768*** 0.110+ 0.073 0.149 (0.032) (0.200) (0.060) (0.138) (0.090) Listening Comprehension 0.072 -0.335* 0.108 0.113 (0.132) (0.159) (0.146) (0.121) 8 Some longitudinal studies have shown that phonemic awareness is highly predictive of decoding and that phonemic awareness is necessary, although not sufficient, for learning how to read. International Reading Association, 1998. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 29 English Receptive Vocabulary 0.134 (0.233) Robust standard errors in parentheses. *** p<0.001, ** p<0.01, * p<0.05, + p<0.1. All regressions include school fixed effects, individual controls and a constant term. Individual controls include age and sex of learner, dummy for age missing, score on household assets index, dummies for student living with both parents, someone at home reading to the student and language spoken at home. Regressions for English language include control for local language. We find that, after the third year of implementation, the intervention started to have some effects in letter sound scores in Lumasaaba and in English. In Lumasaaba, treatment learners are able to identify 5.6 more letter sounds (0.65 standard deviations) than their counterparts in the control group due to the program. In English, the effect of the program is 2.4 letter sounds (0.26 standard deviations). The effects are not estimated with precision for Runyoro/Rutooro, Acoli or Lugbarati and, therefore, cannot be distinguished from zero. Figure 5 shows the average number of letter sounds identified correctly by treatment and control learners at baseline and in October 2016 by each local language in cluster 2. As explained above, the difference in knowledge change between the treatment and control is statistically significant in Lumasaaba. Runyoro-Rutooro is the subgroup that shows the highest average level of letter sound knowledge, around 15 correct sounds among treated learners. In general, we find that the absolute performance scores of students for this task are low for all languages and particularly for Acoli. Figure 5: Cluster 2, Letter Sound Scores at baseline and after two years of intervention, by language Error bars indicate 95% confidence interval 2.37 14.24 11.59 13.36 9.04 .34 12.33 .31 15.23 2.92 .9 14.24 13.36 6.28 9.04 .31 12.33 .9 2.92 14.24 1.42 9.04 2.37 12.33 11.59 6.28 2.37 1.42 2.92 9.04 15.23 .34 1.42 6.28 2.37 2.92 9.04 2.49 11.59 1.8 .9 1.42 9.04 13.36 12.33 15.23 5.7 2.92 6.28 2.37 13.36 .34 12.33 14.24 15.23 6.28 .9 2.49 14.24 11.59 1.42 9.04 6.28 15.23 13.36 .34 .31 12.33 1.8 9.04 6.28 15.23 2.37 13.36 9.04 .34 15.23 1.8 1.42 2.37 2.49 11.59 2.92 6.28 13.36 12.33 5.7 9.04 6.28 2.37 2.92 .31 9.04 13.36 1.42 2.92 14.24 1.8 2.49 11.59 5.7 1.8 14.24 15.23 6.28 2.37 1.42 2.92 9.04 14.24 5.7 6.28 12.33 1.8 2.49 .31 13.36 9.04 .34 11.59 2.37 14.24 .31 1.42 1.8 6.28 2.49 2.92 5.7 .9 9.04 13.36 15.23 .31 2.92 5.7 2.49 6.28 .31 13.36 1.8 12.33 .9 15.23 2.37 13.36 6.28 .9 11.59 .31 14.24 15.23 2.37 1.42 6.28 9.04 13.36 5.7 12.33 .34 11.59 .9 9.04 12.33 2.37 2.49 13.36 5.7 .9 12.33 14.24 6.28 1.8 2.49 11.59 2.92 .31 15.23 6.28 14.24 2.49 9.04 2.92 .9 12.33 1.8 13.36 .9 .31 9.04 2.92 5.7 2.37 15.23 11.59 1.42 9.04 13.36 12.33 14.24 2.37 1.42 15.23 .9 2.49 .34 .31 9.04 2.92 1.8 6.28 11.59 1.42 13.36 2.49 5.7 .31 1.8 9.04 6.28 2.37 14.24 13.36 11.59 15.23 5.7 6.28 .34 11.59 14.24 .9 15.23 12.33 11.59 13.36 .9 2.37 5.7 1.8 13.36 15.23 11.59 12.33 .9 2.37 .34 15.23 13.36 5.7 .9 6.28 12.33 9.04 .34 5.7 .9 2.37 11.59 14.24 1.8 9.04 6.28 .34 .31 1.42 14.24 9.04 2.92 5.7 .34 13.36 12.33 2.49 11.59 2.92 .31 9.04 5.7 14.24 1.8 15.23 6.28 .31 2.49 15.23 .34 11.59 .9 5.7 14.24 12.33 .31 15.23 6.28 1.8 14.24 1.42 .9 13.36 5.7 2.49 1.42 2.37 9.04 13.36 12.33 15.23 6.28 2.37 1.8 5.7 9.04 11.59 2.37 .9 .34 .31 13.36 11.59 6.28 12.33 14.24 1.42 .34 6.28 2.92 5.7 15.23 2.37 13.36 11.59 .31 2.92 1.42 2.37 12.33 9.04 .34 1.8 15.23 2.37 1.8 2.92 2.49 1.42 6.28 9.04 5.7 .31 1.42 1.8 5.7 15.23 14.24 .31 9.04 12.33 .9 .34 2.49 6.28 1.8 13.36 2.92 11.59 6.28 12.33 1.42 2.49 9.04 .34 11.59 2.37 12.33 1.8 15.23 1.42 13.36 2.37 .9 2.92 5.7 12.33 6.28 9.04 .31 .9 11.59 2.92 15.23 6.28 2.37 .9 .34 9.04 2.92 1.8 14.24 2.37 .9 9.04 15.23 .34 13.36 2.92 11.59 2.49 .34 15.23 9.04 11.59 6.28 2.49 2.92 12.33 2.37 .31 15.23 12.33 1.8 2.37 .31 13.36 11.59 14.24 1.8 .9 2.49 5.7 6.28 .34 15.23 13.36 12.33 1.8 .31 14.24 9.04 11.59 6.28 2.49 .34 15.23 1.42 6.28 2.37 12.33 11.59 2.49 2.92 5.7 .9 11.59 2.92 5.7 2.37 2.49 .31 12.33 11.59 5.7 15.23 13.36 .34 12.33 2.49 1.42 9.04 5.7 13.36 .31 2.49 11.59 15.23 5.7 1.8 .31 9.04 .34 2.37 14.24 2.49 1.42 6.28 13.36 .9 .34 9.04 2.37 2.49 12.33 .9 13.36 .34 2.37 .31 11.59 12.33 1.42 6.28 2.49 11.59 .34 5.7 2.37 1.8 1.42 12.33 .31 2.49 11.59 14.24 .34 2.37 1.42 15.23 2.49 12.33 5.7 .9 11.59 6.28 .31 2.49 .34 5.7 12.33 11.59 .31 5.7 15.23 1.8 2.49 13.36 14.24 .9 .31 14.24 15.23 .9 9.04 .34 6.28 2.37 1.8 .9 2.92 2.49 6.28 .31 12.33 .34 9.04 13.36 6.28 5.7 15.23 .34 2.92 2.37 12.33 5.7 .31 13.36 .34 12.33 9.04 15.23 6.28 11.59 .34 2.49 13.36 .9 1.42 .31 15.23 2.49 13.36 .9 .31 6.28 5.7 1.42 2.37 14.24 15.23 .31 6.28 13.36 12.33 9.04 1.42 2.49 6.28 11.59 15.23 13.36 9.04 1.8 .9 1.42 13.36 14.24 .31 5.7 13.36 14.24 15.23 .34 .31 2.92 .9 2.49 13.36 1.42 12.33 2.49 14.24 6.28 2.92 15.23 .31 1.42 11.59 2.37 14.24 2.49 1.42 11.59 6.28 .31 1.8 14.24 11.59 13.36 12.33 .9 9.04 11.59 .31 2.49 6.28 12.33 13.36 .9 .34 14.24 9.04 6.28 .34 .31 11.59 2.37 12.33 15.23 6.28 .34 1.8 14.24 .9 12.33 15.23 2.37 12.33 6.28 13.36 2.37 1.42 1.8 2.92 6.28 5.7 13.36 9.04 1.42 .31 2.49 15.23 6.28 2.92 13.36 14.24 11.59 .34 2.92 6.28 15.23 .34 2.92 13.36 2.37 9.04 6.28 2.92 5.7 11.59 15.23 9.04 14.24 1.42 .34 .31 5.7 13.36 1.8 14.24 11.59 13.36 9.04 14.24 .34 .31 11.59 15.23 .34 .31 6.28 2.92 12.33 5.7 1.42 11.59 .31 2.49 15.23 6.28 5.7 1.42 2.92 15.23 13.36 6.28 5.7 12.33 1.42 2.49 1.8 9.04 .34 .31 15.23 11.59 6.28 12.33 1.8 13.36 5.7 .34 .9 11.59 14.24 1.8 5.7 15.23 2.92 12.33 9.04 13.36 6.28 2.49 15.23 14.24 2.37 5.7 12.33 .9 11.59 15.23 9.04 5.7 .31 6.28 2.92 13.36 12.33 1.42 .9 6.28 2.37 .34 11.59 15.23 9.04 13.36 .34 6.28 .31 5.7 14.24 9.04 6.28 15.23 .34 2.92 2.49 11.59 6.28 .31 13.36 .34 14.24 5.7 9.04 .9 1.8 .31 1.42 2.37 14.24 12.33 2.49 11.59 15.23 6.28 14.24 .34 2.37 1.42 12.33 5.7 14.24 .34 11.59 12.33 1.8 .34 15.23 2.92 1.8 2.37 9.04 1.42 15.23 11.59 2.37 1.8 2.49 .34 6.28 2.37 5.7 1.8 11.59 12.33 5.7 .9 14.24 15.23 .34 2.92 5.7 .9 14.24 11.59 9.04 12.33 13.36 .31 1.8 14.24 2.49 9.04 1.42 2.37 13.36 9.04 .31 6.28 5.7 13.36 11.59 14.24 15.23 6.28 2.37 .31 13.36 1.42 1.8 5.7 15.23 2.37 13.36 12.33 .31 1.8 2.92 13.36 2.37 .9 6.28 9.04 .31 15.23 2.49 14.24 6.28 .9 12.33 .34 13.36 5.7 2.37 2.49 1.8 15.23 1.42 6.28 2.92 .31 11.59 .9 9.04 14.24 2.37 13.36 12.33 6.28 2.92 .31 13.36 14.24 11.59 .34 5.7 1.42 .9 9.04 .31 2.92 6.28 11.59 12.33 .34 .31 1.42 13.36 .9 9.04 14.24 11.59 1.8 .31 2.92 5.7 9.04 14.24 13.36 11.59 12.33 2.49 .34 15.23 14.24 1.42 5.7 .34 1.8 6.28 9.04 .9 2.49 1.8 12.33 .31 13.36 1.42 6.28 .9 15.23 14.24 9.04 6.28 11.59 2.49 1.8 5.7 .31 12.33 9.04 .34 1.42 13.36 11.59 12.33 2.92 1.8 9.04 .31 12.33 2.49 14.24 2.37 1.42 1.8 .9 .31 2.37 11.59 .34 2.92 .31 6.28 5.7 15.23 14.24 1.8 11.59 .34 .31 1.42 5.7 9.04 14.24 15.23 12.33 2.49 11.59 9.04 6.28 .9 15.23 11.59 13.36 6.28 9.04 2.49 11.59 .34 12.33 14.24 9.04 15.23 12.33 .34 13.36 6.28 1.42 2.37 .31 2.92 15.23 2.49 1.8 9.04 .34 11.59 .31 12.33 6.28 2.92 13.36 11.59 2.37 5.7 15.23 11.59 2.37 13.36 9.04 .34 .9 5.7 1.42 2.37 .31 6.28 9.04 14.24 2.37 1.42 6.28 2.49 15.23 .31 9.04 13.36 5.7 2.92 6.28 15.23 9.04 .34 11.59 2.37 13.36 6.28 .34 2.49 12.33 2.92 5.7 13.36 .31 11.59 9.04 2.37 15.23 .34 1.42 9.04 2.37 5.7 11.59 2.92 2.49 .31 6.28 2.37 13.36 15.23 12.33 1.8 2.49 1.42 2.37 13.36 2.49 14.24 .31 5.7 2.92 15.23 1.42 11.59 .9 .31 2.49 9.04 13.36 .34 2.37 15.23 2.92 13.36 .31 2.49 14.24 5.7 2.37 2.92 1.8 12.33 9.04 .31 2.49 15.23 2.92 13.36 6.28 15.23 2.92 9.04 1.42 15.23 1.8 2.92 11.59 2.49 14.24 12.33 2.92 .34 9.04 13.36 11.59 2.92 2.49 .34 .9 15.23 5.7 2.92 9.04 13.36 .34 .31 1.8 6.28 14.24 12.33 5.7 1.8 14.24 6.28 .34 15.23 .9 2.37 13.36 1.42 2.92 6.28 14.24 .31 13.36 2.49 5.7 .9 11.59 1.8 2.37 .31 2.49 5.7 13.36 14.24 15.23 11.59 .34 12.33 6.28 1.8 2.37 15.23 2.49 11.59 2.92 .34 .31 9.04 6.28 .9 1.8 5.7 14.24 9.04 1.42 .34 1.8 13.36 2.92 11.59 .31 9.04 15.23 12.33 .31 1.42 2.37 .9 9.04 15.23 14.24 .31 2.37 13.36 2.92 11.59 1.42 .9 5.7 2.49 .31 1.8 6.28 .9 11.59 14.24 13.36 2.92 .31 1.42 2.37 .9 14.24 6.28 .31 11.59 1.8 .9 2.49 .31 11.59 14.24 2.37 12.33 1.42 2.92 2.37 .9 11.59 2.49 14.24 12.33 .31 1.42 1.8 2.92 9.04 2.37 2.49 15.23 1.8 13.36 6.28 .9 14.24 13.36 12.33 6.28 2.37 .34 1.8 11.59 13.36 5.7 9.04 2.37 1.42 .31 5.7 2.92 9.04 12.33 .34 6.28 1.8 .9 14.24 5.7 11.59 9.04 2.49 1.8 6.28 2.37 .34 .9 2.92 5.7 9.04 .31 .9 2.49 1.42 12.33 5.7 .31 13.36 .9 14.24 12.33 15.23 .34 2.49 .9 .31 13.36 .34 5.7 12.33 2.92 14.24 1.42 .34 6.28 13.36 2.92 1.8 5.7 .9 13.36 2.37 2.49 1.42 2.92 .31 6.28 14.24 15.23 13.36 5.7 .31 1.42 9.04 .9 1.8 2.37 6.28 2.92 11.59 .9 2.37 12.33 5.7 .31 2.49 5.7 12.33 13.36 9.04 1.8 2.49 2.92 15.23 14.24 1.8 2.37 .9 12.33 1.42 2.92 1.8 12.33 15.23 11.59 2.37 .31 13.36 9.04 5.7 1.42 2.92 2.37 .9 12.33 2.49 9.04 5.7 11.59 .34 5.7 1.42 9.04 .9 13.36 2.49 15.23 12.33 .31 13.36 2.92 15.23 11.59 1.42 5.7 13.36 .9 2.49 5.7 1.8 2.92 1.42 15.23 9.04 5.7 2.37 2.49 .34 6.28 .31 2.37 12.33 15.23 6.28 2.37 13.36 9.04 5.7 .34 2.49 11.59 1.42 9.04 .31 12.33 2.92 11.59 .9 13.36 14.24 9.04 15.23 1.42 .9 13.36 2.92 12.33 6.28 11.59 1.42 2.49 .34 14.24 9.04 12.33 1.42 5.7 9.04 .34 .9 14.24 15.23 6.28 2.37 2.92 1.8 9.04 14.24 2.37 13.36 1.8 1.42 15.23 12.33 .9 9.04 11.59 .34 13.36 15.23 5.7 11.59 1.42 9.04 15.23 12.33 1.42 2.49 6.28 15.23 1.42 13.36 .34 .9 5.7 2.49 15.23 12.33 .9 1.42 5.7 2.49 .31 12.33 13.36 15.23 6.28 5.7 1.42 .31 2.49 .34 15.23 5.7 13.36 6.28 14.24 5.7 2.92 1.42 6.28 .34 14.24 12.33 1.8 15.23 13.36 .9 2.49 12.33 9.04 .34 14.24 11.59 6.28 5.7 2.92 .9 .34 14.24 1.8 15.23 11.59 2.49 13.36 5.7 15.23 6.28 14.24 .9 15.23 .9 9.04 12.33 1.8 15.23 12.33 11.59 .34 6.28 2.92 9.04 2.37 11.59 .34 1.8 1.42 13.36 2.37 12.33 5.7 11.59 14.24 2.49 15.23 .31 2.37 2.92 .34 13.36 15.23 11.59 2.37 1.42 12.33 6.28 .9 9.04 13.36 2.37 12.33 2.92 2.49 .31 2.37 2.92 11.59 15.23 .9 2.49 .31 2.92 9.04 .31 6.28 5.7 .31 13.36 5.7 6.28 15.23 11.59 .34 14.24 13.36 12.33 5.7 11.59 .34 .9 13.36 2.92 1.42 9.04 1.8 6.28 11.59 .9 2.92 1.8 15.23 .31 13.36 14.24 12.33 1.8 6.28 12.33 2.92 13.36 9.04 .34 15.23 12.33 14.24 11.59 2.92 1.8 9.04 5.7 1.42 .34 2.37 15.23 12.33 6.28 1.8 15.23 2.92 .34 14.24 12.33 .31 13.36 2.37 2.92 15.23 2.49 5.7 .9 14.24 13.36 12.33 2.37 5.7 15.23 13.36 9.04 2.49 2.92 .9 13.36 12.33 .31 6.28 1.8 9.04 15.23 12.33 1.42 .34 .31 2.92 2.49 1.8 .9 5.7 2.49 12.33 .31 6.28 13.36 .9 2.37 5.7 11.59 14.24 .31 12.33 6.28 2.37 11.59 1.8 2.49 5.7 15.23 .9 2.37 12.33 11.59 6.28 1.42 15.23 .34 5.7 14.24 1.42 2.37 .31 2.92 1.8 11.59 13.36 6.28 15.23 12.33 .34 14.24 2.37 2.92 15.23 12.33 9.04 11.59 2.92 .9 2.37 13.36 12.33 2.49 14.24 12.33 .34 5.7 .9 .31 14.24 12.33 9.04 5.7 2.37 15.23 2.49 1.42 6.28 14.24 .34 2.92 5.7 11.59 9.04 13.36 12.33 2.92 1.8 6.28 12.33 .9 2.37 2.92 9.04 1.8 5.7 2.37 2.92 13.36 14.24 .9 1.8 6.28 15.23 2.49 12.33 1.42 11.59 13.36 14.24 12.33 .34 .9 6.28 2.49 15.23 1.8 9.04 5.7 .34 14.24 .9 12.33 1.42 11.59 .31 1.8 2.92 13.36 2.37 6.28 1.42 .34 .9 5.7 6.28 12.33 13.36 .34 .31 1.42 15.23 2.92 5.7 6.28 13.36 .31 9.04 6.28 5.7 .9 12.33 11.59 1.42 14.24 2.49 15.23 .31 12.33 5.7 9.04 2.49 6.28 13.36 5.7 1.42 2.49 .34 6.28 14.24 11.59 1.42 2.37 12.33 9.04 .34 5.7 6.28 13.36 2.92 2.37 5.7 11.59 12.33 2.92 15.23 2.49 .31 2.37 9.04 2.92 .9 1.42 6.28 2.92 9.04 2.37 .9 2.49 .31 5.7 11.59 14.24 6.28 1.42 13.36 2.37 5.7 12.33 15.23 1.42 6.28 1.8 11.59 2.92 2.37 12.33 13.36 6.28 2.92 11.59 15.23 2.37 12.33 9.04 11.59 2.49 2.92 12.33 .31 6.28 9.04 2.92 2.49 13.36 1.42 11.59 .9 12.33 2.37 2.49 15.23 9.04 13.36 5.7 .34 2.37 .31 15.23 .9 9.04 2.92 14.24 15.23 12.33 .9 2.49 1.42 2.92 9.04 12.33 2.92 1.8 15.23 2.37 1.42 .34 13.36 2.92 .31 12.33 2.49 6.28 15.23 14.24 13.36 .31 2.92 1.8 9.04 .9 1.42 .34 12.33 11.59 13.36 9.04 2.49 1.42 11.59 .31 1.8 15.23 13.36 6.28 2.92 12.33 2.37 11.59 .31 1.42 5.7 15.23 13.36 12.33 .31 2.92 5.7 .34 1.8 2.37 13.36 9.04 2.92 12.33 2.37 .9 1.42 .34 2.92 9.04 11.59 14.24 12.33 5.7 2.92 13.36 2.49 1.8 14.24 9.04 .34 .31 2.92 6.28 5.7 9.04 11.59 1.42 .31 15.23 6.28 2.92 2.37 9.04 .9 2.49 5.7 6.28 .34 .31 9.04 12.33 1.42 13.36 .9 2.49 5.7 15.23 2.92 13.36 .9 1.8 9.04 11.59 .9 12.33 15.23 .31 6.28 5.7 2.49 2.92 12.33 11.59 .31 .9 1.8 5.7 14.24 6.28 2.92 .34 2.49 5.7 .9 15.23 2.92 2.37 14.24 1.42 9.04 11.59 13.36 14.24 9.04 2.92 12.33 11.59 2.37 15.23 .34 13.36 .31 12.33 6.28 11.59 2.49 13.36 .9 12.33 15.23 2.49 9.04 13.36 1.42 .34 2.92 14.24 12.33 6.28 5.7 13.36 15.23 9.04 2.49 2.92 1.8 13.36 .34 12.33 11.59 2.92 1.8 5.7 13.36 2.37 9.04 .31 .9 .34 11.59 .31 12.33 9.04 .34 5.7 1.42 14.24 13.36 .31 5.7 .31 6.28 14.24 2.37 11.59 .9 2.92 15.23 2.49 1.8 .34 5.7 11.59 14.24 2.49 1.42 11.59 9.04 .9 .34 2.49 5.7 12.33 1.42 9.04 .31 13.36 .9 9.04 13.36 .31 2.49 .9 12.33 6.28 .31 9.04 15.23 .9 1.42 13.36 1.8 15.23 14.24 9.04 .9 12.33 5.7 2.37 2.49 1.42 15.23 11.59 1.8 15.23 .34 1.8 1.42 14.24 2.92 2.37 .9 1.42 13.36 5.7 6.28 9.04 .34 .31 .9 2.92 12.33 14.24 11.59 15.23 5.7 13.36 6.28 11.59 15.23 .9 2.49 1.8 6.28 .9 9.04 2.37 2.49 13.36 15.23 .9 5.7 2.37 .31 1.8 13.36 2.92 15.23 14.24 .34 1.8 .31 1.42 14.24 12.33 6.28 11.59 2.37 2.49 .34 14.24 .9 5.7 15.23 6.28 13.36 .31 14.24 6.28 9.04 11.59 5.7 15.23 12.33 1.8 2.49 13.36 .34 2.92 15.23 1.42 2.37 .9 2.49 5.7 .9 15.23 12.33 13.36 1.8 5.7 1.42 14.24 12.33 2.49 1.42 .9 11.59 13.36 6.28 1.8 .34 2.92 2.49 15.23 .9 13.36 11.59 1.8 2.49 15.23 2.92 6.28 1.42 15.23 13.36 2.37 .9 14.24 2.92 9.04 15.23 14.24 .31 13.36 1.8 12.33 2.37 14.24 1.42 2.92 .9 15.23 6.28 14.24 12.33 1.42 2.49 2.92 5.7 13.36 2.92 1.8 13.36 6.28 1.42 2.92 9.04 14.24 11.59 12.33 5.7 2.37 14.24 9.04 1.42 2.92 1.8 .34 .31 2.49 12.33 .9 2.92 15.23 1.42 12.33 6.28 .34 1.42 2.37 2.49 14.24 5.7 1.42 1.8 9.04 11.59 13.36 .31 2.49 1.42 9.04 5.7 .31 14.24 13.36 1.8 1.42 2.49 .9 .34 6.28 1.8 15.23 14.24 12.33 5.7 2.92 1.42 15.23 1.8 6.28 .9 2.49 .34 13.36 1.8 6.28 12.33 .9 9.04 14.24 2.37 2.49 1.8 12.33 14.24 6.28 .31 9.04 15.23 13.36 1.42 .31 14.24 2.92 13.36 11.59 6.28 1.8 .34 12.33 .31 2.37 2.49 11.59 12.33 9.04 14.24 1.42 5.7 15.23 2.49 1.8 12.33 9.04 11.59 13.36 2.92 15.23 13.36 1.8 11.59 5.7 14.24 12.33 .31 .9 12.33 2.37 .34 .31 5.7 2.49 6.28 1.8 12.33 15.23 13.36 14.24 2.92 9.04 .34 11.59 2.37 1.8 6.28 12.33 15.23 2.92 14.24 13.36 6.28 .34 2.37 14.24 .9 9.04 15.23 2.49 12.33 13.36 2.92 2.37 .34 1.8 14.24 12.33 11.59 5.7 .9 1.42 2.49 12.33 .31 13.36 9.04 6.28 2.92 15.23 5.7 .34 .31 14.24 13.36 1.42 5.7 2.37 14.24 13.36 5.7 9.04 2.49 14.24 1.8 9.04 11.59 13.36 6.28 .34 14.24 .9 2.92 13.36 5.7 .9 .34 14.24 2.92 15.23 2.37 1.8 2.49 9.04 .31 12.33 13.36 2.37 .34 .9 1.8 6.28 .31 2.49 2.92 1.8 .9 .34 2.92 11.59 15.23 14.24 .34 .31 6.28 1.42 9.04 13.36 12.33 6.28 2.37 9.04 2.92 .9 15.23 1.42 .31 12.33 6.28 11.59 9.04 2.37 2.92 15.23 1.42 14.24 12.33 .34 .9 .31 6.28 13.36 11.59 5.7 .9 12.33 11.59 2.37 9.04 .9 15.23 2.92 11.59 .31 14.24 9.04 1.8 2.49 .9 13.36 .34 5.7 2.37 14.24 .34 2.49 13.36 5.7 11.59 14.24 .34 2.37 2.49 11.59 1.8 1.42 2.92 2.37 .9 6.28 15.23 .31 2.37 5.7 .34 .9 6.28 2.92 .31 12.33 13.36 5.7 6.28 9.04 14.24 .31 1.42 2.37 11.59 13.36 12.33 5.7 .9 15.23 13.36 9.04 2.92 12.33 14.24 15.23 6.28 2.37 1.8 5.7 2.92 2.49 .9 6.28 15.23 1.8 .34 5.7 12.33 11.59 2.37 .31 9.04 11.59 1.8 5.7 15.23 13.36 12.33 2.37 1.8 2.92 9.04 .9 5.7 2.92 .34 .31 2.49 13.36 11.59 1.42 2.92 12.33 6.28 9.04 13.36 .31 12.33 .9 9.04 .34 2.92 2.37 .31 9.04 14.24 2.92 13.36 2.37 5.7 1.42 .9 .31 15.23 .34 13.36 14.24 1.42 .31 2.92 2.49 1.8 .9 6.28 15.23 2.92 .34 14.24 1.8 11.59 .9 .31 6.28 .34 15.23 5.7 .34 .31 2.92 .9 9.04 13.36 15.23 .9 14.24 12.33 9.04 1.8 .31 5.7 11.59 .34 .9 13.36 .31 1.42 6.28 12.33 .34 2.49 2.92 13.36 9.04 2.37 15.23 .31 11.59 14.24 5.7 13.36 1.42 .9 9.04 12.33 6.28 15.23 11.59 14.24 5.7 2.92 .31 .9 2.37 .9 9.04 12.33 1.8 15.23 .31 2.37 11.59 .9 15.23 9.04 .31 5.7 13.36 11.59 9.04 1.42 .9 .34 14.24 .31 2.49 15.23 5.7 12.33 9.04 .9 13.36 6.28 .34 5.7 2.92 14.24 6.28 11.59 12.33 5.7 .31 13.36 1.8 .9 2.37 5.7 2.92 9.04 .31 2.37 13.36 14.24 12.33 2.37 .9 13.36 2.92 2.49 .34 14.24 1.8 .31 2.49 5.7 9.04 11.59 .31 5.7 2.37 6.28 9.04 2.92 2.37 14.24 .9 12.33 5.7 2.49 13.36 .9 14.24 11.59 9.04 .31 11.59 2.49 5.7 9.04 .31 2.37 6.28 .34 .9 2.92 1.8 13.36 9.04 11.59 12.33 1.8 11.59 5.7 .31 13.36 2.37 2.49 5.7 13.36 9.04 12.33 1.8 11.59 1.42 .31 2.49 12.33 .34 5.7 2.92 2.49 1.8 14.24 11.59 .9 13.36 .34 1.42 2.92 2.37 12.33 6.28 .31 9.04 1.8 1.42 2.92 11.59 .31 5.7 .34 2.49 13.36 6.28 12.33 11.59 2.37 1.8 .34 11.59 5.7 .9 15.23 9.04 6.28 2.37 .34 .31 .9 6.28 15.23 13.36 12.33 2.92 1.8 15.23 6.28 .9 9.04 13.36 6.28 11.59 1.8 5.7 .31 1.42 2.37 13.36 14.24 12.33 6.28 15.23 11.59 .31 9.04 2.49 2.92 13.36 9.04 .31 .9 12.33 1.42 11.59 6.28 2.37 15.23 13.36 12.33 2.92 .9 6.28 14.24 2.37 1.8 12.33 1.42 9.04 15.23 6.28 1.8 2.92 14.24 .34 .31 12.33 5.7 6.28 2.92 .9 .34 2.49 14.24 12.33 13.36 11.59 6.28 9.04 5.7 .31 2.92 12.33 15.23 1.42 .31 11.59 9.04 5.7 13.36 .9 .31 1.8 2.49 1.42 2.92 .9 15.23 2.49 13.36 5.7 2.92 1.42 14.24 .9 1.8 15.23 9.04 6.28 13.36 2.49 .9 1.8 9.04 1.42 2.37 2.92 12.33 6.28 .31 12.33 5.7 9.04 .9 14.24 2.92 2.37 11.59 6.28 13.36 1.42 2.49 2.92 13.36 1.8 14.24 11.59 5.7 12.33 2.37 14.24 .34 15.23 .9 6.28 1.8 9.04 2.92 14.24 13.36 9.04 11.59 1.42 15.23 12.33 1.8 9.04 .9 2.37 2.49 .34 15.23 5.7 14.24 12.33 13.36 9.04 2.37 11.59 1.42 15.23 1.8 13.36 2.92 6.28 .31 2.49 2.92 6.28 .34 12.33 1.8 14.24 13.36 5.7 12.33 15.23 9.04 1.42 2.37 6.28 1.8 13.36 12.33 11.59 2.49 13.36 .34 11.59 14.24 1.42 1.8 13.36 .34 15.23 9.04 12.33 .9 11.59 .31 2.37 13.36 2.92 6.28 14.24 1.8 13.36 1.42 6.28 1.8 12.33 2.49 14.24 2.37 15.23 9.04 .31 2.37 12.33 2.92 5.7 11.59 1.8 2.49 14.24 11.59 .9 13.36 2.92 1.8 9.04 11.59 1.42 .9 12.33 13.36 2.49 9.04 5.7 15.23 2.37 11.59 2.92 .9 12.33 9.04 2.37 1.8 15.23 11.59 2.49 2.92 6.28 1.42 1.8 13.36 15.23 9.04 2.92 11.59 .34 13.36 1.42 5.7 14.24 15.23 12.33 2.49 11.59 .31 1.42 14.24 13.36 .34 .9 15.23 11.59 2.92 12.33 9.04 .9 .31 2.49 11.59 15.23 6.28 .9 9.04 11.59 12.33 2.49 6.28 15.23 13.36 1.42 .31 12.33 13.36 5.7 1.42 .31 6.28 2.92 2.49 .9 2.37 11.59 9.04 12.33 .9 13.36 2.92 2.49 .31 12.33 15.23 .9 14.24 .34 13.36 6.28 2.37 1.8 5.7 6.28 13.36 12.33 9.04 5.7 .31 15.23 2.92 2.49 9.04 .9 14.24 2.37 2.92 1.8 15.23 5.7 12.33 11.59 9.04 1.42 .31 14.24 2.37 .9 .31 12.33 1.8 1.42 14.24 6.28 13.36 .31 1.42 15.23 13.36 9.04 .9 2.37 2.49 1.8 .34 11.59 5.7 13.36 9.04 1.42 2.37 14.24 .9 2.92 15.23 5.7 .31 1.42 11.59 2.92 .34 9.04 6.28 1.8 14.24 1.42 2.92 12.33 11.59 13.36 2.37 9.04 2.49 15.23 .31 12.33 9.04 14.24 5.7 13.36 9.04 .9 14.24 1.8 11.59 6.28 .9 .31 1.8 2.37 12.33 2.49 9.04 1.42 11.59 2.92 .31 14.24 2.49 1.42 6.28 9.04 12.33 2.37 1.8 5.7 15.23 2.92 12.33 .9 14.24 .31 1.42 9.04 2.92 11.59 .31 12.33 1.42 14.24 2.49 5.7 .34 1.42 14.24 11.59 15.23 2.92 1.8 12.33 11.59 .31 9.04 1.8 15.23 5.7 .34 12.33 2.92 2.37 .9 6.28 11.59 5.7 1.42 9.04 15.23 12.33 2.37 11.59 .9 2.92 1.8 .31 2.49 5.7 .9 9.04 15.23 .34 2.92 14.24 9.04 2.37 12.33 1.8 2.92 1.42 .34 14.24 2.37 9.04 5.7 13.36 15.23 .34 .31 14.24 1.8 11.59 .9 2.92 2.49 14.24 5.7 9.04 2.37 13.36 12.33 1.8 5.7 2.37 5.7 6.28 15.23 11.59 1.8 9.04 14.24 1.42 1.8 .9 12.33 2.92 11.59 15.23 14.24 6.28 11.59 5.7 12.33 .9 2.92 11.59 1.8 9.04 2.92 .9 12.33 6.28 13.36 2.37 1.42 14.24 2.49 5.7 12.33 1.8 .31 2.49 1.42 13.36 5.7 2.37 2.92 15.23 2.49 1.8 6.28 .34 15.23 13.36 1.8 12.33 6.28 13.36 2.37 2.92 9.04 2.49 .34 6.28 12.33 14.24 13.36 2.37 2.92 .34 11.59 14.24 15.23 5.7 12.33 2.49 1.42 2.92 14.24 12.33 2.37 2.92 6.28 14.24 12.33 15.23 .34 13.36 9.04 .31 1.8 2.49 6.28 .34 .9 .31 15.23 11.59 14.24 .34 2.92 1.42 5.7 14.24 13.36 11.59 1.42 6.28 9.04 2.49 .31 .9 2.37 .34 1.8 9.04 6.28 12.33 1.42 5.7 9.04 2.37 .9 2.49 14.24 15.23 1.8 5.7 2.49 2.92 14.24 5.7 .9 13.36 1.42 2.37 .31 12.33 1.8 15.23 9.04 1.42 13.36 6.28 14.24 .9 11.59 15.23 2.49 1.42 13.36 5.7 2.92 .34 14.24 13.36 12.33 2.37 6.28 2.49 .34 2.92 15.23 9.04 .9 11.59 6.28 2.37 14.24 5.7 .34 1.42 12.33 .31 6.28 9.04 5.7 2.92 2.37 11.59 2.49 15.23 9.04 12.33 11.59 .34 .31 15.23 6.28 13.36 2.49 11.59 1.42 2.37 2.49 12.33 5.7 14.24 1.42 .9 13.36 1.8 2.37 .31 9.04 11.59 .34 12.33 2.49 15.23 13.36 1.42 .9 12.33 1.8 .31 15.23 2.92 11.59 2.37 2.49 .9 14.24 2.92 11.59 6.28 12.33 .9 2.49 14.24 1.8 12.33 13.36 5.7 .9 2.49 6.28 5.7 1.42 15.23 .31 2.92 12.33 .9 13.36 14.24 .31 1.8 12.33 13.36 2.92 6.28 2.49 11.59 5.7 9.04 .31 2.92 11.59 12.33 6.28 9.04 13.36 .31 6.28 15.23 14.24 11.59 13.36 14.24 6.28 .31 .9 1.42 2.92 11.59 12.33 5.7 .9 .34 1.8 9.04 .34 .31 .9 11.59 2.49 14.24 2.92 15.23 2.37 11.59 .31 9.04 13.36 1.42 5.7 2.49 .9 .31 5.7 12.33 2.37 1.8 2.92 5.7 .9 1.8 .31 11.59 1.42 14.24 12.33 .34 .9 2.49 6.28 5.7 12.33 2.92 14.24 13.36 .9 1.8 .31 1.42 6.28 .9 .34 2.37 15.23 9.04 1.8 5.7 .9 14.24 2.92 .34 2.49 13.36 14.24 15.23 6.28 .9 11.59 13.36 1.42 .34 14.24 11.59 15.23 2.49 6.28 1.42 5.7 13.36 2.37 .31 1.8 9.04 12.33 6.28 13.36 12.33 2.37 .34 .31 14.24 5.7 15.23 6.28 1.8 .34 2.49 .9 13.36 .31 12.33 9.04 11.59 5.7 .34 15.23 2.92 9.04 .9 14.24 11.59 2.37 15.23 12.33 1.42 .9 1.8 .34 6.28 15.23 1.42 9.04 14.24 2.49 13.36 .34 12.33 5.7 2.49 1.42 11.59 .9 15.23 5.7 14.24 .31 1.42 1.8 11.59 14.24 .34 9.04 2.37 .9 .31 11.59 2.49 9.04 2.92 5.7 2.37 11.59 1.8 9.04 13.36 6.28 .31 5.7 11.59 9.04 1.8 12.33 6.28 15.23 2.92 2.37 13.36 1.8 .34 .31 9.04 .9 2.92 .34 6.28 11.59 1.42 2.37 .31 2.49 .9 1.8 2.92 9.04 2.37 6.28 1.42 9.04 5.7 2.49 11.59 13.36 14.24 1.42 .31 2.37 15.23 1.42 2.92 6.28 11.59 .9 5.7 15.23 12.33 14.24 1.42 9.04 2.92 6.28 1.42 1.8 .31 15.23 2.49 9.04 2.37 11.59 6.28 5.7 .34 13.36 2.49 12.33 2.92 9.04 2.37 1.8 11.59 6.28 15.23 9.04 .9 2.37 2.49 14.24 15.23 2.92 6.28 12.33 5.7 9.04 13.36 6.28 2.49 11.59 9.04 1.8 13.36 1.42 14.24 2.49 5.7 2.37 .34 9.04 14.24 1.42 2.49 6.28 13.36 .31 14.24 15.23 13.36 1.42 2.92 2.49 9.04 14.24 13.36 1.42 15.23 12.33 14.24 5.7 .31 11.59 9.04 12.33 .9 .31 2.92 13.36 .34 6.28 15.23 14.24 9.04 2.37 11.59 .31 1.42 2.92 9.04 2.49 13.36 15.23 .9 .31 9.04 2.49 11.59 14.24 5.7 6.28 1.8 13.36 9.04 12.33 .31 .9 15.23 1.8 5.7 13.36 6.28 2.49 9.04 .9 5.7 1.42 9.04 .34 1.8 11.59 14.24 15.23 2.49 12.33 11.59 1.42 2.37 6.28 .31 13.36 5.7 15.23 12.33 1.8 1.42 13.36 15.23 9.04 12.33 2.92 1.8 .9 9.04 14.24 6.28 9.04 5.7 2.37 1.8 11.59 12.33 .34 9.04 13.36 1.8 11.59 2.49 .9 13.36 6.28 5.7 1.42 14.24 11.59 2.37 12.33 .31 14.24 2.92 11.59 .9 1.8 13.36 2.92 11.59 6.28 2.37 15.23 .34 12.33 11.59 14.24 2.37 .9 .34 13.36 9.04 11.59 2.92 .9 1.42 2.37 6.28 9.04 .9 15.23 5.7 12.33 2.49 .34 2.37 .31 2.92 6.28 .34 13.36 .9 5.7 9.04 2.37 6.28 .31 12.33 1.8 5.7 1.42 11.59 15.23 12.33 13.36 11.59 9.04 2.37 .31 15.23 .34 2.92 12.33 1.42 6.28 15.23 11.59 2.92 5.7 1.42 11.59 .34 13.36 9.04 12.33 5.7 .31 1.42 11.59 2.37 15.23 13.36 11.59 6.28 .9 2.37 12.33 5.7 .9 9.04 15.23 14.24 6.28 11.59 15.23 2.37 9.04 .9 13.36 2.49 12.33 6.28 1.42 14.24 11.59 15.23 9.04 .9 2.49 5.7 12.33 2.37 13.36 11.59 9.04 .34 .31 6.28 14.24 13.36 1.42 12.33 2.92 2.49 11.59 13.36 2.92 .9 11.59 2.37 13.36 2.49 .9 2.92 12.33 6.28 14.24 13.36 1.8 2.92 12.33 9.04 13.36 6.28 15.23 1.8 .9 .34 5.7 6.28 9.04 2.92 11.59 2.49 1.8 5.7 2.92 13.36 .31 2.37 6.28 15.23 12.33 9.04 11.59 2.49 .9 1.42 .31 9.04 14.24 2.37 .9 6.28 5.7 1.8 11.59 .34 13.36 14.24 9.04 15.23 2.37 11.59 1.42 13.36 15.23 2.92 2.37 11.59 1.8 .31 2.37 1.42 12.33 2.49 1.8 .9 1.42 .31 2.37 13.36 9.04 15.23 6.28 5.7 12.33 11.59 9.04 13.36 2.37 .34 15.23 5.7 13.36 6.28 9.04 2.37 12.33 .34 2.92 .31 13.36 5.7 2.37 12.33 14.24 15.23 .31 5.7 .9 11.59 13.36 2.49 6.28 15.23 .34 2.92 13.36 12.33 11.59 2.49 9.04 2.92 .9 15.23 2.37 11.59 14.24 .34 2.92 2.37 6.28 1.42 5.7 9.04 2.37 13.36 .31 1.42 .9 2.37 2.49 15.23 14.24 11.59 5.7 .34 2.37 14.24 13.36 5.7 12.33 .34 .31 5.7 11.59 1.42 .9 1.8 14.24 .31 6.28 1.42 2.92 2.37 12.33 5.7 14.24 15.23 .34 .31 2.37 6.28 2.49 .31 5.7 11.59 12.33 1.8 2.37 15.23 12.33 9.04 .9 5.7 15.23 1.42 2.49 11.59 13.36 14.24 6.28 15.23 .34 1.8 2.37 11.59 1.42 13.36 5.7 .9 2.37 15.23 14.24 2.92 12.33 1.42 .34 9.04 11.59 2.49 14.24 15.23 2.92 6.28 5.7 13.36 9.04 .9 6.28 1.42 11.59 2.49 15.23 14.24 1.42 .31 12.33 15.23 1.42 .31 5.7 12.33 1.8 14.24 .31 6.28 .34 .9 2.49 11.59 15.23 2.92 .34 1.42 12.33 6.28 15.23 .9 9.04 1.42 12.33 2.49 5.7 2.37 2.92 .31 15.23 .34 2.37 2.92 1.42 2.49 13.36 5.7 1.8 2.49 1.42 9.04 12.33 14.24 .34 15.23 1.8 5.7 2.49 9.04 .34 1.8 14.24 5.7 1.8 6.28 13.36 11.59 9.04 5.7 15.23 2.92 9.04 .9 11.59 13.36 6.28 1.8 15.23 9.04 5.7 14.24 2.37 2.49 12.33 .34 5.7 9.04 6.28 1.42 2.37 11.59 .34 1.42 2.92 12.33 9.04 5.7 15.23 .9 .34 2.92 13.36 .31 6.28 12.33 5.7 .34 2.37 9.04 .31 13.36 1.8 1.42 2.92 6.28 12.33 11.59 1.42 .34 6.28 11.59 2.37 2.49 1.42 .9 6.28 5.7 11.59 2.37 .31 2.92 9.04 1.42 5.7 11.59 13.36 14.24 .9 2.92 9.04 5.7 1.42 13.36 6.28 2.37 11.59 12.33 .9 2.92 .34 1.42 15.23 6.28 5.7 2.92 1.42 1.8 6.28 12.33 .31 .9 11.59 2.49 5.7 .34 14.24 6.28 .9 9.04 15.23 2.37 1.42 1.8 2.49 5.7 .31 13.36 11.59 15.23 12.33 2.92 2.49 .34 15.23 5.7 14.24 .9 2.49 1.42 1.8 5.7 12.33 9.04 13.36 15.23 .34 2.37 12.33 5.7 .9 15.23 6.28 2.92 13.36 11.59 1.8 9.04 6.28 2.49 .9 2.37 15.23 13.36 6.28 5.7 2.37 .9 2.92 .31 13.36 1.8 5.7 6.28 12.33 9.04 .31 .9 13.36 15.23 11.59 6.28 2.49 5.7 9.04 15.23 2.49 .31 5.7 2.92 6.28 14.24 15.23 12.33 9.04 1.42 15.23 5.7 13.36 .31 12.33 .9 2.37 11.59 2.92 .31 5.7 2.37 15.23 6.28 .9 12.33 .34 14.24 2.92 15.23 1.8 2.49 9.04 5.7 13.36 .9 14.24 12.33 2.49 11.59 .9 2.92 5.7 15.23 12.33 .34 2.49 11.59 13.36 9.04 .31 2.49 15.23 1.42 13.36 2.49 5.7 15.23 .9 6.28 9.04 2.49 1.8 1.42 2.37 .31 14.24 .9 6.28 2.37 5.7 1.42 13.36 14.24 .9 .31 6.28 1.42 11.59 2.49 12.33 .34 .31 6.28 1.8 2.37 15.23 .9 .34 9.04 2.37 5.7 1.42 .31 9.04 2.37 6.28 2.92 5.7 .9 9.04 2.37 .31 2.49 .34 5.7 13.36 1.42 14.24 2.49 6.28 15.23 2.37 5.7 2.49 14.24 .9 .34 15.23 6.28 1.42 11.59 5.7 2.92 15.23 1.42 .9 .34 2.92 14.24 9.04 11.59 1.42 2.37 12.33 15.23 1.42 2.92 5.7 6.28 2.49 14.24 2.37 15.23 .9 1.8 .31 5.7 9.04 2.49 6.28 12.33 .31 13.36 2.37 9.04 12.33 15.23 13.36 1.42 2.49 2.92 14.24 2.37 9.04 1.8 2.49 12.33 .34 2.37 9.04 13.36 5.7 .9 2.37 2.49 13.36 2.92 9.04 11.59 6.28 12.33 1.8 15.23 14.24 2.92 9.04 12.33 6.28 .31 1.42 13.36 2.49 .34 13.36 9.04 14.24 .31 1.42 5.7 2.49 12.33 14.24 5.7 11.59 15.23 9.04 2.49 .9 1.42 2.92 11.59 14.24 9.04 1.8 12.33 1.42 6.28 2.92 14.24 11.59 .34 1.42 6.28 12.33 .9 2.92 6.28 1.8 .34 2.37 .9 5.7 13.36 6.28 9.04 1.42 11.59 12.33 2.92 9.04 .31 1.8 6.28 13.36 15.23 9.04 12.33 14.24 5.7 .9 .34 11.59 12.33 15.23 2.49 14.24 1.8 5.7 11.59 2.49 9.04 2.92 .34 6.28 14.24 11.59 2.49 1.8 1.42 13.36 12.33 5.7 .9 6.28 2.37 9.04 5.7 13.36 11.59 9.04 12.33 15.23 2.37 5.7 13.36 9.04 12.33 6.28 .34 2.37 14.24 6.28 1.42 11.59 13.36 1.8 2.37 6.28 2.49 12.33 .34 15.23 2.37 1.8 13.36 2.92 0 5 10 15 Number of correct letter sounds Control Treatment Control Treatment Control Treatment Control Treatment RR Acoli Lugbarati Lumasaaba Baseline Endline Letter Sound Identification SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 30 Figure 6: Cluster2, SHRP estimated effects on letter sound identification since baseline, by language *** p<0.001, ** p<0.01, * p<0.05, + p<0.1Estimates from OLS regressions including school fixed effects, individual controls and a constant term. Individual controls include age and sex of learner, dummy for age missing, score on household assets index, dummies for student living with both parents, someone at home reading to the student and language spoken at home. Regressions for English language include control for local language Figure 6 shows the effects of SHRP on letter sound identification in this cohort at the end of each grade since the start of the program. In general, the effects have been quite modest at best and have been stable over time, with the exception of Lumasaaba in the last year which shows some improvement. We find some reductions due to the program in the proportion of learners with zero letter sound score. The second row in Table 5 shows the estimates. We find statistically significant reductions in Lumasaaba and English of 11 and 8 percentage points (0.36 and 0.16 standard deviations) respectively. On the other hand, we find that in Lugbarati the reduction in zero scores was actually larger for control learners than for those receiving SHRP, the effect is 11.5 percentage points (0.35 standard deviations) more improvement among the control group. In all languages included in Cluster 2 there are still learners in SHRP schools unable to identify a single letter sound correctly, the percentages remain high in Lugbarati, 27.2 percent, and in Acoli where the percentage reaches 55.2 percent. We show the impact of SHRP on word segmenting in row 3 of Table 5. None of the estimates are statistically significant different from zero. The effects on the non-word decoding task are presented in the following row. It can be seen that the estimates are positive and statistically significant for English, Runyoro-Rutooro, Acoli and Lumasaaba. For Acoli the effect is below one word (0.24 standard deviation) and only significant at 10 percent. For English and Lumasaaba the impact is a bit larger, 1.7 and 1.6 words (0.25 and 0.29 standard deviations) respectively. We find a larger effect of 5.2 words (0.51 standard deviation) for the Runyoro-Rutooro subgroup. In terms of the impact of the intervention on fluency we find a positive effect of 3.4 words (0.28 standard deviation) in English, 7.2 words (0.5 standard deviation) in Runyoro-Rutooro, 2.4 words for both Acoli and in Lumasaaba (0.32 and 0.36 standard deviations respectively). We 0.0 2.4* 0.8 2.6 1.6+ 1.4 2.6 -0.8 -1.0 -1.7 0.7+ 0.9 5.6** 0.8 0.4 2.4* -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 Baseline At P1 At P2 At P3 Difference in Letter Sound Identification Runyoro-Rutoro Acoli Lugbarati Lumasaaba English SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 31 find no effect among Lugbarati speaking learners, which is not surprising given the lack of effects on other subtasks of the assessment. Figure 7: Cluster 2, Oral reading fluency at baseline and after three years of intervention, by language Error bars indicate 95% confidence interval Learners in Cluster 2 still read very few words in a connected text at the end of P3, after receiving the SHRP interventions of three years. Figure 7 shows the raw data from the baseline assessment at the beginning of P1 and from October 2016 at the end of the third grade for all four local languages in cluster 2. The performance is very poor for P3 learners in all languages. Runyoro-Rutooro treatment learners are the subgroup that reaches the highest level of reading fluency which, on average, is only 14 words in a minute. Figure 8 illustrates one aspect of this low performance. At the end of P3, the majority of learners, in both treatment or in control schools, is still unable to read even one word from the assessment paragraph. The proportion is relatively better among the Runyoro-Rutooro treatment group (40%) but still very high. We find some small effects in reading comprehension in English and Acoli of 0.08 and 0.11 questions respectively (0.19 and 0.25 standard deviation). A larger effect of 0.77 questions (0.53 standard deviation) was estimated for the Runyoro-Rutooro group. Like in the case of Cluster 1, for listening comprehension, we have identified a negative effect of 0.33 questions (0.41 standard deviation) in Acoli. Finally, there is no effect on the English receptive vocabulary subtask. .07 6.09 4.11 6.56 3.43 .02 7.05 0 14.23 .02 .01 6.09 6.56 1.72 3.43 0 7.05 .02 .01 6.09 .01 3.43 .07 7.05 4.11 1.72 .07 .01 .01 3.43 14.23 .01 .02 1.72 .07 .01 3.43 .04 4.11 .02 .01 .01 3.43 6.56 7.05 14.23 .96 .01 1.72 .07 6.56 .02 7.05 6.09 14.23 1.72 .04 .02 6.09 4.11 .01 3.43 1.72 14.23 6.56 .02 0 7.05 .01 3.43 1.72 14.23 .07 6.56 3.43 .02 14.23 .07 .04 .01 .01 4.11 .01 1.72 6.56 7.05 .96 3.43 1.72 .07 .01 0 3.43 6.56 .01 .01 6.09 .04 .01 4.11 .96 .01 6.09 14.23 1.72 .07 .01 .01 3.43 6.09 .96 1.72 7.05 .04 .01 0 6.56 3.43 .02 4.11 .07 6.09 .01 .01 0 1.72 .04 .01 .96 .02 3.43 6.56 14.23 .01 0 .96 .04 1.72 0 6.56 .01 7.05 .02 14.23 .07 6.56 1.72 .02 4.11 0 6.09 14.23 .07 .01 1.72 3.43 6.56 .96 7.05 .02 4.11 .02 3.43 7.05 .07 .04 6.56 .96 .02 7.05 6.09 1.72 .04 .01 4.11 .01 0 14.23 1.72 6.09 .04 3.43 .02 .01 7.05 .01 6.56 .02 0 3.43 .01 .96 .07 14.23 4.11 .01 3.43 6.56 7.05 6.09 .07 .01 14.23 .04 .02 .02 0 3.43 .01 .01 1.72 4.11 .01 6.56 .04 .96 .01 0 3.43 1.72 .07 6.09 6.56 4.11 14.23 .96 1.72 .02 4.11 6.09 .02 14.23 7.05 4.11 6.56 .07 .02 .96 .01 6.56 14.23 4.11 7.05 .07 .02 .02 14.23 6.56 .96 .02 1.72 7.05 3.43 .02 .96 .07 .02 4.11 6.09 .01 3.43 1.72 .01 .02 0 6.09 3.43 .01 .96 .02 6.56 7.05 .04 4.11 .01 0 3.43 .96 6.09 .01 14.23 1.72 .04 0 14.23 .02 4.11 .02 .96 6.09 7.05 0 14.23 1.72 .01 6.09 .02 .01 6.56 .96 .07 .04 .01 3.43 6.56 7.05 14.23 1.72 .07 .01 .96 3.43 4.11 .07 .02 .02 0 6.56 4.11 1.72 7.05 6.09 .01 .02 1.72 .01 .96 14.23 .07 6.56 4.11 .07 .01 .01 0 7.05 3.43 .01 .02 14.23 .07 .04 .01 .01 .01 1.72 3.43 .96 .01 .01 0 .96 14.23 6.09 0 3.43 7.05 .04 .02 .02 1.72 .01 6.56 .01 4.11 1.72 7.05 .04 .01 3.43 .02 4.11 .07 7.05 .01 14.23 .01 6.56 .07 .02 .01 .96 7.05 1.72 3.43 .02 0 4.11 .01 14.23 1.72 .07 .02 .02 3.43 .01 .01 6.09 .07 .02 3.43 14.23 .02 6.56 .01 4.11 .04 .02 14.23 3.43 4.11 1.72 .04 .01 7.05 .07 0 14.23 7.05 .07 .01 0 6.56 4.11 6.09 .04 .02 .01 .96 1.72 .02 14.23 6.56 7.05 .01 0 6.09 3.43 4.11 1.72 .04 .02 14.23 .01 1.72 .07 7.05 4.11 .04 .01 .96 .02 4.11 .01 .96 .07 .04 0 7.05 4.11 .96 14.23 6.56 .02 7.05 .04 .01 3.43 .96 6.56 .04 0 4.11 14.23 .96 .01 0 3.43 .07 .02 6.09 .04 .01 1.72 6.56 .02 .02 3.43 .07 .04 7.05 .02 6.56 .07 .02 0 4.11 7.05 .01 1.72 .04 4.11 .02 .96 .07 .01 .01 7.05 .04 0 4.11 6.09 .07 .01 .02 14.23 .04 7.05 .96 .02 4.11 1.72 .04 .02 0 .96 7.05 4.11 0 .96 14.23 .04 .01 6.56 6.09 .02 0 6.09 14.23 .02 3.43 .02 1.72 .07 .04 .02 .01 .01 1.72 0 7.05 .02 3.43 6.56 1.72 .96 14.23 .07 .01 .02 7.05 .96 0 6.56 .02 7.05 3.43 14.23 1.72 4.11 .04 .02 6.56 .02 .01 0 14.23 .04 6.56 .02 0 1.72 .96 .07 .01 6.09 14.23 0 1.72 6.56 7.05 3.43 .04 .01 1.72 4.11 14.23 6.56 3.43 .02 .01 .01 6.56 6.09 0 .96 6.56 6.09 14.23 .04 .02 .01 .02 0 6.56 .01 7.05 .04 6.09 1.72 .01 14.23 .01 0 4.11 .07 6.09 .04 .01 4.11 1.72 .01 0 6.09 4.11 6.56 7.05 .02 3.43 4.11 .04 0 1.72 7.05 6.56 .02 .02 6.09 3.43 1.72 .02 0 4.11 .07 7.05 14.23 1.72 .01 .02 6.09 .02 7.05 14.23 .07 7.05 1.72 6.56 .07 .01 .01 .01 1.72 .96 6.56 3.43 .04 .01 0 14.23 1.72 .01 6.56 6.09 4.11 .01 .02 1.72 14.23 .01 .02 6.56 .07 3.43 1.72 .01 .96 4.11 14.23 3.43 6.09 .01 .02 0 .96 6.56 .01 6.09 4.11 6.56 3.43 6.09 .02 0 4.11 14.23 .02 0 1.72 .01 7.05 .96 .01 4.11 .04 0 14.23 1.72 .96 .01 .01 14.23 6.56 1.72 .96 7.05 .04 .01 .01 3.43 .02 0 14.23 4.11 1.72 7.05 .01 6.56 .96 .02 .02 4.11 6.09 .01 .96 14.23 .01 7.05 3.43 6.56 1.72 .04 14.23 6.09 .07 .96 7.05 .02 4.11 14.23 3.43 .96 0 1.72 .01 6.56 7.05 .02 .01 1.72 .07 .02 4.11 14.23 3.43 6.56 .02 1.72 0 .96 6.09 3.43 1.72 14.23 .04 .01 .02 4.11 1.72 0 6.56 .02 6.09 .96 3.43 .07 .02 .01 .01 0 6.09 7.05 .04 4.11 14.23 1.72 6.09 .07 .01 .02 7.05 .96 6.09 .02 4.11 7.05 .01 .02 14.23 .07 .01 .01 3.43 .01 14.23 4.11 .07 .04 .01 .02 1.72 .07 .96 .01 4.11 7.05 .96 .02 6.09 14.23 .01 .02 .96 .02 6.09 4.11 3.43 7.05 6.56 .01 0 6.09 .04 3.43 .07 .01 6.56 3.43 0 1.72 .96 6.56 4.11 6.09 14.23 1.72 .07 0 6.56 .01 .01 .96 14.23 .07 6.56 7.05 .01 .01 0 6.56 .07 .02 1.72 3.43 0 14.23 .04 6.09 1.72 .02 7.05 .02 6.56 .96 .07 .04 .01 14.23 .01 1.72 .01 0 4.11 .02 3.43 6.09 .07 6.56 7.05 1.72 .01 0 6.56 6.09 4.11 .02 .96 .02 .01 3.43 .01 0 1.72 4.11 7.05 .01 .02 0 6.56 .02 3.43 6.09 4.11 .01 .01 0 .96 3.43 6.09 6.56 4.11 7.05 .04 .02 14.23 6.09 .01 .96 .01 .02 1.72 3.43 .04 .02 .01 7.05 0 6.56 .01 1.72 .02 14.23 6.09 3.43 1.72 4.11 .04 .01 .96 0 7.05 3.43 .01 .02 6.56 4.11 7.05 .01 .01 3.43 0 7.05 .04 6.09 .07 .02 .01 .01 0 4.11 .01 .02 0 1.72 .96 14.23 6.09 .01 4.11 .01 .02 0 .96 3.43 6.09 14.23 7.05 .04 4.11 3.43 1.72 .02 14.23 4.11 6.56 1.72 3.43 .04 4.11 .02 7.05 6.09 3.43 14.23 7.05 .02 6.56 1.72 .07 .01 .01 0 14.23 .04 .01 3.43 .02 4.11 0 7.05 1.72 .01 6.56 4.11 .07 .96 14.23 4.11 .07 6.56 3.43 .02 .02 .96 .07 .01 0 1.72 3.43 6.09 .07 .01 1.72 .04 14.23 0 3.43 6.56 .96 .01 1.72 14.23 3.43 .02 4.11 .07 6.56 1.72 .04 .02 7.05 .01 .96 6.56 0 4.11 3.43 .07 14.23 .01 .02 3.43 .07 .96 4.11 .04 .01 0 1.72 .07 6.56 14.23 7.05 .07 .04 .01 .01 6.56 .04 6.09 0 .96 .01 14.23 .01 4.11 .04 .02 0 3.43 6.56 .07 .02 14.23 .01 6.56 .04 0 6.09 .96 .07 .01 .01 7.05 3.43 .04 0 14.23 .01 6.56 1.72 14.23 .01 3.43 .01 14.23 .01 .01 4.11 .04 6.09 7.05 .01 .02 3.43 6.56 4.11 .04 .02 .01 .02 14.23 .96 .01 3.43 6.56 .01 .02 0 1.72 6.09 7.05 .96 .01 6.09 1.72 .02 14.23 .07 .02 6.56 .01 .01 1.72 6.09 0 6.56 .04 .96 .02 4.11 .07 .04 .01 0 .96 6.56 6.09 14.23 4.11 .02 7.05 1.72 .07 .01 14.23 .04 4.11 .01 .02 0 3.43 1.72 .02 .01 .96 6.09 3.43 .01 .01 .02 6.56 .01 4.11 0 3.43 14.23 7.05 .07 .02 .01 0 3.43 14.23 6.09 .07 0 6.56 .01 4.11 .02 .01 .96 .04 .01 0 1.72 .02 4.11 6.09 6.56 .07 .02 .01 .01 0 6.09 1.72 0 4.11 .04 .02 .01 0 4.11 6.09 .07 7.05 .07 .02 .01 .01 4.11 .04 6.09 7.05 .01 .01 .01 0 3.43 .07 .04 14.23 .01 6.56 1.72 .02 6.09 6.56 7.05 1.72 .07 .01 .02 4.11 6.56 .96 3.43 .07 .01 0 .96 .01 3.43 7.05 .02 1.72 .02 .01 6.09 .96 4.11 3.43 .04 .01 1.72 .07 .02 .01 .02 .96 3.43 .04 .02 .01 0 7.05 .96 0 6.56 .02 6.09 7.05 14.23 .04 .02 .02 0 6.56 .02 .96 7.05 .01 6.09 .01 .02 1.72 6.56 .01 .01 .96 .02 6.56 .07 .04 .01 .01 0 1.72 6.09 14.23 6.56 .96 .01 0 3.43 .07 .02 .01 1.72 .01 4.11 .07 .02 7.05 .96 .04 0 .96 7.05 6.56 3.43 .04 .01 .01 14.23 6.09 .07 .02 .01 7.05 .01 .01 .01 7.05 14.23 4.11 .07 0 6.56 3.43 .96 .07 .02 .01 .01 7.05 .04 3.43 .96 4.11 .02 .96 .01 3.43 .02 6.56 .04 14.23 7.05 0 6.56 .01 14.23 4.11 .01 .96 6.56 .04 .02 .96 .01 .01 .01 14.23 3.43 .96 .07 .04 .02 1.72 .07 0 7.05 14.23 1.72 .07 6.56 3.43 .96 .04 .02 4.11 .01 3.43 0 7.05 .01 4.11 .02 6.56 6.09 3.43 14.23 .02 .01 6.56 .01 7.05 1.72 4.11 .04 .01 .02 6.09 3.43 7.05 .01 .96 3.43 .02 .02 6.09 14.23 1.72 .07 .01 .01 3.43 6.09 .07 6.56 .01 .01 14.23 7.05 .02 3.43 4.11 .02 6.56 14.23 .96 4.11 .01 3.43 14.23 7.05 .04 .01 1.72 14.23 .01 6.56 .02 .02 .96 .04 14.23 7.05 .02 .01 .96 .04 0 7.05 6.56 14.23 1.72 .96 .04 .01 .02 0 14.23 .96 6.56 1.72 6.09 .96 .01 .01 1.72 .02 6.09 7.05 .01 14.23 6.56 .04 .02 7.05 3.43 .02 6.09 4.11 1.72 .96 .02 .01 .02 6.09 .01 14.23 4.11 .04 6.56 .96 14.23 1.72 6.09 .02 14.23 .02 3.43 7.05 .01 14.23 7.05 4.11 .02 1.72 .01 3.43 .07 4.11 .01 .01 .02 6.56 .07 7.05 .96 4.11 6.09 .04 14.23 .07 .01 .02 0 6.56 14.23 4.11 .07 .01 7.05 1.72 .02 3.43 6.56 .07 7.05 .07 .04 .01 0 4.11 14.23 .04 .02 .01 0 3.43 0 1.72 .96 0 6.56 .96 1.72 14.23 4.11 .02 6.09 6.56 7.05 .96 4.11 .02 .02 6.56 .01 .01 3.43 .01 1.72 4.11 .02 .01 .01 14.23 0 6.56 6.09 7.05 .01 1.72 7.05 .01 6.56 3.43 .02 14.23 7.05 6.09 4.11 .01 .01 3.43 .96 .07 .01 .02 14.23 7.05 1.72 .01 14.23 .01 .02 6.09 7.05 0 6.56 .07 .01 14.23 .04 .96 .02 6.09 6.56 7.05 .07 .96 14.23 6.56 3.43 .04 .02 .01 6.56 7.05 0 1.72 .01 3.43 14.23 7.05 .04 .02 .01 .01 .01 .02 0 .96 .04 7.05 0 1.72 6.56 .07 .02 .96 4.11 6.09 0 7.05 1.72 .07 4.11 .04 .01 .96 14.23 .07 .02 7.05 4.11 1.72 .01 14.23 .02 .96 6.09 .07 .01 .01 .01 0 4.11 6.56 1.72 14.23 7.05 .02 6.09 .07 .01 14.23 7.05 3.43 4.11 .07 .02 .01 6.56 7.05 .04 6.09 7.05 .02 .96 .02 0 6.09 7.05 3.43 .96 .07 14.23 .04 .01 1.72 6.09 .01 .02 .96 4.11 3.43 6.56 7.05 .01 .01 1.72 7.05 .07 .02 .01 3.43 .01 .96 .07 .01 6.56 6.09 .02 .01 1.72 14.23 .04 7.05 .01 4.11 6.56 6.09 7.05 .02 .02 1.72 .04 14.23 .01 3.43 .96 .02 6.09 .02 7.05 .01 4.11 .01 .01 0 6.56 .07 1.72 .02 .01 .02 .96 1.72 7.05 6.56 .01 .02 0 14.23 .01 .96 1.72 6.56 0 3.43 1.72 .96 .02 7.05 4.11 .01 6.09 .04 14.23 0 7.05 .96 3.43 .04 1.72 6.56 .96 .04 .01 .02 1.72 6.09 4.11 .07 .01 7.05 3.43 .02 .96 1.72 6.56 .07 .01 .96 4.11 7.05 .01 14.23 .07 .04 0 3.43 .02 .01 .01 1.72 .01 3.43 .07 .04 .02 0 .96 4.11 6.09 1.72 .01 6.56 .07 .96 7.05 14.23 .01 1.72 .01 4.11 .07 .01 7.05 6.56 1.72 .01 4.11 14.23 .07 7.05 3.43 4.11 .04 .01 7.05 0 1.72 3.43 .04 .01 6.56 .01 4.11 .02 7.05 .07 .04 14.23 3.43 6.56 .96 .07 .02 0 14.23 .02 3.43 .01 6.09 14.23 7.05 .04 .02 .01 .01 3.43 7.05 .01 .01 14.23 .07 .01 .02 6.56 .01 0 7.05 .04 1.72 14.23 6.09 6.56 .01 .01 0 3.43 .02 .01 .02 7.05 4.11 6.56 3.43 .04 .01 4.11 .01 0 14.23 6.56 1.72 .01 7.05 .07 4.11 .01 0 .96 14.23 6.56 7.05 .01 0 .96 .07 .01 .02 6.56 3.43 .01 7.05 .07 .02 .01 .01 .02 3.43 4.11 6.09 7.05 .96 .01 6.56 .04 .01 6.09 3.43 .01 .02 0 1.72 .96 3.43 4.11 .01 0 14.23 1.72 .07 .01 3.43 .04 .02 .96 1.72 .02 0 3.43 7.05 .01 6.56 .04 .02 .96 14.23 .01 6.56 .02 .01 3.43 4.11 .02 7.05 14.23 0 1.72 .96 .04 .01 7.05 4.11 .02 .01 0 .96 6.09 1.72 .04 .01 .02 .96 .02 14.23 .07 .01 6.09 .01 3.43 4.11 6.56 6.09 3.43 .01 7.05 4.11 .07 14.23 .02 6.56 0 7.05 1.72 4.11 .04 6.56 .02 7.05 14.23 .04 3.43 6.56 .01 .01 .02 6.09 7.05 1.72 .96 6.56 14.23 3.43 .04 .01 .01 6.56 .02 7.05 4.11 .01 .01 .96 6.56 .07 3.43 .02 .02 0 4.11 0 7.05 3.43 .02 .96 .01 6.09 6.56 0 .96 0 1.72 6.09 .07 4.11 .02 .01 14.23 .04 .01 .02 .96 4.11 6.09 .04 .01 4.11 3.43 .04 .02 .02 .96 7.05 .01 3.43 0 6.56 .02 3.43 6.56 .04 .02 0 7.05 1.72 0 3.43 14.23 .02 .01 6.56 .01 14.23 6.09 3.43 .02 7.05 .96 .07 .04 .01 14.23 4.11 .01 14.23 .01 .01 .02 6.09 .07 .02 .01 .01 6.56 .96 1.72 3.43 .02 .01 .02 0 7.05 6.09 4.11 14.23 .96 6.56 1.72 4.11 14.23 .04 .02 .01 1.72 .02 3.43 .07 .04 6.56 14.23 .02 .96 .07 .01 0 6.56 .01 14.23 6.09 .01 .01 .02 0 6.09 7.05 1.72 4.11 .07 .04 .02 6.09 .02 .96 14.23 1.72 6.56 0 6.09 1.72 3.43 4.11 .96 14.23 7.05 .04 .01 6.56 .01 .02 14.23 .07 .04 .02 .01 .96 .02 14.23 7.05 6.56 .01 .96 .01 6.09 7.05 .04 .02 .01 4.11 6.56 1.72 .04 .01 .01 .02 14.23 .02 6.56 4.11 .04 .01 14.23 .01 1.72 .01 14.23 6.56 .07 .02 6.09 .01 3.43 14.23 6.09 0 6.56 .01 7.05 .07 6.09 .02 .01 .01 14.23 1.72 6.09 7.05 .04 .01 .01 .96 6.56 .01 .01 6.56 1.72 .01 .01 3.43 6.09 4.11 7.05 .96 .07 6.09 3.43 .04 .01 .01 .01 .02 0 7.05 .02 .01 14.23 .01 7.05 1.72 .07 .04 .01 .02 6.09 .96 .01 .01 3.43 4.11 6.56 .04 .01 0 3.43 .96 0 6.09 6.56 .04 .02 .01 .01 .02 1.72 .01 14.23 6.09 7.05 .96 .01 .01 14.23 .01 1.72 .04 .02 .02 6.56 .01 1.72 7.05 .02 3.43 6.09 .07 .04 .01 7.05 6.09 1.72 0 3.43 14.23 6.56 .01 0 6.09 .01 6.56 4.11 1.72 .01 .02 7.05 .07 .04 0 4.11 7.05 3.43 6.09 .01 .96 14.23 .04 .01 7.05 3.43 4.11 6.56 .01 14.23 6.56 .01 4.11 .96 6.09 7.05 .02 0 7.05 .07 .02 0 .96 .04 1.72 .01 7.05 14.23 6.56 6.09 .01 3.43 .02 4.11 .07 .01 1.72 7.05 14.23 .01 6.09 6.56 1.72 .07 .02 6.09 .02 3.43 14.23 .04 7.05 6.56 .07 .01 .01 .02 6.09 7.05 4.11 .96 .04 .02 .01 7.05 0 6.56 3.43 1.72 .01 14.23 .96 .02 0 6.09 6.56 .01 .96 .07 6.09 6.56 .96 3.43 .04 6.09 .01 3.43 4.11 6.56 1.72 .02 6.09 .02 .01 6.56 .96 .02 .02 6.09 .01 14.23 .07 .04 .01 3.43 0 7.05 6.56 .07 .02 .01 .02 1.72 .04 .02 .01 .01 .02 0 4.11 14.23 6.09 .02 0 1.72 .01 3.43 6.56 7.05 1.72 .07 3.43 .02 .01 14.23 .01 0 7.05 1.72 4.11 3.43 .07 .01 14.23 .01 6.09 7.05 .02 .02 0 1.72 6.56 4.11 .96 .02 7.05 4.11 .07 3.43 .02 14.23 .01 4.11 0 6.09 3.43 .04 .02 .01 6.56 .02 .96 .07 6.09 .04 .02 6.56 .96 4.11 6.09 .07 .04 .02 4.11 .07 .02 .01 .01 .01 1.72 14.23 .07 0 .96 .02 .02 1.72 .01 0 7.05 6.56 .96 1.72 3.43 6.09 .07 .01 0 4.11 6.56 7.05 .96 .02 14.23 6.56 3.43 .01 7.05 6.09 14.23 1.72 .07 .01 .96 .04 .02 .01 1.72 14.23 .01 .02 .96 7.05 4.11 .07 0 3.43 4.11 .01 .96 14.23 6.56 7.05 .07 .01 .01 3.43 .02 .96 .04 .01 .02 0 6.56 4.11 .01 .01 7.05 1.72 3.43 6.56 0 7.05 .02 3.43 .07 .01 .02 0 3.43 6.09 .01 6.56 .07 .96 .02 .01 0 14.23 .02 6.56 6.09 .04 .02 .01 .01 .01 0 1.72 14.23 .01 .02 6.09 .01 4.11 .02 0 1.72 .02 14.23 .96 .02 .01 .02 0 3.43 6.56 14.23 .02 6.09 7.05 3.43 .01 0 .96 4.11 .02 .02 6.56 .01 0 1.72 7.05 .04 .01 .02 6.56 3.43 .07 14.23 0 4.11 6.09 .96 6.56 .02 .01 3.43 7.05 1.72 14.23 4.11 6.09 .96 .07 .02 .01 0 3.43 7.05 .01 14.23 .07 0 4.11 .02 14.23 3.43 0 .96 6.56 4.11 3.43 .02 .01 .02 6.09 .04 0 14.23 .96 7.05 3.43 .02 6.56 1.72 .02 .96 .01 6.09 1.72 4.11 7.05 .96 0 6.56 .07 .02 .01 .96 .01 3.43 .07 0 6.56 6.09 7.05 .07 .02 6.56 .04 .01 .02 6.09 .04 .01 0 .96 3.43 4.11 0 .96 .07 1.72 3.43 .07 .01 6.09 .02 7.05 .96 .04 6.56 .02 6.09 4.11 3.43 0 4.11 .04 .96 3.43 .07 0 1.72 .02 .01 .01 .02 6.56 3.43 4.11 7.05 .01 4.11 .96 0 6.56 .07 .04 .96 6.56 3.43 7.05 .01 4.11 .04 .01 0 7.05 .02 .96 .04 .01 .01 6.09 4.11 .02 6.56 .07 .01 .01 .02 7.05 1.72 0 3.43 .01 .01 .01 4.11 0 .96 .04 .02 6.56 1.72 7.05 4.11 .07 .01 .02 4.11 .96 .02 14.23 3.43 1.72 .07 .02 .02 0 1.72 14.23 6.56 7.05 .01 .01 14.23 1.72 .02 3.43 6.56 1.72 4.11 .01 .96 .07 .01 0 6.56 6.09 7.05 1.72 14.23 4.11 0 3.43 .04 .01 6.56 3.43 .02 0 7.05 .01 4.11 1.72 .07 14.23 6.56 7.05 .02 .01 1.72 6.09 .07 .01 7.05 .01 3.43 14.23 1.72 .01 .01 6.09 .02 0 7.05 .96 1.72 .04 .02 .01 .02 6.09 7.05 6.56 4.11 1.72 3.43 .96 .01 0 7.05 14.23 .01 0 4.11 3.43 .96 6.56 .04 .02 .01 .01 .01 0 14.23 .04 6.56 .96 .01 .01 6.09 .02 .01 14.23 3.43 1.72 6.56 .04 .02 .01 3.43 .07 .01 .01 7.05 1.72 0 7.05 .96 3.43 .02 6.09 .07 .01 4.11 1.72 6.56 .04 .01 .01 6.56 .01 6.09 4.11 .96 7.05 .07 6.09 .02 14.23 .02 1.72 .01 3.43 .01 6.09 6.56 3.43 4.11 .01 14.23 7.05 .01 3.43 .07 .04 .02 .02 14.23 .96 6.09 7.05 6.56 3.43 .07 4.11 .01 14.23 .01 6.56 .01 1.72 .04 .01 0 1.72 .02 7.05 .01 6.09 6.56 .96 7.05 14.23 3.43 .07 .01 1.72 .01 6.56 7.05 4.11 .04 6.56 .02 4.11 6.09 .01 .01 6.56 .02 14.23 3.43 7.05 .02 4.11 .07 0 6.56 .01 1.72 6.09 .01 6.56 .01 1.72 .01 7.05 .04 6.09 .07 14.23 3.43 .07 0 7.05 .01 .96 4.11 .04 .01 6.09 4.11 .02 6.56 .01 .01 3.43 4.11 .02 .01 7.05 6.56 .04 3.43 .96 14.23 .07 4.11 .02 .01 7.05 3.43 .07 .01 14.23 4.11 .04 .01 1.72 .01 .01 6.56 14.23 3.43 .01 4.11 .02 6.56 .01 .96 6.09 14.23 7.05 .04 4.11 .01 0 6.09 6.56 .02 .02 14.23 4.11 .01 7.05 3.43 .04 .02 0 4.11 14.23 1.72 .02 3.43 4.11 7.05 .04 1.72 14.23 6.56 .01 0 7.05 6.56 .96 .01 0 1.72 .07 .04 .02 .01 4.11 3.43 7.05 .02 6.56 .04 .01 0 7.05 14.23 .02 6.09 .02 6.56 1.72 .07 .01 .96 1.72 6.56 7.05 3.43 .96 0 14.23 .04 .01 3.43 .02 6.09 .07 .01 .01 14.23 .96 7.05 4.11 3.43 .01 0 6.09 .07 .02 0 7.05 .01 .01 6.09 1.72 6.56 .01 0 14.23 6.56 3.43 .07 .04 .02 .01 .02 4.11 .96 6.56 3.43 .07 .01 6.09 .02 .01 14.23 .96 .01 0 4.11 .01 .02 3.43 1.72 .01 6.09 .01 .01 7.05 4.11 6.56 .07 3.43 .04 14.23 0 7.05 3.43 6.09 .96 6.56 3.43 .02 6.09 .01 4.11 1.72 .07 .02 .01 0 7.05 .04 3.43 .01 4.11 .01 0 6.09 .04 .01 1.72 3.43 7.05 .07 .01 .96 14.23 .01 7.05 .02 6.09 .01 0 3.43 .01 4.11 0 7.05 .01 6.09 .04 .96 .01 .02 6.09 4.11 14.23 .01 .01 7.05 4.11 0 3.43 .01 14.23 .96 .02 7.05 .07 .02 .01 1.72 4.11 .96 .01 3.43 14.23 7.05 .07 4.11 .04 .02 .01 .01 0 .96 .02 3.43 14.23 .01 .02 6.09 3.43 .07 7.05 .01 .01 .01 .02 6.09 .07 3.43 .96 6.56 14.23 .02 0 6.09 .01 4.11 .04 .02 .01 6.09 .96 3.43 .07 6.56 7.05 .01 .96 .07 .96 1.72 14.23 4.11 .01 3.43 6.09 .02 .01 .01 7.05 .01 4.11 14.23 6.09 1.72 4.11 .96 7.05 .02 .01 4.11 .01 3.43 .02 .01 7.05 1.72 6.56 .07 .01 6.09 .04 .96 7.05 .04 .01 .01 0 6.56 .96 .07 .01 14.23 .04 .01 1.72 .02 14.23 6.56 .01 7.05 1.72 6.56 .07 .01 3.43 .04 .02 1.72 7.05 6.09 6.56 .07 .01 .02 4.11 6.09 14.23 .96 7.05 .04 .01 .01 6.09 7.05 .07 .01 1.72 6.09 7.05 14.23 .02 6.56 3.43 .04 .01 0 1.72 .02 .02 0 14.23 4.11 6.09 .01 .01 .02 .96 6.09 6.56 4.11 .01 1.72 3.43 .07 .04 .02 .01 .02 0 3.43 1.72 7.05 .01 .96 3.43 .07 .04 .02 6.09 14.23 .01 .96 .04 .01 6.09 .96 .02 6.56 .07 .01 0 7.05 .01 14.23 3.43 .01 6.56 1.72 6.09 .02 4.11 14.23 .04 .01 6.56 .96 .01 .02 6.09 6.56 7.05 .07 1.72 .04 .01 .02 14.23 3.43 .02 4.11 1.72 .07 6.09 .96 .01 .02 7.05 0 1.72 3.43 .96 .07 .01 4.11 .04 14.23 3.43 7.05 4.11 .02 0 14.23 1.72 6.56 .04 4.11 .07 .04 .01 7.05 .96 6.09 .02 .01 6.56 .07 .01 0 3.43 4.11 .02 7.05 .04 14.23 6.56 .02 .01 7.05 .01 0 14.23 .01 4.11 .07 .04 .02 6.09 .01 4.11 1.72 7.05 .04 .02 6.09 .01 7.05 6.56 .96 .04 .02 1.72 .96 .01 14.23 .01 0 7.05 .02 6.56 6.09 .01 0 7.05 6.56 .01 1.72 .04 4.11 .96 3.43 .01 0 4.11 7.05 1.72 3.43 6.56 0 1.72 14.23 6.09 4.11 6.56 6.09 1.72 .02 .01 .01 0 4.11 7.05 .96 .02 .01 .02 3.43 .02 .02 0 4.11 .04 6.09 .01 14.23 .07 4.11 0 3.43 6.56 .01 .96 .04 .02 0 .96 7.05 .07 .01 .01 .96 .02 .01 0 4.11 .01 6.09 7.05 .04 .02 .02 1.72 .96 7.05 .01 6.09 6.56 .02 .01 .01 0 1.72 .07 .02 .02 14.23 3.43 .01 .96 .02 6.09 .04 .01 .02 6.56 6.09 14.23 1.72 .02 4.11 6.56 .01 .02 6.09 4.11 14.23 .04 1.72 .01 .96 6.56 .07 .01 0 3.43 7.05 1.72 6.56 7.05 .07 .02 0 6.09 .96 14.23 1.72 .04 .02 .01 .02 6.56 0 7.05 3.43 4.11 .96 .02 14.23 .01 3.43 .02 6.09 4.11 .07 14.23 7.05 .02 .01 .01 .02 1.72 14.23 .01 3.43 6.09 .04 6.56 .02 7.05 .96 .04 .01 4.11 .02 14.23 .96 6.09 .01 .01 0 4.11 6.09 .02 3.43 .07 .02 0 4.11 .04 3.43 .01 .96 .07 4.11 .01 3.43 6.56 1.72 0 .96 4.11 3.43 .01 7.05 1.72 14.23 .07 .01 6.56 .01 .02 0 3.43 .02 .01 .02 1.72 4.11 .07 .04 .02 .01 .01 .01 0 3.43 .07 1.72 .01 3.43 .96 .04 4.11 6.56 6.09 .07 .01 0 14.23 .01 .01 1.72 4.11 .02 .96 14.23 7.05 6.09 .01 3.43 .01 1.72 .01 .01 0 14.23 .04 3.43 .07 4.11 1.72 .96 .02 6.56 .04 7.05 .01 3.43 .07 .01 4.11 1.72 14.23 3.43 .07 .04 .02 6.09 14.23 .01 1.72 7.05 .96 3.43 6.56 1.72 .04 4.11 3.43 .01 6.56 .01 6.09 .04 .96 .07 .02 3.43 6.09 .04 .01 1.72 6.56 0 6.09 14.23 6.56 .04 .01 .01 3.43 6.09 6.56 .01 14.23 7.05 6.09 .96 0 4.11 3.43 7.05 .02 .01 0 6.56 .02 1.72 14.23 6.09 3.43 .07 4.11 .01 .01 0 3.43 .04 6.56 14.23 .02 0 3.43 .04 4.11 6.09 .96 1.72 .01 6.56 3.43 7.05 .02 0 14.23 .01 .96 6.56 1.72 .04 3.43 .02 .96 .01 3.43 .01 .02 4.11 6.09 14.23 .04 7.05 4.11 .07 .01 1.72 0 6.56 .96 14.23 7.05 .01 .01 6.56 14.23 3.43 7.05 .02 .01 .01 3.43 6.09 1.72 3.43 .96 .07 .01 4.11 7.05 .02 3.43 6.56 .01 4.11 .04 .02 6.56 1.72 .96 .01 6.09 4.11 .07 7.05 0 6.09 .01 4.11 .02 .01 6.56 .01 4.11 1.72 .07 14.23 .02 7.05 4.11 6.09 .07 .02 .02 6.56 3.43 4.11 .07 .02 .01 .01 1.72 3.43 .02 14.23 .96 7.05 .07 .04 .01 .02 0 1.72 .02 6.56 .02 .96 3.43 .07 1.72 0 7.05 .01 .96 .01 4.11 14.23 7.05 6.56 4.11 3.43 .07 0 14.23 .01 .02 7.05 .01 1.72 14.23 4.11 .01 .96 .01 4.11 .02 6.56 3.43 7.05 .96 .01 0 4.11 .07 14.23 6.56 4.11 1.72 .07 .02 7.05 .96 .02 3.43 14.23 6.09 1.72 4.11 14.23 .07 3.43 .02 6.56 .04 7.05 1.72 .01 6.09 4.11 14.23 3.43 .04 .02 .96 7.05 .07 6.56 4.11 3.43 .02 0 1.72 6.09 6.56 .01 7.05 .04 .01 4.11 6.56 .02 .01 4.11 .07 6.56 .04 .02 .01 7.05 1.72 6.09 6.56 .01 .01 7.05 3.43 6.56 1.72 14.23 .02 .01 .02 .96 1.72 3.43 .01 4.11 .04 .01 .96 .01 6.56 .07 0 1.72 14.23 7.05 3.43 4.11 .04 .02 .01 0 3.43 6.09 .07 .02 1.72 .96 .01 4.11 .02 6.56 6.09 3.43 14.23 .07 4.11 .01 6.56 14.23 .07 .01 4.11 .07 .01 .01 0 7.05 .07 .04 .02 .01 .01 0 6.56 3.43 14.23 1.72 .96 7.05 4.11 3.43 6.56 .07 .02 14.23 .96 6.56 1.72 3.43 .07 7.05 .01 .02 0 6.56 .96 .07 7.05 6.09 14.23 0 .96 .02 4.11 6.56 .04 1.72 14.23 .01 .02 6.56 7.05 4.11 .04 3.43 .02 .01 14.23 .07 4.11 6.09 .07 .01 .02 1.72 .01 .96 3.43 .07 6.56 .07 .04 .02 .01 0 14.23 6.09 4.11 .96 .07 .02 6.09 6.56 .96 7.05 .02 0 .96 4.11 .02 .01 .01 6.09 0 1.72 .07 .01 .01 7.05 .96 6.09 14.23 .07 .02 0 1.72 .04 0 .96 4.11 7.05 .07 .01 14.23 7.05 3.43 .02 .96 14.23 .04 .01 4.11 6.56 6.09 1.72 14.23 .07 .01 .02 4.11 .01 6.56 .96 .07 .02 14.23 6.09 .01 7.05 .01 .02 3.43 4.11 .04 6.09 14.23 .01 1.72 .96 6.56 3.43 .02 1.72 .01 4.11 .04 14.23 6.09 .01 0 7.05 14.23 .01 0 .96 7.05 .01 6.09 0 1.72 .04 .02 .02 4.11 14.23 .01 .01 .02 7.05 1.72 14.23 .02 3.43 .01 7.05 .04 .96 .07 .01 0 14.23 .07 .04 .01 .01 .02 6.56 .96 .04 .01 .01 3.43 7.05 6.09 .02 14.23 .01 .96 .04 3.43 .01 .02 6.09 .96 .01 1.72 6.56 4.11 3.43 .96 14.23 .01 3.43 .02 4.11 6.56 1.72 .01 14.23 3.43 .96 6.09 .07 .04 7.05 .02 .96 3.43 1.72 .07 .01 4.11 .01 .01 .02 7.05 3.43 .96 14.23 .02 .01 .02 6.56 0 1.72 7.05 .96 .07 .02 3.43 0 6.56 .01 .01 .01 1.72 7.05 4.11 .01 .02 1.72 4.11 .07 .04 .02 .01 1.72 .96 4.11 .07 .01 0 3.43 .01 .96 4.11 6.56 6.09 .02 .01 3.43 .96 .01 6.56 1.72 .07 4.11 7.05 .02 .01 .01 .02 14.23 1.72 .96 .01 .01 .01 1.72 7.05 .02 0 4.11 .04 .96 .02 6.09 1.72 .02 3.43 14.23 .07 .04 .01 .01 .96 0 6.56 4.11 14.23 7.05 .04 .01 .02 14.23 .96 6.09 .04 .02 .01 .01 .96 7.05 3.43 6.56 14.23 .07 .02 7.05 .96 .02 14.23 1.72 .01 6.56 4.11 .01 3.43 1.72 .07 .04 .02 14.23 6.56 1.72 .96 .07 .02 .01 0 6.56 .01 .96 1.72 7.05 3.43 .02 0 6.56 14.23 4.11 1.72 .04 .96 3.43 14.23 .04 0 .96 .01 1.72 6.09 14.23 7.05 3.43 .01 14.23 .96 6.56 0 7.05 .07 .02 4.11 .01 0 .96 .07 14.23 1.72 .02 7.05 .02 6.09 .01 14.23 .04 .01 3.43 .96 6.56 .02 6.09 7.05 .04 4.11 .02 .01 .96 14.23 7.05 .04 .02 4.11 6.56 3.43 .04 0 14.23 .01 6.56 .04 .96 14.23 .02 1.72 3.43 .07 .04 .01 .01 0 6.09 .02 1.72 .07 .96 .01 6.56 6.09 .02 0 1.72 .01 4.11 .04 7.05 .02 0 1.72 .07 .01 14.23 .02 .02 3.43 .07 .96 .01 0 3.43 .07 1.72 .01 .96 .02 3.43 .07 .04 .02 0 .96 6.56 .01 6.09 .04 1.72 14.23 .07 .96 .04 6.09 .02 .02 14.23 1.72 .01 4.11 .96 .01 14.23 .02 .01 .01 .02 6.09 3.43 4.11 .07 .01 7.05 14.23 .01 .01 .96 1.72 .04 6.09 .07 14.23 .02 .01 0 .96 3.43 .04 1.72 7.05 0 6.56 .07 3.43 7.05 14.23 6.56 .04 .01 .01 6.09 .07 3.43 .04 .01 7.05 .07 .02 3.43 6.56 .96 .07 .04 .02 6.56 .01 3.43 4.11 1.72 7.05 .01 14.23 6.09 .01 3.43 7.05 1.72 .01 0 6.56 .04 .02 6.56 3.43 6.09 .01 0 .96 .04 7.05 6.09 .96 4.11 14.23 3.43 .04 .02 .01 .01 4.11 6.09 3.43 .01 7.05 .01 1.72 .01 6.09 4.11 .01 .02 1.72 7.05 .02 .01 1.72 .07 .02 .01 .02 .96 6.56 1.72 3.43 .01 4.11 7.05 .01 3.43 .01 0 1.72 6.56 14.23 3.43 7.05 6.09 .96 .02 .02 4.11 7.05 14.23 .04 6.09 .01 .96 4.11 .04 3.43 .01 .02 1.72 6.09 4.11 .04 .01 .01 6.56 7.05 .96 .02 1.72 .07 3.43 .96 6.56 4.11 3.43 7.05 14.23 .07 .96 6.56 3.43 7.05 1.72 .07 .02 6.09 1.72 .01 4.11 6.56 .07 .01 1.72 .04 7.05 .02 14.23 .07 .01 6.56 .01 0 5 10 15 Fluency - Number of correct words Control Treatment Control Treatment Control Treatment Control Treatment RR Acoli Lugbarati Lumasaaba Baseline Endline Correct Words SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 32 Figure 8: Cluster 2, Zero Scores on oral reading fluency at baseline and after three years of intervention, by language Error bars indicate 95% confidence interval C.3 CLUSTER 3 SCHOOLS In this section we present the impact estimations for Cluster 3. The learners in this cluster started P1 in February 2015, when the baseline was collected and received two years of program by the time the endline data was collected. The estimates of the impact of the SHRP program during this year is shown in Table 6. The first row in the table shows the estimated impacts of the program on the letter sound identification sub-task. There are positive effects for Lugwere, Lusoga, Ngakarimojong and English. For the Lugwere subgroup the treatment learners increase their score by almost two letter sounds (0.48 standard deviation) more than their control group counterparts. The effect for Lusoga and English is around 4 letter sounds (0.76 and 0.60 standard deviation) and for the Ngakarimojong subgroup the effect reaches 7.6 letter sounds, which is also 0.76 standard deviation. The estimate is not different from zero for the Lhukonzo subgroup. 1 .68 .71 .66 .82 1 .58 1 .37 .99 .99 .68 .66 .83 .82 1 .58 .99 .99 .68 .99 .82 1 .58 .71 .83 1 .99 .99 .82 .37 .99 1 .83 1 .99 .82 1 .71 .99 .99 .99 .82 .66 .58 .37 .93 .99 .83 1 .66 1 .58 .68 .37 .83 1 .99 .68 .71 .99 .82 .83 .37 .66 1 1 .58 .99 .82 .83 .37 1 .66 .82 1 .37 1 1 .99 .99 .71 .99 .83 .66 .58 .93 .82 .83 1 .99 1 .82 .66 .99 .99 .68 1 .99 .71 .93 .99 .68 .37 .83 1 .99 .99 .82 .68 .93 .83 .58 1 .99 1 .66 .82 1 .71 1 .68 .99 .99 1 .83 1 .99 .93 .99 .82 .66 .37 .99 1 .93 1 .83 1 .66 .99 .58 .99 .37 1 .66 .83 .99 .71 1 .68 .37 1 .99 .83 .82 .66 .93 .58 1 .71 .99 .82 .58 1 1 .66 .93 .99 .58 .68 .83 1 .99 .71 .99 1 .37 .83 .68 1 .82 .99 .99 .58 .99 .66 .99 1 .82 .99 .93 1 .37 .71 .99 .82 .66 .58 .68 1 .99 .37 1 .99 1 1 .82 .99 .99 .83 .71 .99 .66 1 .93 .99 1 .82 .83 1 .68 .66 .71 .37 .93 .83 1 .71 .68 .99 .37 .58 .71 .66 1 .99 .93 .99 .66 .37 .71 .58 1 .99 1 .37 .66 .93 .99 .83 .58 .82 1 .93 1 .99 .71 .68 .99 .82 .83 .99 1 1 .68 .82 .99 .93 1 .66 .58 1 .71 .99 1 .82 .93 .68 .99 .37 .83 1 1 .37 1 .71 .99 .93 .68 .58 1 .37 .83 .99 .68 .99 .99 .66 .93 1 1 .99 .82 .66 .58 .37 .83 1 .99 .93 .82 .71 1 .99 1 1 .66 .71 .83 .58 .68 .99 1 .83 .99 .93 .37 1 .66 .71 1 .99 .99 1 .58 .82 .99 1 .37 1 1 .99 .99 .99 .83 .82 .93 .99 .99 1 .93 .37 .68 1 .82 .58 .99 1 1 .83 .99 .66 .99 .71 .83 .58 1 .99 .82 1 .71 1 .58 .99 .37 .99 .66 1 .99 .99 .93 .58 .83 .82 1 .99 .71 .99 .37 .83 1 .99 1 .82 .99 .99 .68 1 .99 .82 .37 1 .66 .99 .71 1 1 .37 .82 .71 .83 1 .99 .58 1 1 .37 .58 1 .99 1 .66 .71 .68 1 .99 .99 .93 .83 1 .37 .66 .58 .99 1 .68 .82 .71 .83 1 1 .37 .99 .83 1 .58 .71 1 .99 .93 .99 .71 .99 .93 1 1 1 .58 .71 .93 .37 .66 1 .58 1 .99 .82 .93 .66 1 1 .71 .37 .93 .99 1 .82 1 1 .68 1 .99 .83 .66 .99 1 .82 1 1 .58 .99 .66 1 1 1 .71 .58 .99 .83 1 .71 1 .93 1 .99 .99 .58 1 1 .71 .68 1 .99 1 .37 1 .58 .93 .99 .71 .83 1 1 1 .93 .58 .71 1 .93 .37 1 .99 .66 .68 .99 1 .68 .37 .99 .82 1 .83 1 1 .99 .99 .99 .83 1 .58 1 .82 .66 .83 .93 .37 1 .99 1 .58 .93 1 .66 1 .58 .82 .37 .83 .71 1 1 .66 .99 .99 1 .37 1 .66 .99 1 .83 .93 1 .99 .68 .37 1 .83 .66 .58 .82 1 .99 .83 .71 .37 .66 .82 .99 .99 .99 .66 .68 1 .93 .66 .68 .37 .99 1 1 1 .99 .66 .99 .58 1 .68 .83 .99 .37 .99 1 .71 1 .68 1 .99 .71 .83 .99 1 .68 .71 .66 .58 .99 .82 .71 1 1 .83 .58 .66 .99 1 .68 .82 .83 1 1 .71 1 .58 .37 .83 .99 1 .68 .99 .58 .37 1 .58 .83 .66 1 .99 .99 .99 .83 .93 .66 .82 1 .99 1 .37 .83 .99 .66 .68 .71 .99 1 .83 .37 .99 1 .66 1 .82 .83 .99 .93 .71 .37 .82 .68 .99 1 1 .93 .66 .99 .68 .71 .66 .82 .68 1 1 .71 .37 1 1 .83 .99 .58 .93 .99 .71 1 1 .37 .83 .93 .99 .99 .37 .66 .83 .93 .58 1 .99 .99 .82 1 1 .37 .71 .83 .58 .99 .66 .93 1 .99 .71 .68 .99 .93 .37 .99 .58 .82 .66 .83 1 .37 .68 1 .93 .58 .99 .71 .37 .82 .93 1 .83 .99 .66 .58 .99 .99 .83 1 1 .71 .37 .82 .66 1 .83 1 .93 .68 .82 .83 .37 1 .99 1 .71 .83 1 .66 1 .68 .93 .82 1 .99 .99 .99 1 .68 .58 1 .71 .37 .83 .68 1 .99 1 .58 .93 .68 1 .71 .58 .99 1 .37 1 .99 .99 .82 .99 .37 .71 1 1 .99 1 .83 1 .93 .99 .71 .58 .93 .99 .68 .37 .99 1 .93 .99 .68 .71 .82 .58 .66 .99 1 .68 1 .82 1 .99 .66 .82 1 .83 .93 .66 .71 .68 .37 .83 1 1 .66 .99 .99 .93 .37 1 .66 .58 .99 .99 1 .66 1 .99 .83 .82 1 .37 1 .68 .83 .99 .58 1 .66 .93 1 1 .99 .37 .99 .83 .99 1 .71 .99 .82 .68 1 .66 .58 .83 .99 1 .66 .68 .71 1 .93 .99 .99 .82 .99 1 .83 .71 .58 .99 1 1 .66 .99 .82 .68 .71 .99 .99 1 .93 .82 .68 .66 .71 .58 1 1 .37 .68 .99 .93 .99 1 .83 .82 1 .99 .99 .58 1 .66 .99 .83 .99 .37 .68 .82 .83 .71 1 .99 .93 1 .58 .82 .99 1 .66 .71 .58 .99 .99 .82 1 .58 1 .68 1 .99 .99 .99 1 .71 .99 1 1 .83 .93 .37 .68 .99 .71 .99 1 1 .93 .82 .68 .37 .58 1 .71 .82 .83 .99 .37 .71 .66 .83 .82 1 .71 1 .58 .68 .82 .37 .58 1 .66 .83 1 .99 .99 1 .37 1 .99 .82 1 .71 1 .58 .83 .99 .66 .71 1 .93 .37 .71 1 .66 .82 1 .99 .93 1 .99 1 .83 .82 .68 1 .99 .83 1 .37 1 .82 .66 .93 .99 .83 .37 .82 1 .71 1 .66 .83 1 1 .58 .99 .93 .66 1 .71 .82 1 .37 .99 1 .82 1 .93 .71 1 .99 1 .83 1 .66 .37 .58 1 1 .99 .99 .66 1 .68 1 .93 .99 .37 .99 .71 .99 1 1 .82 .66 1 1 .37 .99 .66 1 1 .68 .93 1 .99 .99 .58 .82 1 1 .37 .99 .66 .83 .37 .99 .82 .99 .37 .99 .99 .71 1 .68 .58 .99 1 .82 .66 .71 1 .99 1 .99 .37 .93 .99 .82 .66 .99 1 1 .83 .68 .58 .93 .99 .68 .83 1 .37 1 .99 .66 .99 .99 .83 .68 1 .66 1 .93 .99 .71 1 1 .99 1 .93 .66 .68 .37 .71 1 .58 .83 1 .99 .37 1 .71 .99 1 1 .82 .83 .99 .99 .93 .68 .82 .99 .99 1 .66 .99 .71 1 .82 .37 .58 1 .99 1 .99 .82 .37 .68 1 1 .66 .99 .71 .99 .99 .93 1 .99 1 .83 .99 .71 .68 .66 1 .99 .99 1 .99 .68 .83 1 .71 1 .99 .99 1 .71 .68 1 .58 1 .99 .99 .99 .71 1 .68 .58 .99 .99 .99 1 .82 1 1 .37 .99 .66 .83 .99 .68 .66 .58 .83 1 .99 1 .71 .66 .93 .82 1 .99 1 .93 .99 .82 .58 1 .83 .99 .99 .68 .93 .71 .82 1 .99 .83 1 1 .99 .99 .93 .82 1 1 .99 .99 .58 .93 1 .66 .99 .68 .58 .37 1 1 .99 1 .66 1 .93 .58 .99 .68 .99 1 .83 .66 .99 .99 .93 .99 .66 1 1 .99 .99 1 .83 .68 .37 .66 .93 .99 1 .82 1 .99 .99 .83 .99 .71 1 .99 .58 .93 1 1 .93 .58 .66 .82 1 .99 .99 .37 .68 1 .99 .99 .58 .99 .99 .99 .58 .37 .71 1 1 .66 .82 .93 1 .99 .99 .99 .58 1 .82 .93 .71 1 .93 .99 .82 .99 .66 1 .37 .58 1 .66 .99 .37 .71 .99 .93 .66 1 .99 .93 .99 .99 .99 .37 .82 .93 1 1 1 .83 1 1 .58 .37 .83 1 .66 .82 .93 1 1 .71 .99 .82 1 .58 .99 .71 .99 .66 .68 .82 .37 .99 .99 .66 .99 .58 .83 .71 1 .99 1 .68 .82 .58 .99 .93 .82 1 .99 .68 .37 .83 1 .99 .99 .82 .68 1 .66 .99 .99 .37 .58 .99 .82 .71 1 .66 .37 .93 .71 .99 .82 .37 .58 1 .99 .83 .37 .99 .66 1 .99 .93 1 .37 .58 .99 .99 .93 1 1 .58 .66 .37 .83 .93 1 .99 1 1 .37 .93 .66 .83 .68 .93 .99 .99 .83 1 .68 .58 .99 .37 .66 1 .99 .58 .82 1 .68 .71 .83 .93 .99 .99 1 .68 .99 .37 .71 1 .66 .93 .37 .83 .68 .99 .37 .99 .82 .58 .99 .37 .58 .71 1 .83 .99 .82 1 .71 .99 .99 1 .66 1 .58 .93 .71 .68 1 .37 1 .99 1 1 .66 .37 .71 1 .99 .58 .83 .99 .82 .66 1 .58 1 1 .99 1 .71 .37 1 .99 .99 1 .82 1 .83 .93 1 .66 .93 .83 .37 .71 1 .68 .66 .58 .93 .71 1 .99 .66 .99 .99 .82 .99 .83 .71 .99 .99 .99 .37 1 .66 .68 .58 .99 .83 .58 .99 .66 .82 1 .37 .58 .68 .71 .99 .99 .82 .93 1 .99 1 .37 .58 .83 .99 .37 .99 1 .68 .58 1 .66 1 .99 .37 1 .93 .99 .68 .66 .58 1 .93 .37 .66 .82 1 .99 .99 .66 .58 1 .83 .99 .82 .37 .58 1 .99 .99 .99 1 1 .99 .93 1 .58 1 .83 .66 1 .99 .93 .71 .68 1 .58 .83 1 .71 1 .99 .93 .37 1 .99 .58 .71 .83 .99 .37 1 .93 .68 1 .99 .99 .99 1 .71 .66 .83 .37 .58 1 .68 1 .99 .37 .58 .82 .71 1 .99 .99 .66 .58 1 .68 .58 1 .93 .99 1 .68 .58 .82 .93 1 .37 1 .99 .83 .68 .99 1 .93 .71 .82 .66 .58 .99 .99 .83 .58 1 .99 .99 .82 .99 .93 1 .99 .66 .68 .99 .99 .83 .37 1 .58 .99 .71 .66 .68 .58 1 .99 .83 1 .37 .99 .82 .93 1 .68 .99 .58 .99 .71 .99 .99 1 .66 1 .83 .99 1 .99 .93 .83 .58 .66 .99 1 1 .37 .99 .93 .83 .66 1 .82 .83 .93 .99 .58 .71 .99 .68 1 .37 1 .58 .93 .82 1 .83 .66 .93 1 .99 1 .83 .68 .71 1 .99 .58 .82 1 .93 .83 .66 1 .99 .93 .71 .58 .99 .37 1 1 1 .82 .99 .99 .99 .83 .99 .82 1 1 .99 1 .93 .71 .68 .83 .99 .66 1 .93 .58 .37 .99 .83 .99 .71 1 .99 .58 .66 .83 .99 .71 .37 1 .58 .82 .71 1 .99 .58 1 .83 .82 1 .99 .66 .99 .71 .99 .58 1 1 .37 .82 .66 .93 1 1 1 .37 .99 .82 .99 .68 .37 .58 1 .99 .99 .99 .82 .58 .99 .99 .37 1 .99 1 .66 .99 1 .58 1 .83 .37 .68 .66 .99 .99 1 .82 .99 .99 1 .58 .71 .66 .82 1 .99 .71 .99 1 .37 .66 .83 .99 .58 1 .71 .99 1 .93 .37 .66 .58 .99 1 .93 1 .99 1 .66 .82 .99 .58 1 .99 .99 .99 1 .82 .71 .68 .58 .93 .99 .66 1 .99 .68 .82 .99 1 1 .83 .93 .82 .71 .99 1 .37 .83 1 .99 .82 1 .99 .93 .83 1 1 .82 .58 .99 .66 1 .99 .93 .37 .99 .66 .99 .99 .82 .71 .99 .58 .37 1 .83 .93 1 .99 .58 .71 1 .99 .99 .93 .68 .83 1 .99 1 .93 .99 .37 1 .99 .68 .99 .82 .71 .66 .68 .82 .99 .58 .71 1 .37 1 .66 1 .58 .83 .71 1 .66 .99 .58 .37 1 .82 .66 .99 .99 1 .68 .58 .83 .93 .66 .37 .82 1 .99 .99 .66 1 .58 .71 .99 .99 .93 .66 1 .82 1 .99 1 .71 1 .58 .82 1 .93 .99 .68 .66 1 .93 1 .83 .68 1 .71 .99 .99 .37 1 .99 1 .93 .71 .68 1 .99 .71 .82 .99 1 1 .93 .58 .99 .82 1 .66 .99 .82 .66 1 1 .99 .58 .83 1 .82 .37 .99 .99 .66 .99 .37 .68 .82 .99 .58 .93 1 1 .99 .37 .71 .99 .37 .99 .99 1 .68 1 .99 .99 .99 .66 .93 .83 .82 1 1 .99 .99 .58 .68 .71 .37 .93 .66 .83 .71 .37 1 .99 .99 .83 .99 .82 1 1 .66 .37 .99 .93 1 .99 1 .66 .99 .37 .68 .99 .99 1 1 .68 .58 .83 .71 1 1 1 .68 .99 .93 .37 .83 .66 1 .68 .83 .82 .71 .93 .37 .58 1 .99 .66 .99 1 .37 1 1 .99 .99 .93 .99 .37 .58 .66 .99 .93 .99 .68 .58 1 .99 .99 .71 .66 .83 1 .99 .99 1 .37 .99 .66 .71 1 .99 .37 .99 .83 .99 .37 .66 1 .99 .68 .99 .82 .37 .68 1 .66 .99 .58 1 .68 .99 .99 .99 .37 .83 .68 .58 1 .99 .99 .93 .66 .99 .99 .66 .83 .99 .99 .82 .68 .71 .58 .93 1 .68 .82 1 .99 .99 .99 1 1 .58 .99 .99 .37 .99 .58 .83 1 1 .99 1 .68 .93 .99 .99 .82 .71 .66 1 .99 1 .82 .93 1 .68 .66 1 .99 .99 .99 1 .83 .99 .37 .68 .58 .93 .99 .99 .37 .99 .83 1 .99 1 .66 .99 .83 .58 .99 .82 .68 1 1 .99 .58 .68 .83 1 .82 .37 .66 .99 1 .68 .99 .66 .71 .83 .99 1 .58 1 1 1 .71 .58 .82 .68 .99 .93 .37 1 .99 .58 .82 .71 .66 .99 .37 .66 .99 .71 .93 .68 .58 1 .99 .58 1 1 1 .93 1 .83 .99 .58 .37 .66 .68 .99 .82 1 .71 1 .99 .83 .58 .37 .99 .68 .66 .83 1 1 .68 .99 .82 .37 1 .58 .66 1 .99 .99 1 .68 .58 .71 .93 1 .99 .99 .58 1 .66 .82 .83 .99 .37 .93 1 1 .68 .66 .99 .93 1 .68 .66 .93 .82 1 .68 .99 .82 .71 .66 .83 1 .68 .99 .99 .66 .93 .99 1 .68 .99 .37 1 1 .99 .82 1 .58 .66 1 1 .99 .99 .83 1 .99 .99 1 .99 .99 1 .71 .37 .68 1 1 .83 .99 .82 .66 .58 .83 1 .82 .99 .99 .37 .99 1 .58 .83 .71 .82 1 .99 .37 .99 .68 .58 1 .99 1 .83 .66 .71 .93 .99 .58 .71 1 .82 .99 .37 .99 .71 1 .68 .82 1 .99 .99 .66 1 .93 1 .68 1 1 .66 .93 .71 .68 1 1 1 .71 1 .99 .99 .99 .99 .83 .37 1 1 .93 1 .99 .83 .99 1 .58 .66 .93 .83 .82 .68 1 .99 1 .71 .66 .58 .93 .99 .37 .66 .82 .99 .58 .68 .37 .83 1 .99 .93 1 .99 .99 .83 .37 .99 1 .93 .58 .71 1 1 .82 .71 .99 .93 .37 .66 .58 1 .99 .99 .82 .99 .93 1 .99 1 1 .66 .71 .99 .99 .58 .83 .82 .66 1 .58 .99 .82 1 .99 1 1 .82 .68 .99 .66 1 .93 .99 .99 1 .37 1 .66 .68 1 .99 .99 .99 1 .99 .83 .37 .99 1 .68 .99 .71 .99 1 .83 1 .37 .93 .99 1 1 .99 .82 .66 .37 .99 .68 .58 .82 .99 1 .93 .71 1 .99 .66 .99 1 .83 .58 1 .99 1 .66 .82 1 .37 1 .71 .68 .93 .66 .99 .99 .82 .58 .83 .37 .71 .68 .93 .99 1 1 .99 .82 .58 .99 .37 1 1 .71 .99 .37 .82 1 .93 .66 .71 .82 .99 .99 1 .68 1 1 .37 .93 .58 .82 .99 .66 .83 1 .93 .99 .68 .83 .71 .58 .93 1 .66 1 .99 .99 .93 .99 .82 1 1 .66 .68 .58 1 .99 .66 1 .99 1 .68 1 .99 1 .93 .82 .71 1 .93 1 .83 .82 1 .99 .68 .99 .58 .93 1 .66 .99 .68 .71 .82 1 .71 1 .93 .82 1 1 .83 1 .99 .99 .99 .66 .82 .71 .58 .99 .71 .93 1 .66 1 1 .93 .66 .82 .58 .99 .71 1 .99 1 .58 1 .93 1 .99 .99 .68 .71 .99 .66 1 .99 .99 1 .58 .83 1 .82 .99 .99 .99 .71 1 .93 1 1 .66 .83 .58 .71 1 .99 1 .71 .93 .99 .37 .82 .83 1 1 1 .99 .83 .37 .66 .58 .99 .99 .37 .83 .99 .82 .66 .83 .71 .99 .93 1 .99 1 .66 .68 .58 .83 .37 .71 1 .82 1 .99 .66 .82 1 .99 .58 .99 .71 .83 1 .37 .66 .58 .99 .99 .83 .68 1 .99 .58 .99 .82 .37 .83 .99 .99 .68 1 1 .58 .93 .83 1 .99 .99 1 .68 .58 .66 .71 .83 .82 .93 .99 1 .58 .37 .99 1 .71 .82 .93 .66 .99 1 .99 .99 .99 1 .99 .37 1 .66 .93 .99 .99 .68 .99 .99 .37 .82 .83 .66 1 .99 .99 .82 1 .99 .99 .58 .83 1 .58 .93 .82 .99 .68 1 .99 .71 .83 .66 1 .99 .99 .66 .99 .68 .71 .93 .58 1 .68 1 .37 .99 .83 .99 .82 .99 .68 .66 .82 .71 .99 .37 .58 .99 .82 1 1 .99 1 .37 .93 .68 .58 .66 .82 1 .71 .99 .37 .99 .66 .99 .83 1 .99 1 .83 1 .58 .99 .68 .66 .93 .58 .37 .82 1 .99 .83 .99 .66 .58 .71 1 .66 1 .71 .68 .99 .99 .66 1 .37 .82 .58 .99 .71 1 1 .66 .99 .83 .68 .99 .66 .99 .83 .99 .58 1 .68 1 .37 .82 1 1 .58 .99 .93 .71 1 .99 .68 .71 .99 .66 .99 .99 .82 .71 .99 .99 .58 .66 1 .82 .93 .37 1 .71 .99 .99 .58 .82 1 .99 .37 .71 1 .99 .83 .99 .99 .66 .37 .82 .99 .71 1 .66 .99 .93 .68 .37 .58 1 .71 .99 1 .68 .66 1 .99 .37 .71 .99 .58 .82 .99 1 1 .71 .37 .83 .99 .82 .71 .58 1 .83 .37 .66 .99 1 .58 .66 .93 .99 1 .83 1 1 .99 .99 .71 .82 .58 .99 .66 1 .99 1 .58 .37 .99 .68 1 .66 .83 1 .99 .93 .83 .66 .58 .82 .93 1 .37 1 .99 .82 .99 .68 1 .99 .99 .37 .93 .58 .71 .82 .99 1 .68 1 .99 1 .58 .99 .99 .68 .83 .66 .99 1 .37 .66 .82 1 1 .99 .99 1 .71 .93 .66 .82 1 .99 .68 .99 .99 .37 .93 .99 1 .71 .99 1 .82 .83 .99 .68 .99 .99 .58 .71 .66 1 .82 1 .37 1 .58 .82 .68 .93 .66 .82 .99 .68 .99 .71 .83 .99 1 .99 1 .58 1 .82 .99 .71 .99 1 .68 1 .99 .83 .82 .58 1 .99 .93 .37 .99 .58 .99 .68 .99 1 .82 .99 .71 1 .58 .99 .68 1 .93 .99 1 .68 .71 .37 .99 .99 .58 .71 1 .82 .99 .37 .93 1 .58 1 .99 .99 .83 .71 .93 .99 .82 .37 .58 1 .71 1 .99 .99 .99 1 .93 .99 .82 .37 .99 1 .68 .82 1 .58 .99 .99 .99 1 .68 1 .82 .93 .66 .37 1 1 .68 .99 .71 1 .99 .99 .68 .93 .82 1 .66 .58 .99 .93 1 .93 .83 .37 .71 .99 .82 .68 .99 .99 .99 .58 .99 .71 .37 .68 .83 .71 .93 .58 .99 .99 .71 .99 .82 .99 .99 .58 .83 .66 1 .99 .68 1 .93 .58 1 .99 .99 1 .66 .93 1 .99 .37 1 .99 .83 1 .37 .66 .99 .58 .83 .66 1 .99 .82 1 1 .83 .58 .68 .66 1 .99 1 .71 .68 .37 .93 .58 1 .99 .99 .68 .58 1 .99 .83 .68 .58 .37 1 .66 .82 1 .99 1 .83 1 .99 1 .37 .71 .68 .99 .99 1 .93 .68 .66 .71 .99 .83 .82 1 1 1 .99 .99 1 .82 .83 .58 .99 .93 .82 1 1 .99 .68 .37 .99 .93 1 .99 .68 .93 .99 .66 1 .99 1 .58 .99 .37 .82 .99 .66 .83 .68 .99 .71 .37 1 .99 .66 .93 .99 1 .68 .66 .58 1 .83 1 .99 1 .37 .82 .99 .71 .83 1 .68 .93 .99 1 .58 1 .83 .82 .93 1 .99 .71 1 .37 .82 .58 .71 1 1 .37 .83 .66 1 .71 1 1 .99 .58 .93 .68 .99 .99 .66 1 .99 1 .82 .71 1 .58 1 .37 .66 .99 .99 .58 .99 1 .37 .99 .71 1 1 .99 .68 .99 .71 .83 .58 1 .99 .68 .99 .58 .66 .93 1 .99 .83 .93 .99 .37 .99 1 .58 .99 .66 .68 .99 1 .58 .66 .99 .83 1 .71 .93 .82 .99 1 .71 .58 .83 .82 .66 1 .83 .37 .68 .71 .66 .68 .83 1 .99 .99 .99 .71 .58 .93 .99 .99 1 .82 1 1 .99 .71 1 .68 .99 .37 1 .71 1 .82 .66 .99 .93 1 .99 1 .93 .58 1 .99 .99 .93 .99 .99 1 .71 .99 .68 .58 1 1 .99 .83 .93 .58 .99 .68 .66 .99 .99 .99 1 .83 .99 1 1 .37 .82 .99 .93 .99 .68 1 .99 1 .66 .68 .37 .83 .99 .71 .66 .99 1 .68 .71 .37 1 .83 .99 .93 .66 1 .99 1 .82 .58 .83 .66 .58 1 1 1 .68 .93 .37 .83 1 .99 1 .99 .66 1 .58 .82 .71 .93 1 .37 .99 .82 .99 .68 .71 1 .37 .58 .99 .99 .99 1 .83 .37 .99 .82 .68 1 .66 1 .58 .93 1 .99 .71 .99 .37 .93 .68 .99 .99 1 .71 .68 1 .82 1 .99 1 .71 1 .82 .99 .93 1 .71 .99 .82 .66 .83 1 .93 .71 .82 .99 .58 .83 .37 1 .99 .66 .99 1 1 .82 .99 .99 1 .83 .71 1 1 .99 1 .99 .99 .99 .82 1 .83 .99 .82 .93 1 .71 .66 .68 1 .99 1 .37 .99 .99 .83 .71 .99 .93 .37 .58 .68 .99 .82 .99 .83 .99 .99 1 .37 1 .82 1 .71 .83 .93 1 .66 1 .58 .99 .82 1 .99 .71 .83 .37 .82 1 1 .99 .68 .37 .99 .83 .58 .93 .82 .66 .83 1 .71 .82 .99 .66 .99 .68 1 .93 1 1 .82 .68 1 .99 .83 .66 1 .68 .37 .66 1 .99 .99 .82 .68 .66 .99 .37 .58 .68 .93 1 .71 .82 .58 .99 .99 1 .66 1 .83 .37 .68 .82 1 .71 .99 .99 1 .82 1 .66 .37 .99 1 .82 1 .71 .68 .93 .83 .99 .66 .82 .58 1 .99 .37 .99 .93 .66 .83 1 .82 .99 .93 .99 .82 .99 1 .71 .68 .37 1 .58 .71 1 .99 .83 1 .66 .93 .37 .58 .99 .99 .66 .37 .82 .58 .99 .99 .99 .82 .68 .83 .82 .93 1 .99 .71 .58 1 .82 .66 .99 .71 1 .99 .66 .83 .93 .99 .68 .71 1 .58 1 .68 .99 .71 .99 .99 .66 .99 .71 .83 1 .37 1 .58 .71 .68 1 .99 1 .66 .82 .71 1 .99 .99 .99 .83 .82 .99 .37 .93 .58 1 1 .99 1 1 .83 1 .66 .99 .93 .82 1 .83 1 .58 .99 .93 .99 .71 .37 .58 .66 .71 .82 1 1 .37 .99 1 .58 .99 .83 .37 .71 .99 .93 .99 .71 1 .66 .82 .58 .93 .99 1 .71 1 .37 .66 .71 .83 1 .99 .58 .93 .99 .82 .37 .68 .83 .71 .37 1 .82 .99 .66 1 .58 .83 .99 .68 .71 .37 .82 1 .99 .93 .58 1 .66 .71 .82 1 1 .83 .68 .66 .99 .58 1 .99 .71 .66 .99 .99 .71 1 .66 1 .99 .99 .58 .83 .68 .66 .99 .99 .58 .82 .66 .83 .37 .99 .99 1 .93 .83 .82 .99 .71 1 .99 .93 .99 .66 1 1 .83 .37 .58 .82 .71 1 .99 .99 1 .82 .68 1 .99 .83 .93 .99 .71 1 .66 .68 .82 .37 1 .71 .99 .66 .37 1 .99 .71 1 .99 .99 1 .58 1 1 .99 .99 .99 1 .66 .82 .37 .83 .93 .58 .71 .82 .66 1 1 .37 .93 .66 .83 .82 1 .58 .99 1 1 .66 .93 1 .58 .68 .37 1 .93 .99 .71 .66 1 .83 .37 .99 1 .66 .58 .71 1 .82 .99 .99 .37 1 .71 .68 1 .99 1 .83 .99 .93 .82 1 .66 .99 1 1 1 .99 .37 .68 .71 .93 1 1 .68 .66 .93 .58 1 1 .93 .71 .99 .99 .99 .68 1 .83 1 .99 .99 .58 .93 .68 .37 1 1 1 .83 1 1 .93 .71 .58 1 .99 .37 .58 .82 .99 .93 .37 1 .99 .71 .66 .68 .83 .37 1 .99 1 .71 .99 .66 .93 1 .99 .37 .68 .99 .58 .99 1 .82 .71 1 .68 .37 .99 .83 .93 .66 .82 .99 .83 .99 .71 1 .37 .68 .99 1 .58 .37 .99 1 .93 .58 .99 .68 1 .83 1 1 .99 .71 .37 .99 .99 1 .58 .83 .37 .99 .82 .99 .58 1 .93 1 .99 1 .37 1 1 .99 .99 1 .66 .93 1 .99 .99 .82 .58 .68 1 .37 .99 .93 1 .82 .99 1 .68 .93 .99 .83 .66 .71 .82 .93 .37 .99 .82 .99 .71 .66 .83 .99 .37 .82 .93 .68 1 1 .58 1 .93 .82 .83 1 .99 .71 .99 .99 1 .58 .82 .93 .37 .99 .99 1 .66 1 .83 .58 .93 1 1 .82 1 .66 .99 .99 .99 .83 .58 .71 .99 1 .83 .71 1 1 .99 .99 .83 .93 .71 1 .99 1 .82 .99 .93 .71 .66 .68 .99 .99 .82 .93 .99 .66 .83 1 .71 .58 .99 .99 .99 1 .37 .83 .93 .99 .99 .99 .83 .58 1 .99 .71 1 .93 1 .68 .83 .99 .82 .37 1 1 .99 .99 .93 1 .66 .71 .37 .58 1 .99 1 .37 .93 .68 1 .99 .99 .99 .93 .58 .82 .66 .37 1 1 .58 .93 .99 .37 .83 .99 .66 .71 .99 .82 .83 1 1 .99 .37 .66 .83 .93 1 .99 .99 1 .66 .99 .93 .83 .58 .82 1 .99 .66 .37 .71 .83 1 .93 .82 .37 1 1 .93 .99 .83 .68 .37 .58 .82 .99 .37 .93 .66 1 .58 1 .99 .71 .99 1 .93 1 .37 .83 .99 .58 1 .68 .99 .37 1 .99 .82 .93 .66 .99 .68 .58 1 .71 .99 .99 .93 .37 .58 1 1 .71 .66 .82 1 1 .37 .99 .66 1 .93 .37 .99 .83 .82 1 1 .99 .99 1 .68 .99 .83 1 .93 .99 .66 .68 .99 1 .83 .99 .71 1 .58 1 1 .83 1 .99 .37 .99 1 .82 1 .93 .99 1 .82 1 .83 .99 .93 .99 .82 1 1 1 1 .93 .66 .99 .68 1 .83 .37 1 .93 1 .68 .99 1 .37 .83 .99 .71 .93 .99 .37 .99 .99 .99 1 .68 .82 .71 1 .99 .58 .37 .99 .99 .93 .83 1 .68 1 .37 .99 .99 1 .93 .82 1 .83 .58 1 .66 1 .82 .58 .37 .66 1 .99 .99 .68 1 .82 1 .99 .58 1 1 .82 .66 .93 1 1 .99 .66 .99 .82 .71 .83 .58 .99 .37 .68 .99 .82 .58 .83 .99 1 .66 1 1 .66 .82 .68 .99 1 .93 1 .58 .68 .93 .71 .37 .82 1 .99 .99 .99 .71 .68 .82 .99 .58 .99 .83 .99 .68 .71 .99 1 .83 .58 .99 .99 .83 1 .99 1 .99 .93 .66 .83 .82 .99 .71 .58 .99 .82 .99 1 .83 .66 .37 .82 .58 .68 .93 .99 1 .71 .58 .37 1 .68 .99 .93 .71 1 .82 .99 1 .83 .68 .71 1 .99 .99 .66 .58 .93 .99 .83 1 .82 .93 .66 .71 .82 .58 .37 1 .93 .66 .82 .58 .83 1 1 .68 .83 .99 .71 .66 1 .99 .83 1 .58 1 .37 1 .99 .66 .99 .4 .6 .8 1 Fluency - Proportion with zero correct words Control Treatment Control Treatment Control Treatment Control Treatment RR Acoli Lugbarati Lumasaaba Baseline Endline Proportion with no correct words SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 33 Table 6: Cluster 3 SHRP School Level Effect (1) (2) (3) (4) (5) VARIABLES English Lugwere Ngakarimojong Lhukonzo Lusoga Letter Sound Score 4.030*** 1.995* 7.626*** 0.019 4.542*** (0.588) (0.753) (1.356) (0.777) (0.891) % Letter Sound Zero Score -0.219*** -0.029 -0.005 0.022 -0.159*** (0.038) (0.063) (0.074) (0.065) (0.043) Segmenting -0.169 0.881* 0.429 0.250 (0.158) (0.388) (0.319) (0.313) Fluency 0.044 2.630+ 0.733 0.389* (0.255) (1.434) (0.608) (0.172) Reading Comprehension -0.008 0.299+ 0.088 0.040+ (0.051) (0.168) (0.060) (0.022) Listening Comprehension -0.010 -0.162 -0.030 -0.072 (0.112) (0.122) (0.096) (0.148) English Receptive Vocabulary -0.005 (0.254) Robust standard errors in parentheses. *** p<0.001, ** p<0.01, * p<0.05, + p<0.1All regressions include school fixed effects, individual controls and a constant term. Individual controls include age and sex of learner, dummy for age missing, score on household assets index, dummies for student living with both parents, someone at home reading to the student and language spoken at home. Regressions for English language include control for local language. The second row in Table 6, presents the estimated effect of SHRP on the proportion of learners unable to correctly identify a single letter sound. The program had a reduction effect in English and Lusoga but not in other languages. Figure 9 shows the raw data on letter sound scores at baseline and in October 2016 for treatment and control groups for all local languages in Cluster 3. Despite the fact that the effects of SHRP are not trivial, in absolute values, the performance of most language subgroups –the exception is Ngakarimojong- in this subtask is very low for learners at the end of P2. Figure 10 shows that the evolution of SHRP effects on letter sound identification since baseline. It can be seen that after a very modest start, the effects are more encouraging at endline for all languages with the exception of Lhukonzo where the treatment group does not perform better than the control. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 34 Figure 9: Cluster 3, Letter Sound Scores at baseline and after one year of intervention, by language Error bars indicate 95% confidence interval Figure 10: Cluster 3, SHRP estimated effects on letter sound identification since baseline, by language *** p<0.001, ** p<0.01, * p<0.05, + p<0.1Estimates from OLS regressions including school fixed effects, individual controls and a constant term. Individual controls include age and sex of learner, dummy for age missing, score on household assets index, dummies for student living with both parents, someone at home reading to the student and language spoken at home. Regressions for English language include control for local language 2.48 .78 .76 7.11 .76 7.11 2.48 .78 2.48 .78 2.48 7.11 .76 7.11 .78 .76 2.48 .78 2.48 .78 2.48 .78 2.48 .78 2.48 .78 2.48 .76 7.11 .76 7.11 .76 7.11 .76 7.11 1 3.46 1 3.46 1 3.46 3.84 1.53 1 3.46 1 3.46 3.84 1.53 3.46 1 3.46 1.53 3.84 1.53 3.84 1.53 3.84 1.53 3.84 1.53 3.84 1.53 3.84 1.53 3.84 1.53 3.46 1 3.46 1 3.46 1 3.46 1 3.84 1.53 3.84 1.53 13.86 1.45 13.86 1.45 13.86 1.45 13.86 1.45 5.57 .44 5.57 .43 2.1 .43 2.1 .43 2.1 .43 2.1 4.12 .43 4.12 .43 2.1 .43 2.1 .43 2.1 4.12 .43 4.12 .43 4.12 2.1 .43 2.1 .43 2.1 .43 2.1 .43 2.1 .43 4.12 .43 4.12 .43 4.12 .76 7.11 .76 7.11 .76 7.11 3.84 1.53 3.84 1.53 1 3.46 1.53 3.84 5.57 .44 5.57 13.86 1.45 5.57 .43 .44 4.12 .43 4.12 .44 5.57 1.45 13.86 1.45 .44 5.57 .43 2.1 .43 0 5 10 15 Number of correct letter sounds Control Treatment Control Treatment Control Treatment Control Treatment Lugwere Ngakarimojong Lhukonzo Lusoga Baseline Endline Letter Sound Identification 0.0 1.0+ 4.5** 0.4 1.2* 2.0* 7.6** 0.2 0.0 0.6* 4.0** 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 Baseline At P1 At P2 Difference in Letter Sound Identification Lusoga Lugwere Ngakarimojong Lhukonzo English SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 35 For Cluster 3, we find very few and small effects in the more advanced reading subtasks, even that the learners were assessed at the end of P2. SHRP had a positive impact on the segmenting subtask only in Ngakarimojong. Learners in treatment schools scored on average 0.88 words more than those in control schools (around 0.3 standard deviation). In terms of oral reading fluency, we find positive effects among Ngakarimojong and Lusoga learners but not among other languages. The effect is an advantage of 2.6 words in Ngakarimojong (0.3 standard deviation) and 0.4 words for the Lusoga subgroup (0.2 standard deviation). Figure 11 shows the number of words that learners in each subgroup are able to read in a paragraph. We show the data for treatment and control groups at baseline and at the end of P2, after two years of program intervention. Clearly, the learners cannot read and the vast majority scored zero in this subtask. The average number of correct words read from a paragraph was 3.4 for the treatment Ngakarimojong subgroup, which by far performed the best. Not surprisingly the reading comprehension levels are very low, given that most students cannot read a single word. We only find small effects of the program - 0.3 and 0.04 correct answers (0.33 and 0.18 standard deviations) - among the Ngakarimojong and Lusoga students, respectively, which are only statistically significant at 10 percent. We find no effect among the rest. Finally, no program effect was identified in the listening comprehension subtask for any language or in the English receptive vocabulary subtask. Figure 11: Cluster 3, Oral reading fluency at baseline and after two years of intervention, by language Error bars indicate 95% confidence interval 0 .05 0 .45 0 .45 0 .05 .45 0 .45 0 .05 0 .45 0 .45 0 .45 0 .45 .02 1.26 .02 1.26 .02 1.26 1.82 .02 0 1.26 .02 1.26 1.82 0 1.26 .02 1.26 0 1.82 0 1.82 0 1.82 0 1.82 0 1.82 0 1.82 0 1.82 0 1.26 .02 1.26 .02 1.26 .02 1.26 .02 1.82 0 1.82 0 3.39 0 3.39 0 3.39 0 3.39 0 1.15 0 1.15 .01 .39 .01 .39 .01 .39 .01 .39 .44 .01 .44 .01 .39 .01 .39 .01 .39 .44 .01 .44 .01 .39 .44 .01 .39 .01 .39 .01 .39 .01 .39 .01 .44 .01 .44 .01 .44 0 .45 0 .45 0 .45 1.82 0 1.82 .02 0 1.26 0 1.82 1.15 0 1.15 3.39 0 1.15 .01 0 .44 .01 .44 0 1.15 0 3.39 0 0 1.15 .01 .39 .01 0 1 2 3 4 Fluency - Number of correct words Control Treatment Control Treatment Control Treatment Control Treatment Lugwere Ngakarimojong Lhukonzo Lusoga Baseline Endline Correct Words SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 36 C.4 IMPACT ON BOYS VS. GIRLS In general, we find that the reading performance of boy and girls is quite similar. We present the detail of scores for the letter sound and fluency subtasks for each of the clusters below. In Figure 12 we show that, for cluster 1, there are no statistically significant differences between boys and girls in the letter sound identification subtask. When comparing fluency, it can be seen that girls perform slightly worse in the case of Leblango -15.9 words for boys vs. 13.5 for girls￾and the difference is statistically significant at conventional levels. In the rest of the languages in this cluster, the differences are not significant. Finally, the effect that SHRP has on girls is not different from the effect on boys: no significant differences in the impact of SHRP by gender were identified. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 37 Figure 12: Cluster 1, Letter sound score and oral reading fluency, treatment group at endline, by gender ◊ significant difference, p<0.05 For cluster 2, we show, in Figure 13, that girls perform better than boys in the case of the letter sound identification subtask in Acoli and Runyoro-Rutooro. We also find that the effect of SHRP on the letter sound subtask is differentially larger for girls in the same two languages. There are no significant differences by gender in the number of letter sounds correctly identified for the other languages and there are not differences in levels of fluency for any language. The effect of SHRP is also similar for boys and girls in those cases. 17.9 18.2 10.8 10.1 14.8 15.8 12.1 11.7 0 5 10 15 20 Number of Correct Letter Sounds Treatment Luganda Leblango Ateso Runyankore/Rukiga Letter sound score by gender Male Female 18.6 21.6 15.9 13.5 12.1 13.1 25.4 27.2 0 10 20 30 Fluency - Number of correct words Treatment Luganda Leblango Ateso Runyankore/Rukiga Fluency by gender Male Female SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 38 Figure 13: Cluster 2, Letter sound score and oral reading fluency, treatment group at endline, by gender ◊ significant difference, p<0.05 We finally compare in Figure 14 the performance of boys and girls in letter sound identification and in fluency subtasks for Cluster 3. The only statistically significant difference is found in Lhukonzo letter sound identification levels, where, on average, girls score almost one letter sound more than boys. The effect of SHRP on letter sound identification in Lhukonzo is larger for girls as well but this is not the case in any other language in this cluster. Summarizing, reading performance levels are very similar between boys and girls during the early school grades. SHRP impact is similar for boys and girls, as well. This is, the program does not favor a gender in particular. In general, the interaction between SHRP treatment and gender is not different from zero. 7.8 10.1 13.5 13.2 11.0 12.1 14.1 16.4 0 5 10 15 20 Number of Correct Letter Sounds Treatment Acoli Lugbarati Lumasaaba Runyoro-Rutooro Letter sound score by gender Boy Girl 3.6 3.3 6.8 6.3 3.7 4.5 13.7 14.7 0 5 10 15 Fluency - Number of correct words Treatment Acoli Lugbarati Lumasaaba Runyoro-Rutooro Fluency by gender Boy Girl SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 39 Figure 14: Cluster 3, Letter sound score and oral reading fluency, treatment group at endline, by gender ◊ significant difference, p<0.05 3.9 4.3 14.7 12.8 3.4 4.3 6.9 7.4 0 5 10 15 Number of Correct Letter Sounds Treatment Lugwere Ngakarimojong Lhukonzo Lusoga Letter sound score by gender Male Female 0.5 0.4 4.6 1.9 1.5 2.1 0.5 0.4 0 1 2 3 4 5 Fluency - Number of correct words Treatment Lugwere Ngakarimojong Lhukonzo Lusoga Fluency by gender Male Female SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 40 In summary, the findings of the impact analysis show that estimated effects of SHRP on reading skills among primary school students are quite heterogeneous across languages and across tasks; some languages show positive and statistically significant effects in certain EGRA subtasks while other languages do not. Overall, the analyses suggest that the program has some positive and sometimes sizeable impacts but also several cases where no improvement can be detected. However, even where we see statistically significant improvements, absolute levels of reading competence remain, in general, very low. C.5 FACTORS THAT MAY BE HINDERING SHRP IMPACT ON READING PERFORMANCE In response to the findings of the impact evaluation that showed heterogenous and often small impacts of the SHRP interventions and the persistently low levels of reading competence across almost all languages, NORC proposed additional qualitative research to explore possible causes of poor reading performance among students despite multiple years of focused SHRP interventions. Four potential explanations were identified through key informant interviews and FGDs: 5. Teacher trainings fell short of fully training and/or motivating teachers to implement SHRP techniques in their classrooms; 6. Teacher support supervision did not provide sufficient support to ensure that teachers were able to implement technical effectively in the classroom; 7. Community mobilization efforts were insufficient to complement classroom efforts at home; and 8. High absenteeism among learners and teachers means that they do not meet regularly enough for training, coaching, and learning materials to take effect. These explanations initially emerged during performance evaluations that the NORC team conducted during Year 2 and 4 of the P&IE project. They were discussed and validated during dissemination workshops with local stakeholders. In Year 5 of the P&IE projects, NORC further explored these posited reasons for low reading performance among students in SHRP schools in a series of focus group discussions (FGDs) with teachers who had benefited from SHRP trainings and instructional materials. Below, we summarize the principal findings with illustrative quotes from the FGD. More detail can be found in the full report in Annex F. Teacher training In general, teachers stated that the content of SHRP training was comprehensive but too few days were allocated so that they did not acquire full and sufficient understanding of the techniques being taught. “The facilitators were a bit fast … which was a challenge to the teachers. Most of them did not capture all that was taught during the main trainings and the refresher trainings.” (Pader)9 Teachers found the training to be helpful with respect to letter sounds and pronunciation; developing lesson plans; learning about different types of texts; and in oral literacy, mentioning, for example, the usefulness of having children read news in front of others to gain confidence. 9 Quotes from FGDs are shown in italics followed by the teacher’s district listed in parentheses. All quotes are from teachers who had participated in SHRP trainings. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 41 Most teachers felt that their own pronunciation of letter names and the letter sounds had improved. “I learnt to use proper sounds and teach my pupils correct syllables and illustrations.” (Kamuli) “SHRP books are good for reading and writing…They are also good in helping teachers follow steps in teaching.” (Kamuli) On the other hand, teachers identified a number of weaknesses in teaching materials, commenting in particular that there is too much material for students, that some material is too difficult for the learners and that there are errors in the books. They also believed there were not enough teaching materials. “They have some missing content… They are too brief and not well articulated in grammar, … they are limited in vocabulary…. They do not want us to use any other books, they want us to use their books only.” (Kamuli) “They should also bring more books because…given that we have many children, you can find that all pages are missing and it starts from page 30; so I can’t know where to start from when teaching.” (Kiruhura). Teachers learned methods such as “I do, we do, you do;” and the importance of making lesson plans. They also learned how to foster cooperation and group work discussion and how to use different learning materials from everyday life, play time, and pictures in lessons and how to assess learners. “Assessment helps much in teaching our pupils.” (Kiruhura) But teachers also commented that the new methods require more time, and cannot be completed in the tim available to them. Applying the methods in large classes can be very difficult. “Class control becomes a problem. …When you have a class of 50 pupils, you can’t put them in one group, there should be five groups of ten.” (Kamuli) Most teachers felt they were not well equipped in some areas due to missing content as well as having insufficient time for the training. Some also mentioned facing unforeseen challenges. “We were able to understand what we were taught during the training but when you reach in the classroom you find that the children are coming from the village and … you find the child is reading the book upside down. When you write a letter on the blackboard, the children will not be able to identify it in the book, since the book is upside down, hence a challenge.” (Kamuli)… Teacher support supervision All teachers had received some support supervision, usually 2-3 times per year, mostly from CCTs, head teachers, or deputy head teachers, and in Pader sometimes from parents. Pader was also unusual in that support supervisors sensitized parents about the relevance of SHRP, which in turn improved class attendance. The support included assessment and motivation of the teacher and pupils, provision of teaching materials, guidance on teaching, class control, and lesson planning. Some teachers also mentioned receiving visits by SHRP staff to conduct assessments and give the children small gifts as motivation. Several teachers reported that support supervision resulted in motivation, confidence building, and friendly guidance. Some teachers felt their skills were enhanced and that supervision SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 42 helped them correct their mistakes. But they also said the support visits were not frequent enough, and also complained that the visits were too short. “The support supervisors give us little time; they come at 2:00PM and leave by 3:00. They don’t give us enough time to help and guide us.” (Kiruhura) Teachers also complained about the lack of direct response to requests for guidance, as well as unprofessionalism and even humiliation by the supervisors. “A teacher is not supposed to be harassed, but sometimes the support supervisor comes in such a way that demoralizes the teacher.” (Kiruhura) Community Mobilization / Involvement Teachers were asked in particular about parental involvement. Most teachers reported that there was limited involvement by parents in the SHRP program, due to insufficient knowledge about the program, negative perceptions about the children being taught in the local language, and inadequate mobilization of the community. “Some parents … had a negative attitude towards teaching their children using local language and preferred their children being taught in English.” (Pader) One teacher mentioned “since they introduced SHRP the number of children [coming to school] has dropped.” (Kiruhura) “Parents should conduct school tours and practically see what their children are learning. Parents will know that this is not a local language program but that pupils are learning and reading well.” (Pader) All teachers agreed that it was necessary to involve the parents so parents would understand the benefits and support their children. Teachers reported a few positive effects. “There was need to involve the community because since we started teaching pupils in their mother tongue it has eased communication between our pupils and their people at home. They can easily understand each other instead of talking to them in English.” (Kamuli) “Active participation of parents in guiding their children to read, the right pronunciations of works, the accent.” (Pader) Absenteeism Learners’ and teachers’ absenteeism can be negatively associated with school performance. Previous work in Uganda and other countries have established this link10. Student absenteeism in Uganda is very high. Table 7 shows the proportion of learners that reported having been absent at least one school day in the previous school week by language among the treatment schools at endline11 . The percentages reported are very high in all the language subgroups 10 See for example, Najjumba, Habyarimana, and Bunjo, 2013 Improving Learning In Uganda, Volume 3: School-Based Management: Policy and Functionality. Washington DC. World Bank. 11 Percentages are similar for control schools SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 43 Table 7: Learner Absenteeism in the Previous Week, SHRP Schools at Endline, by Language Language Percentage Luganda 49.5% Leblango 64.0% Ateso 59.4% Runyankore/Rukiga 49.1% Acoli 42.8% Lugbarati 49.3% Lumasaaba 57.3% Runyoro/Rutooro 61.8% Lugwere 53.3% Ngakarimojong 54.9% Lhukonzo 49.3% Lusoga 45.0% Teacher absenteeism is also very high. A World Bank study by Wane and Gayle12 reports the results of a study that visited Ugandan public schools unannounced. On average, they found that less than 40 percent of the teachers were actually in the classroom teaching learners. Twenty-seven percent of the teachers were not present in the school, an additional 30 percent were at the school but not in the classroom, and, at least 6 percent were in the classroom but not teaching. Learners in our study not only reported on their own absenteeism but also their teachers’. Specifically, they were asked whether there was any day in the last week when their teacher was not present. Unfortunately, the interviewed learners in each class did not report consistently if the teacher had been absent in the previous week; instead there were some discrepancies between students. Therefore, we only consider a teacher as absent if more than 50 percent of the interviewed children in her/his classroom reported her/him as such. If the majority of learners reported that the teacher was absent in the previous week, we coded him/her as absent and otherwise are present. In Table 8 we show this proxy variable for SHRP schools at endline, by language. Figures indicate that absenteeism among teachers is high and consistent with previous studies. Table 8: Estimated Teacher Absenteeism in the Previous Week, SHRP Schools at Endline, by Language Language Percentage Luganda 35.1% Leblango 33.6% Ateso 29.6% Runyankore/Rukiga 41.3% Acoli 39.1% Lugbarati 21.7% Lumasaaba 40.7% 12 Wane, Waly and Gayle H. Martin "Education and Health Services in UGANDA Data for Results and Accountability Service Delivery Indicators, Education and Health" Nov. 2013, World Bank SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 44 Runyoro/Rutooro 45.6% Lugwere 55.9% Ngakarimojong 35.1% Lhukonzo 56.9% Lusoga 63.3% NOTE: Teacher is considered absent if more than 50% of the learners in the classroom report so. Absenteeism occurs in both treatment and control schools, and in both groups it tends to depress learning and reading performance, but in addition, given that a large proportion of the SHRP treatment is delivered to the learners at the school and through the teachers, we hypothesize that absenteeism could be reducing SHRP’s potential impact. To test this hypothesis, we estimated SHRP effects as before but now we add learner (teacher) absenteeism and explore the interaction between learner (teacher) absenteeism and SHRP treatment13 . In Table 9 we show the SHRP estimated effects on reading fluency for Cluster 1 including learner absenteeism and the interaction between SHRP and learner absenteeism. Table 9: Cluster 1, Estimated Effects of SHRP on Oral Reading Fluency: Interaction with Learner Absenteeism Runyankore Rukiga Luganda Leblango Ateso SHRP Treatment 6.828 4.546+ 0.876 7.136*** (2.267) (2.208) (2.312) (1.560) SHRP Treatment * Absent Learner -0.675 -0.012 -0.574 -1.521 (1.550) (1.695) (1.982) (1.410) Absent Learner -2.202*** -0.979+ 0.374 -0.174 (0.446) (0.489) (0.599) (0.488) F-test: joint significance (p-value) 13.38 (0.0000) 2.60 (0.0745) 0.17 (0.9172) 9.83 (0.0002) Observations 1,281 1,137 1,452 1,559 R-squared 0.570 0.411 0.300 0.219 Robust standard errors in parentheses. Individual controls include age and sex of learner dummy for learner absent previous week household assets index indicators for living with both parents someone at home reads to the learner language spoken at home district dummies and a constant term. It can be seen that the interaction between SHRP treatment and learner absenteeism is always negative suggesting that the effect of SHRP is different depending on learners’ school attendance, and that it tends to be lower for those that are absent more frequently. Results for Cluster 2 and 3 also show a negative interaction between treatment and student absenteeism (see Annex E). Of course, learner absenteeism can be related to other factors, for example health status or number of siblings. However, we control in this analysis for several characteristics that could be cofounding the effect (being a girl, age, household assets, etc.) and it is credible that those not showing up to school cannot fully benefit from a treatment that is mostly delivered at school. 13 We add an interaction term between SHRP Treatment and learner (teacher) absenteeism and a learner (teacher) absent dummy to equation (1) SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 45 In Table 10 we show the SHRP estimated effects on oral reading fluency for Cluster 1 but including teacher absenteeism and its interaction with the SHRP treatment. The results are a bit less clear than in the case of learner absenteeism. The interaction term is negative for Runyankore/Rukiga and Luganda but not for Leblango and Ateso so conclusions are less clear. It is possible our approximation of the teacher absenteeism variable is not good enough for this kind of analysis or it could be that teacher absenteeism is not creating a differential effect. This last alternative seems unlikely, given the fact that the main component of the SHRP treatment requires a teacher present in the classroom to implement it. Table 10: Cluster 1, Estimated Effects of SHRP on Oral Reading Fluency: Interaction with Teacher Absenteeism Runyankore Rukiga Luganda Leblango Ateso SHRP Treatment 6.632* 5.734* 0.894 6.290** (2.983) (2.589) (2.040) (1.767) SHRP Treatment * Absent Teacher -2.673 -4.726+ 0.109 0.479 (4.281) (2.635) (3.001) (2.858) Absent Teacher -2.554+ 0.992 -3.073* -0.856 (1.393) (2.228) (1.126) (0.837) F-test: joint significance (p-value) 4.11 (0.0164) 1.80 (0.1722) 3.83 (0.0226) 8.02 (0.0006) Observations 1,294 1146 1,468 1,570 R-squared 0.571 0.415 0.306 0.219 Robust standard errors in parentheses. Individual controls include age and sex of, household assets index, indicators for living with both parents, someone at home reads to the learner, language spoken at home and a constant term. If we add late start and early end of the academic terms to learner and teacher absenteeism, it is not surprising that the level of competence of the students is low, despite the concentrated SHRP interventions. We do not have more than anecdotal evidence about late start and early finish of the school year, but we have been systematically alerted to the difficulties of collecting data too early or too late into the academic year because many learners cannot be found in the classroom. Combining reduced teaching time due to student and teacher absenteeism, and the later start and early end of the school year, the net amount of instructional time is a fraction of the intended time that teachers trained and equipped with materials through SHRP are expected to teach their students. In addition to time on task in school, reading behaviors at home can be an important determinant of reading achievement. We show in Table 11 some variables related to reading behavior at home: in our survey, learners report if anyone at home reads to or with them and whether they bring reading materials from school (readers or books from the library, for example) to read at home. The percentages in the table indicate that, in general, there are not important differences between learners in treatment schools and learners in control schools when it comes to reading behavior at home. It is possible that SHRP communication campaigns and activities about the importance of early reading have reached treatment and control communities equally. There is one exception: a higher percentage of learners in cluster 1 - Luganda, Leblango, Ateso and Runyankore/Rukiga- treatment group report that someone reads with or to them at home, so it is possible that local community and parent mobilization in those areas has successfully increased awareness. Finally, it is interesting to note that a very large SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 46 percentage of learners in the Runyankore/Rukiga subgroup report bringing reading materials home, an outlier compared to any other language subgroup. This is also the subgroup that show the highest average fluency among the P4 (Cluster 1) group. Table 11: Reading at Home and Bringing Reading Books from School to Home, at Endline and by Language Language % Learners that report someone reads to or with them at home % Learners that report bringing home reading books from school Treatment Control Treatment Control Luganda 72.4% 53.4% 33.3% 36.5% Leblango 51.1% 30.9% 31.1% 29.8% Ateso 48.6% 33.8% 37.3& 31.1% Runyankore/Rukiga 63.3% 52.8% 92.4% 90.6% Acoli 46.5% 41.0% 20.4% 15.6% Lugbarati 52.2% 59.0% 41.7% 48.1% Lumasaaba 57.7% 58.4% 49.3% 51.1% Runyoro/Rutooro 66.2% 63.2% 34.0% 34.8% Lugwere 52.0% 53.3% 46.6% 48.3% Ngakarimojong 32.8% 36.7% 33.7% 36.1% Lhukonzo 60.2% 54.0% 60.7% 66.5% Lusoga 68.7% 68.8% 58.6% 56.2% Complementing this quantitative analysis, findings from the FGDs shed additional light on teacher and student absenteeism. According to FGDs with teachers, absenteeism among teachers is not infrequent with the principal causes for absenteeism cited as sickness, long distances to schools, weather, lack of motivation, low salaries and delayed paydays, and family responsibilities. The long distance between home and school was among the most common reasons given. Teachers often do not have adequate means of transport. This can be compounded by the weather, especially in the rainy season when many roads become impassable. “The unpredictable weather sometimes affects teachers coming to school. This is very common during the rainy season…. It is sometimes also attributed to bad nature of the roads to the school; when it rains, they cannot access school….” (Kamuli) Sickness of the teacher or a family member, or death of a relative were also mentioned as frequent causes, along with other family responsibilities. However, teachers also mentioned lack of motivation and limited or irregular supervision, as other deterrents to attendance. Some mentioned that no breakfast and lunch are provided for staff and going to get lunch or dinner can take a long time. Delay in salaries was the leading reason given for absenteeism, and was mentioned in the FGDs in all three regions. When salaries are delayed, the teachers stated they have to look for alternative sources of income; some mentioned driving boda boda (motorcycles), farming, and other commercial activities to raise income to take care of the families. Some said that it was common toward the end of the month to stop going to school for these reason. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 47 “Most teachers would love to be in schools to teach but when salaries delay, they are forced to think of alternative ways of sustaining their families.” (Kiruhura) “We should be paid enough salary on time not only to motivate us but also save us from the worries which distract us from focusing on teaching these pupils. For example one may worry about loan officers coming for him or her.” (Kiruhura) Among the deleterious effects of absenteeism, teachers mentioned poor performance among students, greater student absenteeism, lack of discipline, and increased dropouts, due to more crowded classrooms, unfamiliar replacements, and an incomplete syllabus. “Absenteeism affects the pupil’s learning to read. …. It interrupts the content or flow of information and makes teachers to start rushing to finish the syllabus and thus rush pupils who in the end are half baked.” (Pader) Teachers in FGDs identified pupil absenteeism as a major cause of poor performance and listed a number of causes, many related to child labor in economic/agricultural activities. Absences increase with drought as children or families need to go farther away with animals to try to find water and grass, and in the rainy season, when children have to stay home to cultivate crops while they can. Children are also sent to burn charcoal, to markets to sell or buy items and to cut sugarcane. Another reason for absenteeism noted in all the study districts was parents’ negative perceptions of SHRP as they believe learning in the local language is not useful. They, therefore, keep children away from school or put them in private schools, according to FGD participants. Teachers also mentioned corporal punishment as another main causes of pupil absenteeism, with children too afraid to come to school. “Let us say it is a P1 pupil who makes a mistake and you give her a hoe to dig around the school, the child will stop coming to school; like 10 strokes I think that child will be absent the next day.” (Kiruhura) Other reasons mentioned included parental neglect, walking long distances to schools, and a lack of privacy and hygiene concerns for female pupils during menstruation. All teachers saw a clear connection from absenteeism to poor performance and dropping out. Even if the children come back to school, having missed a foundational concept means that it is difficult to catch up. All teachers reported having attempted talking to parents about the problem. “I have taken time to talk with parents and to some it has made a difference because some parents understand the value of education.” (Pader) Some parents, however, reacted negatively and there was no change. Some teachers reported parents threatening them or enforcers with beating. In the same way we explored the interactions of the SHRP treatment with learner and teacher absenteeism, we explore here the interaction between SHRP treatment and having someone reading to or with the learner at home. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 48 Table 12: Cluster 1, Estimated Effects of SHRP on Oral Reading Fluency: Interaction with Reading at Home with Someone Runyankore Rukiga Luganda Leblango Ateso SHRP Treatment 5.216* 4.550 -2.677 5.136** (2.310) (2.951) (2.137) (1.828) SHRP Treatment * Read Home 1.740 -0.078 6.020** 2.467 (1.442) (1.728) (1.880) (1.811) Read Home -0.392 -0.455 -0.646 0.528 (0.489) (0.566) (0.593) (0.651) F-test: joint significance (p-value) 2.99 (0.049) 1.54 (0.2277) 3.45 (0.0324) 7.38 (0.001) Observations 1,294 1146 1,468 1,570 R-squared 0.565 0.412 0.310 0.220 Robust standard errors in parentheses. Individual controls include age and sex of, household assets index, indicators for living with both parents, someone at home reads to the learner, language spoken at home and a constant term. The interaction between SHRP treatment and reading at home with someone is positive for Runyankore/Rukiga, Leblango and Ateso but it is negative for Luganda. In the case of Luganda, the finding suggests that there is no differential effect of the treatment on learners that read at home. For Runyankore/Rukiga and Ateso, learners that read with someone at home seem to benefit differentially more from the treatment than others. In the case of Leblango, the results indicate that learners that read with someone at home are the only ones that benefit (the effect of the treatment shifts completely towards the interaction term)14 . It is important to keep in mind that learners who read at home with someone may have other unobservable characteristics that could also affect their learning. For example, households where someone takes the time to read with a child may be different from others in terms of socioeconomic status, parents’ education, etc. These households may also practice other behaviors that improve the child’s chances of benefiting from the program, such as better nutrition, or caring environment. Other factors may also affect the impact of the program. It is possible that some learners do not stay in the same school from the start of P1 until we assess them at the end of P4, P3 or P2 (depending on the cohort). If the children change from treatment to control schools and vice versa, and do not maintain their original assignment status during the period of the evaluation, it would tend to underestimate the impact of the intervention. Second, if parents more concerned with the education, start sending their children to SHRP-schools part way through the program in search for better education quality, then the selection may tend to overestimate the impact of SHRP, given that it is likely that those households also have other characteristics or behaviors positively associated with learning performance. In order to explore these possibilities, we did two exercises. First, we compared the balance between learners’ characteristics (age, sex, household assets, living arrangements, female teacher, etc.) at endline for all three clusters. We found no important difference between the characteristics of treatment and control groups at endline that should raise concerns about selectivity into treatment schools. We present detail results in Annex D, for each language in all clusters. Second, we used a question that indicates whether the learner has been attending the same school since P1 to see if this status is associated with better reading performance among 14 For cluster 2 we find positive but small interactions and not always statistically significant. No effects were found for Cluster 3. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 49 learners in treatment schools. We found that a good proportion of learners –between 15 and 20 percent on average- do not know the answer to the question. Even when we take this non￾response issue into account, we do not find an association between attending a treatment school since P1 and EGRA subtasks scores. D. LIMITATIONS In general, all evaluation studies have limitations, although we have not identified severe problems in the present work, the following items should be borne in mind: 1. The results discussed in this report correspond to the effects that the program had on the type of learners that were part of the study. SHRP covers 12 languages and many districts in different regions of the country, however results should not be generalized to the whole nation or other mother tongues. Differences in the findings between languages clearly illustrate the point. 2. In addition, learners that were assessed for this study are a random sample of those that were present at school at the time of data collection visits. Learners that tend to be absent from school have, of course, a smaller chance of participating in the study. Given that those that show higher absenteeism tend to perform academically worse, the level of performance of learners evaluated in this study would tend to be higher than the average of all those enrolled in school. 3. We are not aware of problems of contamination from treatment to control schools, however it is impossible to completely rule out interactions between treated and control teachers or head teachers, sharing of teaching approaches or some materials. Note, however, that teacher transfers from treatment schools to control schools or vice versa, should not affect the learners evaluated in this study because, in each cluster, we followed the first cohort of learners treated by SHRP from P1 through higher grades. At the beginning of every year, teachers in the grade that the SHRP student cohort was entering received training; for example, in the case of Cluster 1, P1 teachers were trained in 2013, P2 teachers in 2014, P3 teachers in 2015 and finally P4 teachers in 2016. Given that we followed the cohort of learners that started P1 in 2013, and continued assessing them over the years as they progressed up to P4 in 2016, teacher movements between schools that occurred later could affect the next cohort of students entering the grade, but not the learners under study for this evaluation. There are two exceptions: 1) transfers at some point during the academic year that is relevant to our cohorts, and 2) transfers of teachers between clusters, which is less likely given that the clusters correspond to different languages. E. CONCLUSIONS In this report we presented the results of the impact evaluation of the School Health and Reading Program (SHRP), Result 1, for Clusters 1, 2 and 3 that received the intervention for four, three, and two years respectively. Overall, the analyses suggest that the program has some positive and sometimes sizeable impacts but also several cases where no improvement can be detected. Absolute levels of reading competence remain, in general, very low. The estimated SHRP effects are quite heterogeneous across languages and across tasks; some languages show positive and statistically significant effects in certain EGRA subtasks while other languages do not. For example, in Lumasaaba treatment schools the average SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 50 advantage due to SHRP is 5.6 letter sounds (0.65 standard deviations) while none of the other local languages in the same cluster shows an improvement attributable to SHRP after three years of treatment. Within languages, some EGRA subtasks show improvements while others do not. For example, the Leblango subgroup shows a positive and quite large impact (0.50 standard deviation) in letter sound identification, however after 4 years of treatment there are no effects on non-words decoding or in oral reading fluency. In contrast, Runyankore/Rukiga learners in treatment schools do not show any improvements in letter sound scores due to SHRP, but display a positive and significant impact in non-words, oral reading and oral comprehension of 0.33, 0.44, and 0.40 standard deviations respectively. This is somewhat puzzling. While we do not expect reading learning and performance to be uniform across languages, we would expect more consistency in impacts with improvements in lower-level reading subtasks gradually leading to improvements in higher-level subtasks over time. This is not a pattern we detect in this impact evaluation. After four years of SHRP interventions, most languages subgroups in cluster 1 started to show more positive impacts. SHRP effects are still small in absolute values but effect sizes are larger than in previous years and, hence, more encouraging. The one exception in cluster 1 languages is Leblango, where SHRP only had impact on letter reading sound identification. In general, findings for cluster 2 show lack of or very modest impacts. Runyoro/Rutooro is the only language that seems to show larger and more consistent positive effects of SHRP. Cluster 3 shows positive SHRP effects on the letter sound identification but very little impact on other subtasks. Findings for the Ngakarimojong subgroup are better but still small. There is no evidence that the program makes any difference on the listening comprehension subtask or on English receptive vocabulary for any cluster/language with the exception of Acoli where the impact on listening comprehension is negative. The fact that we do not find any effect in this type of subtasks suggests that the treatment does not generate development of language skills or vocabulary15 above and beyond the regular development in control schools. We find that the reading performance levels of boy and girls are quite similar. There are very few differences and they tend to be very small, sometimes in boys’ favor but in most cases, girls are ahead. In general, no significant differences in the impact of SHRP by gender were identified. The program does not favor a gender in particular. Learner and teacher absenteeism is high and occurs in both treatment and control schools, and in both groups it tends to depress learning and reading performance. Given that a large proportion of the SHRP treatment is delivered to the learners at the school, absenteeism could be reducing SHRP’s potential impact. We show that the interaction between students’ absenteeism and the treatment reduces the impact of the program. Results regarding teachers’ absenteeism are less clear, and may be due to the fact that our measurement of teacher absenteeism is just an approximation. However, teachers themselves acknowledge that their absenteeism negatively affects the learning process. Reports of reading behavior at home are similar between treatment and control learners, although in the case of cluster 1, the percentage of learners that report that someone at home reads with them or to them, is higher for the treatment group. Learners that display this behavior may be benefiting more from the SHRP program, although the evidence is not conclusive. 15 M.M. Dubeck, A. Gove, “The early grade reading assessment (EGRA): Its theoretical foundation, purpose, and limitations / International Journal of Educational Development 40 (2015) 315–322316 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 51 Teachers mentioned that many parents are not supportive of the use of local languages in education. They even suggested that parents may take learners out of SHRP schools to send them to private schools where English is the language of instruction. However, we did not find differences in learners’ characteristics or background at endline that could indicate selective migration towards or from SHRP schools, which could confound results. As well, we did not find an association between consistently attending a treatment school since P1 and EGRA subtasks scores. F. RECOMMENDATIONS A number of recommendations stem from our analysis of quantitative data and FGDs with teachers who participated in the SHRP training. Redesign the training course to allow for more exposure and understanding of the methods and content. Most teachers felt that they could not effectively apply the new methods and content to the children in class because they were not fully equipped to apply the new methods and content due to a short period of exposure to the course. The training course for the teachers should be revised to increase the frequency and length of training, with a systematic follow up plan be put in place for conducting refresher courses. Revise text books. Consideration should be given to a revision of the SHRP textbooks. The issue of missing content and existing errors in the SHRP books should be addressed, perhaps through a participatory approach with a revision team of teachers. In addition, the IP could identify and recommend particular textbooks for cross referencing to support the existing SHRP textbooks. Explore ways to motivate teachers through non-monetary incentives. The teachers noted that they are not recognized for their contribution to learning and reading which has led to low teacher morale. Teachers should be motivated through non-monetary incentives like provision of certificates of attendance and recognition on open days. Improve teacher support supervision: All teachers had received some form of support supervision but this was said to have been inadequate to enable teachers apply the taught content and new methods to the classrooms. It is recommended that a comprehensive and coordinated systematic support supervision plan by developed and communicated to the schools to support the SHRP program. It may also be useful to consider adding peer-to-peer support Efforts to improve school attendance among pupils. Although some reasons behind learners’ absenteeism –such as bad health and child labor – cannot be addressed by the program, there is some room to improve attendance. Efforts should be directed to raise awareness about the importance of attending school every day even in the early grades, starting on the first day and attending through the last day of each term. Non-monetary incentives could be used to recognize learners and families that have started the year on time, or show high attendance. Measures to reduce corporal punishment. Children exposed to violence suffer multiple negative consequences and are also less likely to attend school consistently. Corporal punishment, which is common in Ugandan schools, needs to be addressed. Fortunately, the USAID Literacy Achievement and Retention Activity (LARA) is already working to this end with an intervention that includes teacher, parent and community trainings to improve retention of primary grade students by fostering safe school environments SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 52 Work to change the negative parental perception towards SHRP. Teachers from the study districts noted that parents’ attitude towards SHRP, as a program taught in the vernacular, is negative and that is affecting attendance. Efforts to build community support for the program should be stepped up to target this negative perception. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 53 ANNEX A. SAMPLE SIZE SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 54 The standard approach to determining sample size for analytical surveys is to estimate the sample size required to achieve a specified level of power (probability), such as 90 percent, for detecting a change of a specific magnitude. This sample size depends on a number of factors including the evaluation design, the impact estimate, design of the sample survey used to collect data, the statistical test, and the population under investigation. For the impact evaluation of SHRP Result 1, we estimated the sample required to detect a double-difference measure of impact of magnitude D = 0.20 with a power of 90%. Within each area CCTs were randomly assigned to treatment or control. NORC calculated the number of schools needed in each language/arm cell and within each cell, RTI selected the requisite sample of N treatment schools and N control schools using random assignment. This “balanced” design is an efficient one, with high return of precision and power for survey resources expended. These calculations determined that 14 schools were necessary for each of the cells, where a cell is a combination of language/arm subgroup, and that 30 students would be sampled for each school. CLUSTER 1 The initial group of 11 treatment districts located in the 4 different language areas for cluster 1 was selected by the IP and MoES to participate in the intervention. Initial the IP planned to have 3 different treatment arms. Based on the calculations, 14 schools were necessary for each of the 20 cells (3 treatment arms, 1 control arm and 1 comparison arm, and 4 language subgroups). Table A1 shows the target sample size for the baseline data collection. Table A1: Cluster 1. Baseline Target Sample Size, Treatment Districts Language Group Treatment Arm 1 Treatment Arm 2 Treatment Arm 3 Control Total Luganda 14 schools 420 students 14 schools 420 students 14 schools 420 students 14 schools 420 students 70 schools 2100 students Ateso 14 schools 420 students 14 schools 420 students 14 schools 420 students 14 schools 420 students 70 schools 2100 students Leblango 14 schools 420 students 14 schools 420 students 14 schools 420 students 14 schools 420 students 70 schools 2100 students Runyankor e-Rukiga 14 schools 420 students 14 schools 420 students 14 schools 420 students 14 schools 420 students 70 schools 2100 students Total 168 schools 5040 students 56 schools 1680 students 280 schools 8400 students In October 2016 a smaller randomly selected subsample of learners was assessed. Given that only one treatment arm was rolled out, fewer observations were sufficient. The October 2016 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 55 subsample is used to evaluate the impact of the program after four school years (February 2013 to October 2016). Table A2: Cluster 1. Round 4 Endline Target Sample Size Language Group Treatment Control Total Luganda 14 schools 420 students 14 schools 420 students 70 schools 2100 students Leblango 14 schools 420 students 14 schools 420 students 70 schools 2100 students Ateso 14 schools 420 students 14 schools 420 students 70 schools 2100 students Runyankore-Rukiga 14 schools 420 students 14 schools 420 students 70 schools 2100 students Total 56 schools 1680 students 56 schools 1680 students 280 schools 5040 students In this report we only focus on school level effects and therefore we only use the Treatment and Control schools. In this report we use an analytical sample that includes schools that were surveyed in both: baseline and October 2016 (panel of schools). The estimates using samples that include replacement schools are very similar to those presented here. The analytical sample is presented in Table A3 below and the number of students includes students assessed at baseline and students assessed in October 2016 (cross-sections): Table A3: Cluster 1. Analytical Sample used in this Report Treatment Control Baseline Endline Baseline Endline Luganda 330 333 236 249 (14) (12) Leblango 357 360 388 379 (12) (13) Ateso 422 405 388 389 (14) (13) Runyankore￾Rukiga 307 278 337 385 (13) (14) Total 1416 1376 1349 1402 53 52 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 56 CLUSTER 2 An initial group of districts located in the 4 different language areas for cluster 2 was selected by the IP and MoES to participate in the intervention. NORC calculations determined that 14 schools were necessary in the treatment group and in the control group for each language, and that 30 students would be sampled for each school, for a total of 420 treatment and 420 control students per language. With 4 languages, the total sample size required amounted to 3,360 students from 12 schools per data collection round. Of the 112 schools, half (56 schools) would constitute the treatment group, and half the control group. Table A4 shows the target sample size for each round of data collection. Table A4: Cluster 2. Target Sample Size for Each Round of Data Collection Language Group Treatment Control Total Acoli 14 schools 420 students 14 schools 420 students 28 schools 840 students Lugbarati 14 schools 420 students 14 schools 420 students 28 schools 840 students Lumasaaba 14 schools 420 students 14 schools 420 students 28 schools 840 students Runyoro-Rutooro 14 schools 420 students 14 schools 420 students 28 schools 840 students Total 56 schools 1,680 students 56 schools 1,680 students 112 schools 3,360 students In this report we use an analytical sample that includes schools that were surveyed in both: baseline and October 2016 (panel of schools). The estimates using samples that include replacement schools are very similar to those presented here. The analytical sample is presented in Table A5 below and the number of students includes students assessed at baseline and students assessed in October 2016 (cross-sections): Table A5: Cluster 2. Analytical Sample used in this Report Treatment Control Baseline Endline Baseline Endline Acoli 390 333 400 249 (14) (14) Lugbarati 413 360 347 379 (14) (12) Lumasaaba 416 405 349 389 (14) (12) Runyoro-Rutooro 402 278 404 385 (14) (14) Total 1621 1376 1500 1402 56 52 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 57 CLUSTER 3 As in the case of cluster 2, an initial group of districts located in the 4 different language areas for cluster 3 was selected by the IP and MoES to participate in the intervention. Again NORC sample calculations of 14 schools for treatment and control group for each language, and 30 students for each school were used. Table A46 shows the target sample size for each round of data collection. Table A6: Cluster 3. Baseline and Endline Target Sample Size Language Group Treatment Control Total Lusoga 14 schools 420 students 14 schools 420 students 28 schools 840 students Lugwere 14 schools 420 students 14 schools 420 students 28 schools 840 students Ngakarimojong 14 schools 420 students 14 schools 420 students 28 schools 840 students Lhukonzo 14 schools 420 students 14 schools 420 students 28 schools 840 students Total 56 schools 1680 students 56 schools 1680 students 112 schools 3360 students In this report we use an analytical sample that includes schools that were surveyed in both: baseline and October 2016 (panel of schools). The estimates using samples that include replacement schools are very similar to those presented here. The analytical sample is presented in Table A7 below and the number of students includes students assessed at baseline and students assessed in October 2016 (cross-sections): Table A7: Cluster 3. Analytical Sample used in this Report Treatment Control Baseline Endline Baseline Endline Lusoga 405 278 413 385 (14) (14) Lugwere 426 333 420 249 (14) (14) Ngakarimonjong 365 360 292 379 (14) (13) Lhukonzo 403 405 420 389 (14) (14) Total 1599 1376 1545 1402 56 55 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 58 ANNEX B. SAMPLE BALANCE AT BASELINE SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 59 Table B1: Cluster 1 - Demographics and other characteristics at baseline for treatment and control groups. All language groups. Variable Treatmen t Mean (SE) Contro l Mean (SE) Differenc e (C – T) Age of student 7.338 (0.028) 7.334 (0.053) -0.004 % of cases with missing age information 0.156 (0.005) 0.152 (0.010) -0.004 Gender of student (female = 1) 0.492 (0.007) 0.494 (0.013) 0.002 Number of assets (max = 8) 2.513 (0.018) 2.533 (0.032) 0.020 Lives with both parents (yes = 1) 0.633 (0.007) 0.668 (0.013) 0.035 Does not live with mother (yes = 1) 0.177 (0.006) 0.173 (0.010) -0.004 Reads at home (yes = 1) 0.422 (0.007) 0.417 (0.013) -0.005 Attended preschool (yes = 1) 0.461 (0.007) 0.416 (0.013) -0.045 Student absent any day in the week prior to assessment (yes = 1) 0.523 (0.007) 0.508 (0.014) -0.015 Teacher absent any day in the week prior to assessment (yes = 1) 0.433 (0.008) 0.435 (0.014) 0.002 Gender of teacher (female = 1) 0.756 (0.006) 0.615 (0.013) -0.141 *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 60 Table B2: Cluster 1 - Literacy scores, basic subtasks, by language, for treatment and control groups Literacy Scores Treatment Mean (SE) Control Mean (SE) Difference (C – T) English Letter sound score (max = 100) 1.543 (0.059) 1.522 (0.092) -0.021 Word segmenting score (max = 10) 0.221 (0.014) 0.188 (0.024) -0.033 Nonword decoding score (max = 50) 0.140 (0.018) 0.085 (0.026) -0.055 Runyankore/Rukiga Letter sound score (max = 100) 3.070 (0.171) 3.388 (0.271) -0.319 Word segmenting score (max = 10) 3.278 (0.131) 3.584 (0.229) -0.306 Nonword decoding score (max = 50) 0.423 (0.018) 0.406 (0.026) 0.017 Luganda Letter sound score (max = 100) 2.419 (0.157) 1.852 (0.259) 0.568 Word segmenting score (max = 10) 2.351 (0.115) 3.129 (0.272) -0.777 Nonword decoding score (max = 50) 0.199 (0.044) 0.140 (0.078) 0.060 Lango Letter sound score (max = 100) 0.785 (0.085) 0.693 (0.160) 0.092 Word segmenting score (max = 10) 0.020 (0.008) 0.005 (0.004) 0.015 Nonword decoding score (max = 50) 0.016 (0.012) 0.015 (0.015) 0.000 Ateso Letter sound score (max = 100) 1.810 (0.106) 1.782 (0.196) 0.028 Word segmenting score (max = 10) 1.725 (0.094) 1.380 (0.147) 0.344 Nonword decoding score (max = 50) 0.015 (0.008) 0.067 (0.041) -0.052 *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 61 Table B3: Cluster 2 - English literacy scores, demographics and other characteristics at baseline for treatment and control groups. All language groups. Variable Treatmen t Mean (SE) Contro l Mean (SE) Differenc e (C – T) Age of student 7.645 (0.051) 7.704 (0.051) 0.059 % of cases with missing age information 0.183 (0.010) 0.199 (0.010) 0.016 Gender of student (female = 1) 0.497 (0.012) 0.490 (0.012) -0.007 Number of assets (max = 8) 2.464 (0.034) 2.448 (0.032) -0.016 Lives with both parents (yes = 1) 0.579 (0.012) 0.644 (0.012) 0.065* Does not live with mother (yes = 1) 0.187 (0.010) 0.173 (0.009) -0.014 Reads at home (yes = 1) 0.422 (0.012) 0.417 (0.012) -0.005 Attended preschool (yes = 1) 0.418 (0.012) 0.386 (0.012) -0.032 Student absent any day in the week prior to assessment (yes = 1) 0.443 (0.012) 0.446 (0.013) 0.003 Teacher absent any day in the week prior to assessment (yes = 1) 0.455 (0.013) 0.491 (0.013) 0.036 Gender of teacher (female = 1) 0.700 (0.012) 0.679 (0.013) -0.021 *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 62 Table B4: Cluster 2 - Literacy scores, basic subtasks, by language, for treatment and control groups Literacy Scores Treatment Mean (SE) Control Mean (SE) Difference (C - T) English Letter sound score (max = 100) 0.913 (0.066) 0.585 (0.044) -0.328 Word segmenting score (max = 10) 4.349 (0.041) 4.565 (0.033) 0.216* Nonword decoding score (max = 50) 0.035 (0.014) 0.009 (0.005) -0.026 Runyoro-Rutooro Letter sound score (max = 100) 2.493 (0.252) 2.375 (0.208) -0.118 Word segmenting score (max = 10) 5.570 (0.125) 5.355 (0.105) -0.215 Nonword decoding score (max = 50) 0.087 (0.047) 0.084 (0.049) -0.003 Lugbarati Letter sound score (max = 100) 2.915 (0.240) 1.716 (0.182) -1.199* Word segmenting score (max = 10) 4.864 (0.138) 4.662 (0.128) -0.202 Nonword decoding score (max = 50) 0.010 (0.010) 0.000 (0.000) -0.010 Lumasaaba Letter sound score (max = 100) 0.310 (0.046) 0.293 (0.054) -0.017 Word segmenting score (max = 10) 4.591 (0.078) 4.556 (0.072) -0.035 Nonword decoding score (max = 50) 0.000 (0.000) 0.000 (0.000) 0.000 Acoli Letter sound score (max = 100) 1.421 (0.141) 0.900 (0.129) -0.521 Word segmenting score (max = 10) 5.144 (0.091) 5.175 (0.102) 0.031 Nonword decoding score (max = 50) 0.000 (0.000) 0.020 (0.014) 0.020 *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 63 Table B5: Cluster 3 Demographics and other characteristics at baseline for treatment and control groups. All language groups. Variable Treatmen t Mean (SE) Contro l Mean (SE) Differenc e (C – T) Age of student 7.316 (0.055) 7.154 (0.054) -0.162 % of cases with missing age information 0.332 (0.012) 0.335 (0.012) 0.003 Gender of student (female = 1) 0.487 (0.013) 0.487 (0.013) 0.000 Number of assets (max = 7) 2.587 (0.037) 2.379 (0.036) -0.209 Home has electricity (yes=1) 0.241 (0.011) 0.218 (0.010) -0.023 Lives with both parents (yes = 1) 0.471 (0.012) 0.495 (0.013) 0.024 Does not live with mother (yes = 1) 0.237 (0.011) 0.222 (0.010) -0.015 Reads at home (yes = 1) 0.421 (0.012) 0.403 (0.013) -0.018 Attended preschool (yes = 1) 0.474 (0.013) 0.423 (0.013) -0.051 Student absent any day in the week prior to assessment (yes = 1) 0.459 (0.013) 0.479 (0.013) 0.020 Teacher absent any day in the week prior to assessment (yes = 1) 0.419 (0.013) 0.469 (0.013) 0.050 Gender of teacher (female = 1) 0.856 (0.009) 0.733 (0.012) -0.123 *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 64 Table B6: Cluster 3 Literacy scores, basic subtasks, by language, for treatment and control groups Literacy Scores Treatment Mean (SE) Control Mean (SE) Difference (C – T) English Letter sound score (max = 100) 0.697 (0.053) 0.487 (0.052) -0.210 Print orientation (max = 3) 0.582 (0.021) 0.499 (0.020) -0.083 Letter writing score (max = 5) 0.878 (0.027) 0.759 (0.025) -0.119 Lusoga Letter sound score (max = 100) 0.760 (0.125) 0.782 (0.176) 0.022 Word segmenting score (max = 10) 4.773 (0.088) 4.610 (0.067) -0.163 Fluency (max = 41) 0.002 (0.002) 0.000 (0.000) -0.002 Lugwere Letter sound score (max = 100) 0.432 (0.052) 0.426 (0.051) -0.006 Word segmenting score (max = 10) 4.676 (0.059) 4.793 (0.054) 0.117 Fluency (max = 42) 0.007 (0.005) 0.007 (0.007) 0.000 Ngakarimojong Letter sound score (max = 100) 1.449 (0.158) 0.414 (0.076) -1.035*** Word segmenting score (max = 10) 4.907 (0.109) 4.779 (0.089) -0.128 Fluency (max = 47) 0.003 (0.003) 0.000 (0.000) -0.003 Lhukonzo Letter sound score (max = 100) 1.526 (0.160) 1.002 (0.132) -0.524 Word segmenting score (max = 10) 5.134 (0.106) 4.976 (0.084) -0.158 Fluency (max = 40) 0.005 (0.005) 0.021 (0.013) 0.017 *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 65 ANNEX C. EGRA SUBTASKS, DISTRIBUTIONAL GRAPHS BY LANGUAGE GROUP, BASELINE AND ENDLINE SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 66 Cohort 1 72.9 66.7 24.2 29.4 2.5 2.7 0.4 0.6 0.3 0.3 0 20 40 60 80 Percent of Luganda Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Baseline - Luganda Control Treatment 37.8 21.6 26.9 16.8 24.5 26.7 7.2 19.2 3.2 9.0 0.4 4.8 1.8 0 10 20 30 40 Percent of Luganda Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 English Letter Sound Score at Endline - Luganda Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 67 69.9 68.8 25.0 27.0 4.7 3.3 0.4 0.9 0 20 40 60 80 Percent of Luganda Learners 0 1-10 11-20 21-30 Letter Sound Score at Baseline - Luganda Control Treatment 27.7 15.3 24.1 20.1 31.7 24.0 12.0 21.0 3.2 12.0 0.8 3.9 0.4 3.0 0.3 0.3 0 10 20 30 Percent of Luganda Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 Letter Sound Score at Endline - Luganda Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 68 98.3 97.0 1.3 2.1 0.4 0.9 0 20 40 60 80 100 Percent of Luganda Learners 0 1-10 11-20 Non-words at Baseline - Luganda Control Treatment 28.9 28.2 18.1 11.1 28.9 25.2 16.5 22.2 6.8 11.4 0.8 1.5 0.3 0 10 20 30 Percent of Luganda Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 Non-words at Endline - Luganda Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 69 97.5 96.1 1.7 3.6 0.8 0.3 0 20 40 60 80 100 Percent of Luganda Learners 0 1-10 11-20 Fluency at Baseline - Luganda Control Treatment 29.7 24.3 13.7 10.8 21.7 15.9 17.7 18.9 10.4 17.1 4.0 10.5 2.4 1.8 0.4 0.3 0.3 0 10 20 30 Percent of Luganda Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 Fluency at Endline - Luganda Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 70 100.0 98.5 1.2 0.3 0 20 40 60 80 100 Percent of Luganda Learners 0 1 2 Reading Comprehension at Baseline - Luganda Control Treatment 37.8 33.9 12.4 6.6 14.1 9.0 13.7 18.0 16.9 23.1 5.2 9.3 0 10 20 30 40 Percent of Luganda Learners 0 1 2 3 4 5 Reading Comprehension at Endline - Luganda Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 71 41.1 43.3 25.4 31.8 27.1 21.8 6.4 3.0 0 10 20 30 40 Percent of Luganda Learners 0 1 2 3 Listening Comprehension at Baseline - Luganda Control Treatment 7.2 5.1 18.5 21.9 59.0 54.7 15.3 18.3 0 20 40 60 Percent of Luganda Learners 0 1 2 3 Listening Comprehension at Endline - Luganda Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 72 89.2 88.0 10.6 11.8 0.3 0.3 0 20 40 60 80 100 Percent of Leblango Learners 0 1-10 11-20 English Letter Sound Score at Baseline - Leblango Control Treatment 79.2 56.1 11.9 18.1 7.1 19.2 1.6 4.4 0.3 1.7 0.6 0 20 40 60 80 Percent of Leblango Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Endline - Leblango Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 73 91.8 92.2 5.9 6.7 1.8 1.1 0.3 0.3 0 20 40 60 80 100 Percent of Leblango Learners 0 1-10 11-20 21-30 31-40 Letter Sound Score at Baseline - Leblango Control Treatment 68.1 35.3 13.2 20.0 12.1 27.2 5.0 10.8 0.8 6.1 0.5 0.3 0.3 0 20 40 60 80 Percent of Leblango Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Letter Sound Score at Endline - Leblango Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 74 99.7 99.7 0.3 0.3 0 20 40 60 80 100 Percent of Leblango Learners 0 1-10 Non-words at Baseline - Leblango Control Treatment 45.9 47.5 27.7 24.4 20.6 23.3 5.5 3.1 0.3 1.7 0 10 20 30 40 50 Percent of Leblango Learners 0 1-10 11-20 21-30 31-40 Non-words at Endline - Leblango Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 75 97.9 96.9 2.1 3.1 0 20 40 60 80 100 Percent of Leblango Learners 0 1-10 Fluency at Baseline - Leblango Control Treatment 29.3 26.1 21.6 26.4 20.3 19.2 14.513.9 8.2 5.6 3.2 5.3 2.6 2.8 0.6 0.3 0 10 20 30 Percent of Leblango Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 Fluency at Endline - Leblango Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 76 100.0 100.0 0 20 40 60 80 100 Percent of Leblango Learners 0 Reading Comprehension at Baseline - Leblango Control Treatment 45.9 42.8 16.9 16.4 16.4 21.7 14.8 9.4 4.0 5.0 2.1 4.7 0 10 20 30 40 50 Percent of Leblango Learners 0 1 2 3 4 5 Reading Comprehension at Endline - Leblango Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 77 16.0 14.0 19.8 19.3 34.0 32.5 30.2 34.2 0 10 20 30 40 Percent of Leblango Learners 0 1 2 3 Listening Comprehension at Baseline - Leblango Control Treatment 1.1 0.3 3.7 5.0 29.3 33.9 66.0 60.8 0 20 40 60 80 Percent of Leblango Learners 0 1 2 3 Listening Comprehension at Endline - Leblango Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 78 67.8 69.4 30.2 29.1 1.5 1.4 0.5 0 20 40 60 80 Percent of Ateso Learners 0 1-10 11-20 21-30 English Letter Sound Score at Baseline - Ateso Control Treatment 31.1 22.7 30.1 30.9 30.8 37.0 7.7 6.9 0.3 1.5 1.0 0 10 20 30 40 Percent of Ateso Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Endline - Ateso Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 79 69.3 69.2 25.3 26.8 4.6 4.0 0.5 0.3 0 20 40 60 80 Percent of Ateso Learners 0 1-10 11-20 21-30 31-40 Letter Sound Score at Baseline - Ateso Control Treatment 15.9 10.1 22.6 21.0 38.6 40.7 19.5 22.2 2.8 4.4 0.5 1.5 0 10 20 30 40 Percent of Ateso Learners 0 1-10 11-20 21-30 31-40 41-50 Letter Sound Score at Endline - Ateso Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 80 99.2 99.8 0.5 0.2 0.3 0 20 40 60 80 100 Percent of Ateso Learners 0 1-10 11-20 Non-words at Baseline - Ateso Control Treatment 71.5 51.9 5.9 8.6 13.9 22.7 6.7 12.1 1.3 3.5 0.5 0.2 0.3 0.7 0.2 0 20 40 60 80 Percent of Ateso Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Non-words at Endline - Ateso Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 81 99.2 99.8 0.8 0.2 0 20 40 60 80 100 Percent of Ateso Learners 0 1-10 Fluency at Baseline - Ateso Control Treatment 70.7 47.2 6.4 11.6 8.5 11.4 7.5 11.6 4.6 12.3 1.5 3.5 0.5 2.0 0.3 0.5 0 20 40 60 80 Percent of Ateso Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Fluency at Endline - Ateso Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 82 99.2 100.0 0.5 0.3 0 20 40 60 80 100 Percent of Ateso Learners 0 1 2 Reading Comprehension at Baseline - Ateso Control Treatment 79.4 58.8 9.5 18.5 5.4 8.9 3.3 6.2 2.1 7.2 0.3 0.5 0 20 40 60 80 Percent of Ateso Learners 0 1 2 3 4 5 Reading Comprehension at Endline - Ateso Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 83 12.6 15.6 25.0 22.5 44.3 44.3 18.0 17.5 0 10 20 30 40 Percent of Ateso Learners 0 1 2 3 Listening Comprehension at Baseline - Ateso Control Treatment 1.8 1.5 11.8 13.6 36.5 29.4 49.9 55.6 0 20 40 60 Percent of Ateso Learners 0 1 2 3 Listening Comprehension at Endline - Ateso Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 84 58.5 71.3 33.2 23.1 8.3 4.6 1.0 0 20 40 60 80 Percent of RR Learners 0 1-10 11-20 21-30 English Letter Sound Score at Baseline - RR Control Treatment 36.4 29.1 40.3 41.4 22.3 21.2 0.8 7.2 0.3 1.1 0 10 20 30 40 Percent of RR Learners 0 1-10 11-20 21-30 31-40 English Letter Sound Score at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 85 57.0 64.5 34.1 25.4 8.3 8.1 0.6 1.0 1.0 0 20 40 60 Percent of RR Learners 0 1-10 11-20 21-30 31-40 Letter Sound Score at Baseline - RR Control Treatment 9.1 4.3 40.5 46.8 42.6 35.3 7.5 11.2 0.3 1.8 0.7 0 10 20 30 40 50 Percent of RR Learners 0 1-10 11-20 21-30 31-40 41-50 Letter Sound Score at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 86 95.8 93.2 2.1 4.6 2.1 2.3 0 20 40 60 80 100 Percent of RR Learners 0 1-10 11-20 Non-words at Baseline - RR Control Treatment 25.7 16.9 10.6 9.0 30.6 24.8 24.2 31.3 7.8 14.0 0.8 4.0 0.3 0 10 20 30 Percent of RR Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 Non-words at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 87 96.7 94.5 2.4 3.3 0.6 2.3 0.3 0 20 40 60 80 100 Percent of RR Learners 0 1-10 11-20 21-30 Fluency at Baseline - RR Control Treatment 21.3 13.3 8.8 6.5 14.8 10.8 27.5 23.7 22.1 30.6 4.9 10.8 0.5 3.6 0.7 0 10 20 30 Percent of RR Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Fluency at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 88 98.8 96.4 0.6 2.9 0.6 0.7 0 20 40 60 80 100 Percent of RR Learners 0 1 2 Reading Comprehension at Baseline - RR Control Treatment 26.5 18.0 16.1 7.9 17.1 16.5 26.0 32.0 13.2 19.8 1.0 5.8 0 10 20 30 Percent of RR Learners 0 1 2 3 4 5 Reading Comprehension at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 89 23.1 21.2 25.5 27.7 36.5 34.9 14.8 16.3 0 10 20 30 40 Percent of RR Learners 0 1 2 3 Listening Comprehension at Baseline - RR Control Treatment 2.6 2.2 8.1 1.8 32.7 33.1 56.6 62.9 0 20 40 60 Percent of RR Learners 0 1 2 3 Listening Comprehension at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 90 72.5 73.9 24.2 23.5 3.0 2.1 0.2 0.4 0.1 0.1 0 20 40 60 80 Percent of English Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Baseline - English Control Treatment 46.7 32.5 27.4 26.2 21.0 26.7 4.1 9.3 0.8 3.3 0.1 1.6 0.4 0 10 20 30 40 50 Percent of English Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 English Letter Sound Score at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 91 92.592.5 2.93.2 2.52.2 0.70.7 0.10.3 0.40.2 0.30.1 0.20.2 0.10.2 0.10.1 0.20.2 0 20 40 60 80 100 Percent of English Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - English Control Treatment 15.8 22.5 8.18.8 8.07.8 1.63.2 1.12.8 64.6 49.6 0.31.4 0.11.6 0.30.9 0.21.0 0.4 0 20 40 60 Percent of English Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 92 98.7 98.1 0.9 1.6 0.3 0.3 0.1 0 20 40 60 80 100 Percent of English Learners 0 1-10 11-20 21-30 Non-words at Baseline - English Control Treatment 46.5 40.0 13.3 15.2 24.3 22.6 10.8 14.3 4.6 5.7 0.3 1.5 0.1 0.5 0.1 0 10 20 30 40 50 Percent of English Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Non-words at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 93 97.1 94.6 2.6 4.6 0.2 0.8 0.1 0.1 0 20 40 60 80 100 Percent of English Learners 0 1-10 11-20 21-30 Fluency at Baseline - English Control Treatment 40.8 34.1 8.77.6 8.4 10.110.8 9.4 10.610.9 9.4 11.1 4.9 6.7 3.53.8 1.4 2.9 0.71.3 0.60.7 0 10 20 30 40 Percent of English Learners 0 1-1011-2021-3031-4041-5051-6061-7071-8081-9091-10011-110 111-120 121-130 Fluency at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 94 99.9 99.8 0.1 0.2 0.1 0 20 40 60 80 100 Percent of English Learners 0 1 2 Reading Comprehension at Baseline - English Control Treatment 77.7 73.2 10.1 9.9 5.7 5.9 3.6 4.9 2.0 3.7 0.9 2.5 0 20 40 60 80 Percent of English Learners 0 1 2 3 4 5 Reading Comprehension at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 95 16.4 15.3 48.7 53.9 30.4 23.4 4.2 6.3 0.4 1.1 0 20 40 60 Percent of English Learners 0 1-5 6-10 11-15 16-20 Vocabulary at Baseline - English Control Treatment 0.1 9.5 49.3 51.9 33.1 32.0 8.6 6.6 0 10 20 30 40 50 Percent of English Learners 0 1-5 6-10 11-15 16-20 Vocabulary at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 96 Cohort 2 91.8 85.6 8.3 14.4 0 20 40 60 80 100 Percent of Acoli Learners 0 1-10 English Letter Sound Score at Baseline - Acoli Control Treatment 73.3 55.2 15.8 22.0 8.5 16.7 1.7 5.1 0.6 0.8 0.2 0 20 40 60 80 Percent of Acoli Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Endline - Acoli Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 97 84.8 73.1 13.3 25.4 2.0 1.5 0 20 40 60 80 Percent of Acoli Learners 0 1-10 11-20 Letter Sound Score at Baseline - Acoli Control Treatment 56.3 36.9 23.324.6 14.2 25.7 2.7 8.3 1.7 3.7 1.3 0.8 0.4 0 20 40 60 Percent of Acoli Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 71-80 Letter Sound Score at Endline - Acoli Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 98 3.31.0 2.52.8 2.34.4 3.83.1 3.81.8 67.3 71.0 2.32.6 2.02.8 3.33.1 3.02.8 6.84.6 0 20 40 60 80 Percent of Acoli Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - Acoli Control Treatment 1.52.6 1.92.4 4.23.1 3.82.8 3.83.5 52.7 42.8 6.36.3 5.86.7 6.0 8.8 7.57.1 6.7 13.9 0 10 20 30 40 50 Percent of Acoli Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - Acoli Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 99 99.3 100.0 0.8 0 20 40 60 80 100 Percent of Acoli Learners 0 1-10 Non-words at Baseline - Acoli Control Treatment 95.4 88.4 2.5 6.7 2.1 2.9 1.6 0 20 40 60 80 100 Percent of Acoli Learners 0 1-10 11-20 21-30 41-50 Non-words at Endline - Acoli Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 100 98.8 99.5 1.3 0.5 0 20 40 60 80 100 Percent of Acoli Learners 0 1-10 Fluency at Baseline - Acoli Control Treatment 92.5 82.1 3.5 6.7 2.7 4.9 0.4 2.0 0.8 1.6 2.2 0.2 0.4 0 20 40 60 80 100 Percent of Acoli Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Fluency at Endline - Acoli Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 101 99.8 100.0 0.3 0 20 40 60 80 100 Percent of Acoli Learners 0 1 Reading Comprehension at Baseline - Acoli Control Treatment 96.7 91.2 2.7 4.5 0.4 2.8 0.2 1.2 0.4 0 20 40 60 80 100 Percent of Acoli Learners 0 1 2 3 4 Reading Comprehension at Endline - Acoli Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 102 34.8 19.0 19.3 19.0 16.8 21.0 29.3 41.0 0 10 20 30 40 Percent of Acoli Learners 0 1 2 3 Listening Comprehension at Baseline - Acoli Control Treatment 5.2 3.3 8.8 6.9 15.0 16.5 71.0 73.3 0 20 40 60 80 Percent of Acoli Learners 0 1 2 3 Listening Comprehension at Endline - Acoli Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 103 81.6 69.2 17.6 27.8 0.9 2.7 0 20 40 60 80 Percent of Lugbarati Learners 0 1-10 11-20 31-40 English Letter Sound Score at Baseline - Lugbarati Control Treatment 25.0 27.2 24.1 23.5 39.8 36.3 10.2 10.6 0.5 2.0 0.5 0.2 0.2 0 10 20 30 40 Percent of Lugbarati Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 English Letter Sound Score at Endline - Lugbarati Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 104 73.2 64.2 22.5 26.2 4.0 8.2 0.3 1.5 0 20 40 60 80 Percent of Lugbarati Learners 0 1-10 11-20 21-30 Letter Sound Score at Baseline - Lugbarati Control Treatment 11.3 13.3 27.3 33.7 42.6 33.9 12.712.6 2.8 3.1 1.4 2.6 1.4 0.4 0.2 0.2 0.2 0.2 0 10 20 30 40 Percent of Lugbarati Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 Letter Sound Score at Endline - Lugbarati Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 105 11.510.9 2.93.1 5.85.3 6.6 8.5 5.5 8.0 42.1 35.4 5.24.4 5.85.3 4.3 5.8 4.34.6 6.1 8.7 0 10 20 30 40 Percent of Lugbarati Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - Lugbarati Control Treatment 14.614.3 7.2 4.6 4.6 2.0 3.0 5.0 2.1 3.5 22.2 18.1 4.64.6 5.85.9 5.6 7.4 5.66.1 24.8 28.3 0 10 20 30 Percent of Lugbarati Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - Lugbarati Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 106 100.0 99.8 0.2 0 20 40 60 80 100 Percent of Lugbarati Learners 0 1-10 Non-words at Baseline - Lugbarati Control Treatment 69.4 71.7 12.3 8.3 13.9 14.4 3.5 5.2 0.9 0.4 0 20 40 60 80 Percent of Lugbarati Learners 0 1-10 11-20 21-30 31-40 Non-words at Endline - Lugbarati Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 107 99.4 99.3 0.6 0.7 0 20 40 60 80 100 Percent of Lugbarati Learners 0 1-10 Fluency at Baseline - Lugbarati Control Treatment 68.166.3 9.5 9.1 9.0 10.6 6.9 7.4 5.3 5.0 1.2 1.1 0.4 0.2 0 20 40 60 80 Percent of Lugbarati Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Fluency at Endline - Lugbarati Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 108 100.0 100.0 0 20 40 60 80 100 Percent of Lugbarati Learners 0 Reading Comprehension at Baseline - Lugbarati Control Treatment 77.1 74.3 7.9 9.3 5.6 6.5 4.4 5.6 4.9 3.3 0.2 1.1 0 20 40 60 80 Percent of Lugbarati Learners 0 1 2 3 4 5 Reading Comprehension at Endline - Lugbarati Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 109 13.8 9.7 13.3 16.0 29.1 30.5 43.8 43.8 0 10 20 30 40 Percent of Lugbarati Learners 0 1 2 3 Listening Comprehension at Baseline - Lugbarati Control Treatment 3.7 1.1 9.5 6.3 26.2 26.9 60.6 65.7 0 20 40 60 80 Percent of Lugbarati Learners 0 1 2 3 Listening Comprehension at Endline - Lugbarati Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 110 93.4 91.8 6.6 8.2 0 20 40 60 80 100 Percent of Lumasaaba Learners 0 1-10 English Letter Sound Score at Baseline - Lumasaaba Control Treatment 51.5 25.3 38.7 42.5 8.0 20.6 0.9 6.4 0.2 3.0 0.6 1.4 0.2 0.4 0.2 0 10 20 30 40 50 Percent of Lumasaaba Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 English Letter Sound Score at Endline - Lumasaaba Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 111 86.0 87.3 14.0 12.7 0 20 40 60 80 Percent of Lumasaaba Learners 0 1-10 Letter Sound Score at Baseline - Lumasaaba Control Treatment 14.7 6.4 66.2 50.3 15.8 28.3 1.9 8.6 1.3 4.2 1.6 0.4 0.2 0 20 40 60 80 Percent of Lumasaaba Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Letter Sound Score at Endline - Lumasaaba Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 112 2.9 5.5 5.23.6 3.22.2 2.01.4 2.92.9 80.579.1 1.11.7 0.91.4 0.60.5 0.61.0 0.30.7 0 20 40 60 80 Percent of Lumasaaba Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - Lumasaaba Control Treatment 2.42.8 7.6 5.6 4.54.2 6.9 5.4 3.7 5.4 41.3 27.7 3.5 6.2 4.5 8.2 6.7 10.2 7.1 9.6 11.7 14.8 0 10 20 30 40 Percent of Lumasaaba Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - Lumasaaba Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 113 100.0 100.0 0 20 40 60 80 100 Percent of Lumasaaba Learners 0 Non-words at Baseline - Lumasaaba Control Treatment 82.7 72.7 10.0 12.4 6.9 11.0 0.4 4.0 0 20 40 60 80 Percent of Lumasaaba Learners 0 1-10 11-20 21-30 Non-words at Endline - Lumasaaba Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 114 99.7 100.0 0.3 0 20 40 60 80 100 Percent of Lumasaaba Learners 0 1-10 Fluency at Baseline - Lumasaaba Control Treatment 83.3 71.1 9.7 11.8 5.2 9.6 1.7 6.4 1.0 0.2 0 20 40 60 80 Percent of Lumasaaba Learners 0 1-10 11-20 21-30 31-40 41-50 Fluency at Endline - Lumasaaba Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 115 100.0 100.0 0 20 40 60 80 100 Percent of Lumasaaba Learners 0 Reading Comprehension at Baseline - Lumasaaba Control Treatment 92.4 83.8 4.3 7.8 1.5 5.4 1.3 2.4 0.4 0.6 0 20 40 60 80 100 Percent of Lumasaaba Learners 0 1 2 3 4 Reading Comprehension at Endline - Lumasaaba Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 116 70.5 61.3 21.5 28.6 6.6 8.7 1.4 1.4 0 20 40 60 80 Percent of Lumasaaba Learners 0 1 2 3 Listening Comprehension at Baseline - Lumasaaba Control Treatment 19.0 13.4 35.3 30.5 29.9 33.3 15.8 22.8 0 10 20 30 40 Percent of Lumasaaba Learners 0 1 2 3 Listening Comprehension at Endline - Lumasaaba Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 117 72.8 73.9 26.2 23.9 1.0 1.7 0.2 0.2 0 20 40 60 80 Percent of RR Learners 0 1-10 11-20 21-30 31-40 English Letter Sound Score at Baseline - RR Control Treatment 23.3 13.8 34.8 30.5 30.6 36.5 6.7 15.1 2.8 3.3 1.3 0.6 0.6 0 10 20 30 40 Percent of RR Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 English Letter Sound Score at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 118 63.0 62.4 30.3 30.1 6.2 7.0 0.5 0.2 0 20 40 60 Percent of RR Learners 0 1-10 11-20 21-30 31-40 51-60 Letter Sound Score at Baseline - RR Control Treatment 10.6 5.6 40.6 27.2 31.1 40.8 12.0 19.8 2.4 5.6 2.4 0.8 0.9 0.2 0 10 20 30 40 Percent of RR Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 Letter Sound Score at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 119 3.02.7 2.04.0 2.23.5 3.24.0 2.22.7 67.7 56.0 2.22.2 1.53.7 3.51.7 4.04.2 8.4 15.2 0 20 40 60 80 Percent of RR Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - RR Control Treatment 4.6 2.1 2.82.1 2.41.7 7.2 4.7 4.14.1 17.2 13.8 7.06.8 6.57.2 8.18.0 10.0 12.6 30.0 36.9 0 10 20 30 40 Percent of RR Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 120 99.3 98.8 0.2 1.0 0.5 0.2 0 20 40 60 80 100 Percent of RR Learners 0 1-10 11-20 Non-words at Baseline - RR Control Treatment 64.6 42.9 8.7 10.7 18.7 22.3 7.4 18.6 0.6 4.7 0.8 0 20 40 60 Percent of RR Learners 0 1-10 11-20 21-30 31-40 41-50 Non-words at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 121 99.5 99.5 0.2 0.2 0.2 0 20 40 60 80 100 Percent of RR Learners 0 1-10 11-20 21-30 Fluency at Baseline - RR Control Treatment 58.3 37.5 13.712.4 12.812.8 10.7 20.6 3.9 12.8 0.6 3.3 0.4 0.2 0 20 40 60 Percent of RR Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 Fluency at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 122 99.8 99.8 0 20 40 60 80 100 Percent of RR Learners 0 2 Reading Comprehension at Baseline - RR Control Treatment 68.1 45.0 4.6 4.7 8.1 11.7 16.1 26.0 2.8 11.7 0.2 1.0 0 20 40 60 80 Percent of RR Learners 0 1 2 3 4 5 Reading Comprehension at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 123 34.8 32.3 22.6 23.4 21.9 18.9 20.6 25.4 0 10 20 30 40 Percent of RR Learners 0 1 2 3 Listening Comprehension at Baseline - RR Control Treatment 4.8 3.1 11.5 6.4 30.0 22.9 53.7 67.6 0 20 40 60 80 Percent of RR Learners 0 1 2 3 Listening Comprehension at Endline - RR Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 124 84.7 80.1 14.9 18.6 0.5 1.1 0.1 0.1 0 20 40 60 80 Percent of English Learners 0 1-10 11-20 21-30 31-40 English Letter Sound Score at Baseline - English Control Treatment 43.1 30.3 28.629.5 21.7 27.7 4.8 9.3 1.1 2.3 0.6 0.6 0.2 0.1 0.1 0.0 0 10 20 30 40 Percent of English Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 English Letter Sound Score at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 125 99.7 99.3 0.3 0.7 0.1 0 20 40 60 80 100 Percent of English Learners 0 1-10 11-20 Non-words at Baseline - English Control Treatment 77.4 67.7 10.2 12.6 10.4 13.2 1.8 5.2 0.2 1.1 0.2 0 20 40 60 80 Percent of English Learners 0 1-10 11-20 21-30 31-40 41-50 Non-words at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 126 97.3 97.4 2.7 2.5 0.1 0 20 40 60 80 100 Percent of English Learners 0 1-10 11-20 Fluency at Baseline - English Control Treatment 74.2 67.0 11.810.4 7.47.9 3.65.5 1.53.6 1.23.3 0.31.5 0.20.4 0.2 0.0 0.0 0 20 40 60 80 Percent of English Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-9091-100101-110 Fluency at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 127 100.0 99.9 0.1 0 20 40 60 80 100 Percent of English Learners 0 1 Reading Comprehension at Baseline - English Control Treatment 97.8 93.0 1.3 4.0 0.8 2.1 0.1 0.5 0.2 0.1 0 20 40 60 80 100 Percent of English Learners 0 1 2 3 4 5 Reading Comprehension at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 128 7.1 5.5 62.0 56.9 29.9 35.7 1.0 1.9 0 20 40 60 Percent of English Learners 0 1-5 6-10 11-15 Vocabulary at Baseline - English Control Treatment 0.6 0.5 19.0 15.5 66.0 63.7 13.9 18.8 0.5 1.5 0 20 40 60 80 Percent of English Learners 0 1-5 6-10 11-15 16-20 Vocabulary at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 129 Cohort 3 96.2 93.4 3.8 6.6 0 20 40 60 80 100 Percent of Lugwere Learners 0 1-10 English Letter Sound Score at Baseline - Lugwere Control Treatment 74.3 52.5 23.6 33.0 1.9 12.4 0.2 1.7 0.2 0.2 0 20 40 60 80 Percent of Lugwere Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Endline - Lugwere Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 130 81.7 80.8 18.3 19.2 0 20 40 60 80 Percent of Lugwere Learners 0 1-10 Letter Sound Score at Baseline - Lugwere Control Treatment 40.5 35.9 58.8 53.7 0.7 9.3 0.7 0.5 0 20 40 60 Percent of Lugwere Learners 0 1-10 11-20 21-30 31-40 Letter Sound Score at Endline - Lugwere Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 131 3.1 5.2 1.4 1.2 0.5 0.2 1.0 0.7 1.0 1.6 91.490.1 0.2 0.5 0.7 0.2 0.2 0 20 40 60 80 100 Percent of Lugwere Learners 0 1 2 3 4 5 6 7 9 10 Segment at Baseline - Lugwere Control Treatment 6.9 11.6 2.13.8 3.14.0 3.33.6 3.34.8 65.7 54.9 2.92.9 3.85.0 5.2 2.9 1.7 4.3 1.92.4 0 20 40 60 80 Percent of Lugwere Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - Lugwere Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 132 99.8 99.5 0.2 0.5 0 20 40 60 80 100 Percent of Lugwere Learners 0 1-10 Fluency at Baseline - Lugwere Control Treatment 96.9 95.0 1.4 2.6 1.4 2.1 0.2 0.2 0 20 40 60 80 100 Percent of Lugwere Learners 0 1-10 11-20 21-30 Fluency at Endline - Lugwere Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 133 100.0 100.0 0 20 40 60 80 100 Percent of Lugwere Learners 0 Reading Comprehension at Baseline - Lugwere Control Treatment 97.1 96.0 0.2 2.4 1.0 1.2 1.7 0.5 0 20 40 60 80 100 Percent of Lugwere Learners 0 1 2 3 Reading Comprehension at Endline - Lugwere Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 134 16.7 11.0 25.5 24.6 32.9 35.9 25.0 28.4 0 10 20 30 40 Percent of Lugwere Learners 0 1 2 3 Listening Comprehension at Baseline - Lugwere Control Treatment 7.9 2.4 28.2 27.6 45.3 47.5 18.6 22.6 0 10 20 30 40 50 Percent of Lugwere Learners 0 1 2 3 Listening Comprehension at Endline - Lugwere Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 135 89.0 78.6 11.0 20.3 1.1 0 20 40 60 80 100 Percent of Ngakarimojong Learners 0 1-10 11-20 English Letter Sound Score at Baseline - Ngakarimojong Control Treatment 67.5 26.9 19.5 25.3 12.4 32.2 0.6 10.6 3.8 1.3 0 20 40 60 80 Percent of Ngakarimojong Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Endline - Ngakarimojong Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 136 87.0 70.4 12.7 27.7 0.3 1.6 0.3 0 20 40 60 80 Percent of Ngakarimojong Learners 0 1-10 11-20 21-30 Letter Sound Score at Baseline - Ngakarimojong Control Treatment 27.2 10.9 57.4 32.2 13.6 34.7 1.2 13.1 5.3 2.8 0.9 0 20 40 60 Percent of Ngakarimojong Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 71-80 Letter Sound Score at Endline - Ngakarimojong Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 137 5.15.8 2.43.8 1.73.0 1.71.9 3.82.2 75.3 66.6 1.7 4.7 3.81.6 2.43.6 1.43.0 0.7 3.8 0 20 40 60 80 Percent of Ngakarimojong Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - Ngakarimojong Control Treatment 4.1 2.2 3.64.1 1.2 3.4 3.62.5 3.04.4 56.8 30.9 1.82.5 4.15.6 2.4 4.4 5.95.9 13.6 34.1 0 20 40 60 Percent of Ngakarimojong Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - Ngakarimojong Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 138 100.0 99.7 0.3 0 20 40 60 80 100 Percent of Ngakarimojong Learners 0 1-10 Fluency at Baseline - Ngakarimojong Control Treatment 94.7 80.6 1.2 7.2 1.2 5.0 1.8 3.8 0.6 1.9 0.6 1.3 0.3 0 20 40 60 80 100 Percent of Ngakarimojong Learners 0 1-10 11-20 21-30 31-40 41-50 51-60 Fluency at Endline - Ngakarimojong Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 139 100.0 100.0 0 20 40 60 80 100 Percent of Ngakarimojong Learners 0 Reading Comprehension at Baseline - Ngakarimojong Control Treatment 95.3 84.4 0.6 4.4 1.2 3.8 2.4 4.1 2.8 0.6 0 20 40 60 80 100 Percent of Ngakarimojong Learners 0 1 2 3 4 5 Reading Comprehension at Endline - Ngakarimojong Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 140 9.6 8.8 28.1 23.0 33.6 31.5 28.8 36.7 0 10 20 30 40 Percent of Ngakarimojong Learners 0 1 2 3 Listening Comprehension at Baseline - Ngakarimojong Control Treatment 1.2 0.9 5.9 4.7 16.6 22.8 76.3 71.6 0 20 40 60 80 Percent of Ngakarimojong Learners 0 1 2 3 Listening Comprehension at Endline - Ngakarimojong Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 141 86.4 83.1 12.6 15.6 1.0 1.2 0 20 40 60 80 Percent of Lhukonzo Learners 0 1-10 11-20 English Letter Sound Score at Baseline - Lhukonzo Control Treatment 50.2 47.2 39.3 39.1 9.8 10.8 0.8 1.8 1.1 0 10 20 30 40 50 Percent of Lhukonzo Learners 0 1-10 11-20 21-30 31-40 English Letter Sound Score at Endline - Lhukonzo Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 142 81.7 74.2 16.4 22.1 1.9 3.7 0 20 40 60 80 Percent of Lhukonzo Learners 0 1-10 11-20 Letter Sound Score at Baseline - Lhukonzo Control Treatment 52.5 48.0 36.8 40.6 10.5 9.0 0.3 2.4 0 10 20 30 40 50 Percent of Lhukonzo Learners 0 1-10 11-20 21-30 Letter Sound Score at Endline - Lhukonzo Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 143 4.35.0 2.12.2 0.7 3.2 1.01.2 1.01.7 82.4 70.7 1.21.0 1.22.0 0.5 3.2 1.02.0 4.8 7.7 0 20 40 60 80 Percent of Lhukonzo Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - Lhukonzo Control Treatment 13.0 16.1 4.04.0 4.0 2.1 3.5 1.6 1.51.6 39.0 23.5 2.8 4.2 3.0 5.5 8.07.1 8.3 12.9 13.0 21.4 0 10 20 30 40 Percent of Lhukonzo Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - Lhukonzo Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 144 99.3 99.8 0.7 0.2 0 20 40 60 80 100 Percent of Lhukonzo Learners 0 1-10 Fluency at Baseline - Lhukonzo Control Treatment 88.5 77.0 6.8 16.1 3.0 3.4 1.3 2.9 0.5 0.5 0 20 40 60 80 Percent of Lhukonzo Learners 0 1-10 11-20 21-30 31-40 Fluency at Endline - Lhukonzo Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 145 99.3 100.0 0.2 0.2 0.2 0 20 40 60 80 100 Percent of Lhukonzo Learners 0 1 2 3 Reading Comprehension at Baseline - Lhukonzo Control Treatment 94.5 91.6 2.3 2.4 1.0 4.0 2.3 2.1 0 20 40 60 80 100 Percent of Lhukonzo Learners 0 1 2 3 Reading Comprehension at Endline - Lhukonzo Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 146 14.0 8.2 22.6 17.9 31.2 34.5 32.1 39.5 0 10 20 30 40 Percent of Lhukonzo Learners 0 1 2 3 Listening Comprehension at Baseline - Lhukonzo Control Treatment 2.3 1.1 10.3 7.4 31.8 24.0 55.8 67.5 0 20 40 60 80 Percent of Lhukonzo Learners 0 1 2 3 Listening Comprehension at Endline - Lhukonzo Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 147 88.9 81.5 9.0 16.5 1.7 2.0 0.5 0 20 40 60 80 100 Percent of Lusoga Learners 0 1-10 11-20 21-30 English Letter Sound Score at Baseline - Lusoga Control Treatment 88.3 42.4 9.4 32.3 1.3 15.9 1.0 6.8 2.3 0.3 0 20 40 60 80 Percent of Lusoga Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Endline - Lusoga Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 148 86.4 83.0 11.9 16.0 0.5 0.7 1.0 0.2 0.2 0 20 40 60 80 Percent of Lusoga Learners 0 1-10 11-20 21-30 31-40 Letter Sound Score at Baseline - Lusoga Control Treatment 39.6 20.3 57.6 56.0 2.1 15.6 0.3 6.8 0.5 1.3 0 20 40 60 Percent of Lusoga Learners 0 1-10 11-20 21-30 31-40 Letter Sound Score at Endline - Lusoga Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 149 3.65.7 2.71.2 3.94.0 2.22.7 2.23.7 81.6 69.9 1.7 5.2 1.02.2 0.71.5 0.22.2 0.21.7 0 20 40 60 80 Percent of Lusoga Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Baseline - Lusoga Control Treatment 10.2 12.2 3.92.9 3.43.4 1.8 4.9 3.64.9 59.9 39.3 6.5 8.3 3.44.7 2.1 4.4 1.6 7.3 3.6 7.6 0 20 40 60 Percent of Lusoga Learners 0 1 2 3 4 5 6 7 8 9 10 Segment at Endline - Lusoga Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 150 100.0 99.8 0.2 0 20 40 60 80 100 Percent of Lusoga Learners 0 1-10 Fluency at Baseline - Lusoga Control Treatment 99.0 93.0 1.0 5.5 1.0 0.5 0 20 40 60 80 100 Percent of Lusoga Learners 0 1-10 11-20 21-30 Fluency at Endline - Lusoga Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 151 100.0 100.0 0 20 40 60 80 100 Percent of Lusoga Learners 0 Reading Comprehension at Baseline - Lusoga Control Treatment 100.0 97.9 0.8 0.8 0.3 0.3 0 20 40 60 80 100 Percent of Lusoga Learners 0 1 2 3 4 Reading Comprehension at Endline - Lusoga Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 152 15.7 11.4 21.8 17.0 34.6 31.4 27.8 40.2 0 10 20 30 40 Percent of Lusoga Learners 0 1 2 3 Listening Comprehension at Baseline - Lusoga Control Treatment 4.4 1.8 15.1 8.9 34.1 34.4 46.4 54.9 0 20 40 60 Percent of Lusoga Learners 0 1 2 3 Listening Comprehension at Endline - Lusoga Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 153 90.2 84.4 8.9 14.5 0.7 1.1 0.1 0 20 40 60 80 100 Percent of English Learners 0 1-10 11-20 21-30 English Letter Sound Score at Baseline - English Control Treatment 70.3 43.2 23.7 32.7 5.3 17.1 0.7 4.9 1.7 0.4 0 20 40 60 80 Percent of English Learners 0 1-10 11-20 21-30 31-40 41-50 English Letter Sound Score at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 154 9.4 6.4 53.8 48.7 33.7 39.3 3.0 5.2 0.1 0.3 0 20 40 60 Percent of English Learners 0 1-5 6-10 11-15 16-20 Vocabulary at Baseline - English Control Treatment 0.9 0.3 21.8 21.0 67.2 62.7 9.7 15.0 0.4 1.1 0 20 40 60 80 Percent of English Learners 0 1-5 6-10 11-15 16-20 Vocabulary at Endline - English Control Treatment SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 155 ANNEX D. SAMPLE BALANCE AT ENDLINE SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 156 Table D1: Cluster 1 - Demographics and other characteristics at endline for treatment and control groups. Luganda language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 10.756 10.916 0.160 (0.073) (0.085) % of cases with missing age information 0.003 0.000 -0.003 (0.003) (0.000) Gender of student (female = 1) 0.489 0.522 0.033 (0.027) (0.032) Number of assets (max = 8) 3.348 3.012 -0.336* (0.071) (0.083) Lives with both parents (yes = 1) 0.423 0.426 0.002** (0.027) (0.031) Does not live with mother (yes = 1) 0.381 0.357 -0.024 (0.027) (0.030) Reads at home (yes = 1) 0.724 0.683 -0.041 (0.025) (0.030) Attended preschool (yes = 1) 0.813 0.920 0.106* (0.021) (0.017) Student absent any day in the week prior to assessment (yes = 1) 0.495 0.514 0.019 (0.027) (0.032) Teacher absent any day in the week prior to assessment (yes = 1) 0.373 0.462 0.089 (0.027) (0.032) Gender of teacher (female = 1) 0.727 0.695 -0.032 (0.024) (0.032) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 157 Table D2: Cluster 1 - Demographics and other characteristics at endline for treatment and control groups. Leblango language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 11.858 12.008 0.150 (0.071) (0.063) % of cases with missing age information 0.003 0.000 -0.003 (0.003) (0.000) Gender of student (female = 1) 0.506 0.499 -0.007 (0.026) (0.026) Number of assets (max = 8) 2.725 2.839 0.114 (0.060) (0.056) Lives with both parents (yes = 1) 0.608 0.604 -0.004 (0.026) (0.025) Does not live with mother (yes = 1) 0.231 0.253 0.023 (0.022) (0.022) Reads at home (yes = 1) 0.514 0.564 0.050 (0.026) (0.026) Attended preschool (yes = 1) 0.344 0.383 0.039 (0.025) (0.025) Student absent any day in the week prior to assessment (yes = 1) 0.640 0.627 -0.013 (0.025) (0.025) Teacher absent any day in the week prior to assessment (yes = 1) 0.393 0.385 -0.007 (0.026) (0.025) Gender of teacher (female = 1) 0.363 0.427 0.064 (0.026) (0.027) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 158 Table D3: Cluster 1 - Demographics and other characteristics at endline for treatment and control groups. Ateso language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 11.488 11.388 -0.099 (0.075) (0.061) % of cases with missing age information 0.002 0.000 -0.002 (0.002) (0.000) Gender of student (female = 1) 0.546 0.499 -0.047 (0.025) (0.025) Number of assets (max = 8) 2.472 2.519 0.048 (0.057) (0.055) Lives with both parents (yes = 1) 0.637 0.748 0.111** (0.024) (0.022) Does not live with mother (yes = 1) 0.178 0.159 -0.018 (0.019) (0.019) Reads at home (yes = 1) 0.489 0.425 -0.064 (0.025) (0.025) Attended preschool (yes = 1) 0.252 0.309 0.057 (0.022) (0.023) Student absent any day in the week prior to assessment (yes = 1) 0.594 0.570 -0.024 (0.024) (0.025) Teacher absent any day in the week prior to assessment (yes = 1) 0.360 0.386 0.026 (0.024) (0.025) Gender of teacher (female = 1) 0.238 0.726 0.489 (0.023) (0.025) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 159 Table D4: Cluster 1 - Demographics and other characteristics at endline for treatment and control groups. RR language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 11.665 11.760 0.095 (0.087) (0.077) % of cases with missing age information 0.000 0.003 0.003 (0.000) (0.003) Gender of student (female = 1) 0.529 0.532 0.004 (0.030) (0.025) Number of assets (max = 8) 2.806 2.769 -0.037 (0.074) (0.062) Lives with both parents (yes = 1) 0.626 0.691 0.065 (0.029) (0.024) Does not live with mother (yes = 1) 0.209 0.156 -0.053 (0.024) (0.019) Reads at home (yes = 1) 0.638 0.660 0.022 (0.029) (0.024) Attended preschool (yes = 1) 0.518 0.581 0.063 (0.030) (0.025) Student absent any day in the week prior to assessment (yes = 1) 0.491 0.460 -0.031 (0.030) (0.025) Teacher absent any day in the week prior to assessment (yes = 1) 0.424 0.475 0.051 (0.030) (0.026) Gender of teacher (female = 1) 0.343 0.433 0.090 (0.030) (0.034) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 160 Table D5: Cluster 2 - Demographics and other characteristics at endline for treatment and control groups. Acoli language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 9.972 9.926 -0.046 (0.066) (0.072) % of cases with missing age information 0.002 0.008 0.006 (0.002) (0.004) Gender of student (female = 1) 0.530 0.519 -0.012 (0.022) (0.023) Number of assets (max = 8) 2.650 2.433 -0.217 (0.057) (0.057) Lives with both parents (yes = 1) 0.682 0.675 -0.007 (0.021) (0.021) Does not live with mother (yes = 1) 0.136 0.104 -0.031 (0.015) (0.014) Reads at home (yes = 1) 0.467 0.421 -0.047 (0.022) (0.023) Attended preschool (yes = 1) 0.378 0.308 -0.070 (0.022) (0.021) Student absent any day in the week prior to assessment (yes = 1) 0.428 0.442 0.014 (0.022) (0.023) Teacher absent any day in the week prior to assessment (yes = 1) 0.396 0.409 0.014 (0.022) (0.023) Gender of teacher (female = 1) 0.463 0.437 -0.026 (0.030) (0.035) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 161 Table D6: Cluster 2 - Demographics and other characteristics at endline for treatment and control groups. Lugbarati language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 10.915 11.212 0.297 (0.070) (0.081) % of cases with missing age information 0.002 0.005 0.003 (0.002) (0.003) Gender of student (female = 1) 0.494 0.500 0.006 (0.022) (0.024) Number of assets (max = 8) 2.863 3.113 0.251 (0.057) (0.065) Lives with both parents (yes = 1) 0.518 0.556 0.038 (0.022) (0.024) Does not live with mother (yes = 1) 0.297 0.225 -0.072 (0.020) (0.020) Reads at home (yes = 1) 0.525 0.594 0.069 (0.022) (0.024) Attended preschool (yes = 1) 0.203 0.100 -0.104** (0.017) (0.014) Student absent any day in the week prior to assessment (yes = 1) 0.493 0.477 -0.017 (0.022) (0.024) Teacher absent any day in the week prior to assessment (yes = 1) 0.368 0.383 0.016 (0.021) (0.024) Gender of teacher (female = 1) 0.659 0.465 -0.194 (0.023) (0.028) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 162 Table D7: Cluster 2 - Demographics and other characteristics at endline for treatment and control groups. Lumasaaba language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 10.186 10.090 -0.096 (0.070) (0.081) % of cases with missing age information 0.012 0.009 -0.003 (0.005) (0.004) Gender of student (female = 1) 0.505 0.468 -0.037 (0.022) (0.023) Number of assets (max = 8) 2.334 2.414 0.080 (0.052) (0.058) Lives with both parents (yes = 1) 0.564 0.564 0.000 (0.022) (0.023) Does not live with mother (yes = 1) 0.286 0.275 -0.011 (0.020) (0.021) Reads at home (yes = 1) 0.578 0.587 0.009 (0.022) (0.023) Attended preschool (yes = 1) 0.465 0.510 0.045 (0.022) (0.023) Student absent any day in the week prior to assessment (yes = 1) 0.573 0.538 -0.035 (0.022) (0.023) Teacher absent any day in the week prior to assessment (yes = 1) 0.489 0.439 -0.049 (0.023) (0.023) Gender of teacher (female = 1) 0.507 0.647 0.140 (0.027) (0.027) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 163 Table D8: Cluster 2 - Demographics and other characteristics at endline for treatment and control groups. RR language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 10.354 10.228 -0.126 (0.079) (0.074) % of cases with missing age information 0.012 0.007 -0.004 (0.005) (0.004) Gender of student (female = 1) 0.497 0.502 0.005 (0.022) (0.022) Number of assets (max = 8) 2.930 2.807 -0.123 (0.057) (0.050) Lives with both parents (yes = 1) 0.529 0.474 -0.055 (0.022) (0.022) Does not live with mother (yes = 1) 0.251 0.293 0.042 (0.019) (0.020) Reads at home (yes = 1) 0.663 0.633 -0.031 (0.021) (0.021) Attended preschool (yes = 1) 0.742 0.727 -0.015 (0.019) (0.019) Student absent any day in the week prior to assessment (yes = 1) 0.618 0.547 -0.071* (0.021) (0.021) Teacher absent any day in the week prior to assessment (yes = 1) 0.506 0.551 0.046 (0.022) (0.022) Gender of teacher (female = 1) 0.596 0.677 0.081 (0.026) (0.024) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 164 Table D9: Cluster 3 - Demographics and other characteristics at endline for treatment and control groups. Lusoga language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 8.642 8.767 0.126 (0.080) (0.084) % of cases with missing age information 0.099 0.060 -0.039 (0.015) (0.012) Gender of student (female = 1) 0.487 0.508 0.021 (0.026) (0.026) Number of assets (max = 8) 3.281 3.109 -0.172 (0.068) (0.066) Lives with both parents (yes = 1) 0.390 0.390 0.000 (0.025) (0.025) Does not live with mother (yes = 1) 0.277 0.267 -0.010 (0.023) (0.023) Reads at home (yes = 1) 0.688 0.693 0.005 (0.024) (0.024) Attended preschool (yes = 1) 0.722 0.694 -0.028 (0.023) (0.024) Student absent any day in the week prior to assessment (yes = 1) 0.450 0.484 0.034 (0.025) (0.026) Teacher absent any day in the week prior to assessment (yes = 1) 0.634 0.600 -0.034 (0.025) (0.025) Gender of teacher (female = 1) 0.909 0.932 0.023 (0.016) (0.013) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 165 Table D10: Cluster 3 - Demographics and other characteristics at endline for treatment and control groups. Lugwere language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 9.175 9.011 -0.164 (0.081) (0.073) % of cases with missing age information 0.105 0.133 0.029 (0.015) (0.017) Gender of student (female = 1) 0.506 0.500 -0.006 (0.024) (0.024) Number of assets (max = 8) 2.883 2.876 -0.007 (0.062) (0.058) Lives with both parents (yes = 1) 0.613 0.602 -0.011 (0.024) (0.024) Does not live with mother (yes = 1) 0.197 0.193 -0.004 (0.020) (0.019) Reads at home (yes = 1) 0.523 0.533 0.011 (0.024) (0.024) Attended preschool (yes = 1) 0.594 0.586 -0.007 (0.024) (0.024) Student absent any day in the week prior to assessment (yes = 1) 0.533 0.569 0.035 (0.024) (0.024) Teacher absent any day in the week prior to assessment (yes = 1) 0.559 0.585 0.026 (0.024) (0.024) Gender of teacher (female = 1) 0.713 0.786 0.073 (0.022) (0.020) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 166 Table D11: Cluster 3 - Demographics and other characteristics at endline for treatment and control groups. Ngakarimojong language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 9.872 9.699 -0.172 (0.117) (0.141) % of cases with missing age information 0.025 0.095 0.070 (0.009) (0.023) Gender of student (female = 1) 0.444 0.391 -0.053 (0.028) (0.038) Number of assets (max = 8) 2.541 2.521 -0.020 (0.089) (0.126) Lives with both parents (yes = 1) 0.320 0.379 0.059 (0.026) (0.037) Does not live with mother (yes = 1) 0.354 0.266 -0.088 (0.027) (0.034) Reads at home (yes = 1) 0.328 0.367 0.039 (0.026) (0.037) Attended preschool (yes = 1) 0.327 0.262 -0.065 (0.026) (0.034) Student absent any day in the week prior to assessment (yes = 1) 0.549 0.515 -0.034 (0.028) (0.039) Teacher absent any day in the week prior to assessment (yes = 1) 0.351 0.446 0.095 (0.027) (0.039) Gender of teacher (female = 1) 0.544 0.707 0.164 (0.028) (0.038) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 167 Table D12: Cluster 3 - Demographics and other characteristics at endline for treatment and control groups. Lhukonzo language group. Variable Treatment Control Difference Mean Mean (C – T) (SE) (SE) Age of student 9.831 9.751 -0.080 (0.080) (0.075) % of cases with missing age information 0.016 0.005 -0.011 (0.006) (0.004) Gender of student (female = 1) 0.509 0.510 0.001 (0.026) (0.025) Number of assets (max = 8) 3.166 2.509 -0.657*** (0.081) (0.075) Lives with both parents (yes = 1) 0.394 0.448 0.055 (0.025) (0.025) Does not live with mother (yes = 1) 0.245 0.234 -0.010 (0.022) (0.021) Reads at home (yes = 1) 0.608 0.545 -0.063 (0.025) (0.025) Attended preschool (yes = 1) 0.566 0.572 0.005 (0.026) (0.025) Student absent any day in the week prior to assessment (yes = 1) 0.493 0.505 0.012 (0.026) (0.025) Teacher absent any day in the week prior to assessment (yes = 1) 0.568 0.548 -0.020 (0.026) (0.025) Gender of teacher (female = 1) 0.837 0.453 -0.384* (0.019) (0.025) *** p<0.001, ** p<0.01, * p<0.05 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT Final Report | 168 ANNEX E. INTERACTIONS BETWEEN STUDENT ABSENTEEISM AND SHRP TREATMENT SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 169 Table E1: Cluster 1, SHRP Estimated Effects on Oral Reading Fluency: Interaction with Learner Absenteeism Runyankore Rukiga Luganda Leblango Ateso SHRP Treatment 6.828 4.546 0.876 7.136 (2.267) (2.208) (2.312) (1.560) SHRP Treatment * Absent Learner -0.675 -0.012 -0.574 -1.521 (1.550) (1.695) (1.982) (1.410) Absent Learner -2.202 -0.979 0.374 -0.174 (0.446) (0.489) (0.599) (0.488) F-test: joint significance (p-value) 13.38 (0.0000) 2.60 (0.0745) 0.17 (0.9172) 9.83 (0.0002) Observations 1,281 1,137 1,452 1,559 R-squared 0.570 0.411 0.300 0.219 Robust standard errors in parentheses. Individual controls include age and sex of learner dummy for learner absent previous week household assets index indicators for living with both parents someone at home reads to the learner language spoken at home district dummies and a constant term. Table E2: Cluster 2, SHRP Estimated Effects on Oral Reading Fluency: Interaction with Learner Absenteeism Runyoro/ Rutooro Acoli Lugbarati Lumasaaba SHRP Treatment 8.412 2.843 0.954 2.773 (1.908) (1.344) (1.597) (1.240) SHRP Treatment * Absent Learner -1.940 -0.959 -0.458 -0.561 (1.449) (0.625) (1.268) (0.616) Absent Learner 0.096 -0.227 -0.206 -0.224 (0.360) (0.203) (0.470) (0.163) F-test: joint significance (p-value) 6.81 (0.0014) 4.14 (0.0154) 0.26 (0.8505) 2.48 (0.0843) Observations 1,839 1,734 1,715 1,688 R-squared 0.264 0.056 0.143 0.104 Robust standard errors in parentheses. Individual controls include age and sex of learner dummy for learner absent previous week household assets index indicators for living with both parents someone at home reads to the learner language spoken at home district dummies and a constant term SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 170 Table E3: Cluster 3, SHRP Estimated Effects on Oral Reading Fluency: Interaction with Learner Absenteeism Lusoga Lugwere Ngakari￾mojong Lhukonzo SHRP Treatment 0.478+ 0.055 3.067 1.112 (0.269) (0.273) (2.090) (0.737) SHRP Treatment * Absent Learner -0.188 -0.021 -0.799 -0.758 (0.242) (0.164) (1.374) (0.689) Absent Learner -0.023 0.018 0.308 0.033 (0.019) (0.067) (0.307) (0.104) F-test: joint significance (p-value) 3.01 (0.0476) 0.03 (0.9912) 5.22 (0.0059) 0.84 (0.4840) Observations 1,530 1,634 1,122 1,519 R-squared 0.022 0.019 0.094 0.052 Robust standard errors in parentheses. Individual controls include age and sex of learner dummy for learner absent previous week household assets index indicators for living with both parents someone at home reads to the learner language spoken at home district dummies and a constant term SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 171 ANNEX F. EXPLORATION OF POSSIBLE CAUSES OF UNDERPERFORMANCE OF SHRP SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 172 EXECUTIVE SUMMARY NORC, in collaboration with USAID and the implementing organization, conducted a limited, but detailed, qualitative study to explore four possible explanations for the SHRP program’s small impacts as described in NORC’s Performance and Impact Evaluation reports. These are:  Teacher trainings fell short in fully training or fully motivating teachers to implement SHRP techniques in their classrooms;  Teacher support supervision did not provide sufficient support to ensure that teachers were able to implement techniques effectively in the classroom;  Community mobilization efforts were insufficient to provide the needed support to classroom implementation;  Absenteeism is high among learners and teachers. SHRP heavily relies on teacher capacitation, teacher coaching and learning materials for pupils and it can only produce large effects if the teachers and learners actually meet in class with high frequency. To this end, NORC conducted a limited round of data collection, which consisted of six Focus Group Discussions (FGDs) with 40 project teachers sampled from three districts (Apac, Kiruhura and Kamuli) in order to better understand the nuances surrounding teacher trainings, teacher support supervisions, teachers and learners school attendance, and community involvement during the first four years of the project. The results and recommendations described here below are based on the four aspects of evaluation mentioned above. TEACHER TRAINING: The questions pertaining to teacher trainings aimed at exploring the efficiency and usefulness of the instruction provided during the trainings, the accessibility and persuasiveness of the information at the trainings, and the use of training instruction within the actual classrooms. The findings indicate that all the teachers reported being trained both in content and methods, with themajority having been trained twice for the main SHRP reading and learning course. Each of the trainings lasted at least four to five days. A few teachers received additional refresher training, which lasted one to two days and were mainly conducted by the CCTs. Teachers felt that while the content of the course was comprehensive, too few days were allocated for the training and the frequency of the refresher training courses was also limited. As a result, teachers felt d the information provided during the training was rushed and therefore the teachers did not feel comfortable applying it with the pupils in class. However, there were several, aspects of the content that teachers found helpful in guiding them with pupil's reading and learning. These included pronunciation of letter names and the letter sounds as well as developing lesson plans and schemes of work. Teachers identified several weaknesses that hindered them in teaching the content to the children. These included missing content for Literacy one (Science and SST), limited alignment with the government primary school education curriculum, limited knowledge of cross- referencing by the teachers, and limited time allocated for teaching content, particularly for primary one (PI) children. The teachers felt the RTI reading materials have artificial examples and pictures that do not correspond with the surrounding and local environment where the children live and stay. In regard to the methods, all teachers agreed that they were taught methods that included; "I do, we do, you do"; making of lesson plans; ccooperation and group work discussions. These methods have to some extent helped the teachers plan and organize work for pupils, which they noted helps them learn better. Teachers from all districts indicated that the methods taught were difficult to apply in class for pupils learning how to read. They noted that they encountered problems in writing and reading when it came to pupils from P1, as well as SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 173 additional difficulties with other classes. According to the participating teachers, the overarching challenges in applying the methods are:  application of these methods is time consuming compared to the time allocated for a single lesson;  problems associated with controlling a larger class while applying these methods; and  difficulties associated with cross- referencing due to missing information in the SHRP textbooks. Most teachers felt that they were not fully equipped to apply the new methods and felt that they needed more training to effectively apply the new approaches. TEACHER SUPPORT SUPERVISION: A second topic focused on the role and usefulness of the support supervision received by the teachers to apply SHRP teaching methods and instructional materials in the classroom. Teachers were asked if they had received any support supervision, the frequency of these visits, weaknesses and strengths of the support supervision, and if the support enhanced their teaching techniques in class. All teachers reported receiving some form of support supervision by different supervisors, whom they categorized as internal and external supervisors. The majority of the teachers reported that they had received support supervision from internal supervisors at least twice a term and by external supervisors once a year. The type of support provided during the supervision included: assessment and motivation of teachers and pupils, provision of teaching and learning materials, guidance on how to use the methods of teaching, class control and how to handle children of different specialties, lesson planning, and preparing work plans. The teachers' responses as to whether this support positively enhanced their teaching techniques in class were mixed, given the limited interaction between the teachers and the supervisors. The teachers reported increased teacher pupil motivation, confidence building, skills enhancement, constant supervision (which they reported eliminated laziness), friendly guidance vis-a-vis fault finding, and extra on-job trainings as some of the aspects they liked about support supervision. Some of the aspects they reported hindered effective support supervision included limited frequency of the support visits, unprofessionalism by the supervisors, humiliation and limited motivation by the supervisors. ABSENTEEISM: This topic focused on identifying the major challenges faced by teachers and families that resulted in the absenteeism of teachers and pupils. To obtain this information, the teachers were asked to explain the reasons and consequences of absenteeism and the role it plays in how effective teaching techniques can be in the classrooms. All teachers reported that absenteeism was prevalent among teachers and that this affected the performance of the early grade reading programme, since the success of this project relied on the teachers' presence at school. The leading causes of absenteeism included: sickness, long distance to schools, changes in weather, lack of motivation, delayed payment of salaries, low salaries, and personal/family responsibilities. Teachers reported absenteeism varied depending on the reason or nature of the problem teachers encountered. On average, it was noted that a teacher could be absent for around one week (seven days) and, in case of maternity leave, a teacher could be absent for over two months or even the whole term. All the teachers agreed that teacher absenteeism greatly affected children’s performance, promoted school dropout, interrupted the content or flow of information in teaching, promoted indiscipline among the pupils, and greatly promoted pupil absenteeism. Pupil absenteeism was also rated high. Teachers reported child labour in economic/agricultural activities, parental neglect, a negative parental perception towards SHRP, famine and drought, walking long distances to school, lack of sanitary pads for female pupils during menstruation SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 174 periods, parental child neglect, and corporal punishment at school as the major factors leading to pupil absenteeism. All teachers reported having attempted to talk to some parents about pupil absenteeism. However, there were mixed reactions from parents from all the sampled districts of study. Teachers reported that some parents reacted negatively, and thus teachers did not witness any change in their pupils' rate of absenteeism, while the children of parents who reacted positively were sent back to school. COMMUNITY INVOLVEMENT: The intervention of community mobilization was primarily evaluated by asking teachers about parental involvement in the pupils' education to assess whether the community was properly mobilized in order to reach the intended goals of the program. The majority of teachers reported that there was limited involvement by the parents in the SHRP program. The reasons given for limited involvement by parents included: insufficient knowledge about the program, a negative perception that children were being taught in the local language (vernacular), and inadequate mobilization of the community to engage in the program. RECOMMENDATIONS Recommendations emerging from the FGDs covered such topics as teacher training, teaching materials, teacher motivation and support, and combating pupil absenteeism, and included those that could be addressed within such a program as SHRP and those outside the purview of such a program. A selection the most salient recommendations is provided below. Redesign the training course to allow for more exposure and understanding of the methods and content. Most teachers felt that they could not effectively apply the new methods and content to the children in class because they were not fully equipped to do so due to a short period of exposure to the course. Limit transfer of SHRP trained teachers. The district leadership in the SHRP implementing schools should minimize transferring teachers who are trained in the SHRP early grade reading and leaning programme. SHRP English books should be revised to include detailed grammar and literacy one. All the missing content in the SHRP books, particularly grammar and literacy one, should be identified and included in the revised version of the textbooks using a participatory approach. Existing errors in the text books should be identified and corrected/ Align SHRP textbooks with the National primary schools curriculum. Explore ways to motivate teachers. The teachers noted that they are not recognized for their contribution to learning and reading, which has led to low teacher morale. Improve teacher support supervision: At least each teacher had received some form of support supervision from either the internal or external supervisors, but reported this had not been adequate in enabling teachers to apply the content and new methods to the classrooms that had been taught. Improve supply of teaching and learning materials in a sustainable manner. Teachers reported receiving limited support through the consistent provision of teaching, learning and instruction materials. Adapt alternative methods of disciplining children. Corporal punishment in schools was cited as one of the main causes of pupil absenteeism. Corporal punishment inflicts pain and fear upon the children. Teachers in the SHRP schools need to be monitored to ensure that they are adopt alternative methods of disciplining children, as stipulated in the national guidelines on discipline. Work to change the negative parental perception towards SHRP. Teachers from the study districts noted that parents’ attitude towards SHRP was negative. The community considered SHRP's Early Grading Reading and Learning a vernacular school programme. The RTI SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 175 leadership should develop a comprehensive intervention so as to change this negative perception in the community. 1.0 BACKGROUND USAID/Uganda’s School Health and Reading Program (SHRP) is a seven-year USAID-funded cooperative agreement implemented by RTI International in collaboration with several partners.16 The overall objective of the program is to increase literacy and health seeking behaviors. Underlying this is USAID’s goal to support the Government of Uganda (GoU) in developing, implementing, assessing, and bringing to scale a successful approach to reading instruction and to deliver the goal of the Ministry of Education and Sports (MoES) toward achieving a Ugandan-led “Literacy Policy.” To this end, the program is building institutional capacity, supporting policy development, and helping institutionalize the training, support structures, and policies necessary for sustainability. To achieve the stated objectives, the program has two key results: 1) Improved Early Grade Reading and Transition to English, and 2) Improved HIV/AIDS Knowledge, Attitudes, and Practices. This research focuses on the first result of increasing the early grade learners’ ability to read. The development hypothesis of this objective is that by focusing interventions on the nexus of language, pedagogy, and instructional materials, USAID can significantly improve students’ early grade reading and literacy scores. To this end, SHRP created local language early grade reading materials in twelve local languages and English for P1 – P4, totaling 52 pupil primers and 52 teachers’ guides with scripted lesson plans which are now being scaled up beyond SHRP’s 31 districts17 into 55 other districts and the Global Partnership for Education, covering 86 of Uganda’s 112 districts, almost 80% of the country. SHRP has worked to build up a cadre of master trainers and district level trainers who in turn train and provide ongoing support to teachers. SHRP has trained 14,210 teachers, 9,277 head teachers, and approximately 153 coordinating center tutors (CCTs) in early grade reading methodologies and leadership. Over the life of the program, May 2012-August 2019, reading improvements are being directly supported in 3,476 schools by working through the established MoES systems to train Primary Teacher College (PTC) tutors, coordinating center tutors (CCTs), district and areas inspectors and head teachers to train and support teachers to effectively teach reading. Another important component of the program is supporting the National Curriculum Development Centre (NCDC) to develop language boards, reading instructional materials, and pupil primers in both English and 12 local languages as shown in Table 1 below. 16 SIL Language Education and Development supports local language reading, World Education Inc. (WEI) supports HIV/AIDS and health programming, and Perkins International which supports Special Needs Education. Other partners include Voluntary Services Overseas (VSO) which supports continuous professional development in five of SHRP’s 31 districts, and the International Book Bank and Books for Africa which provide supplementary reading materials for cost share. The Center for Social Research supported EGRA implementation through May 2016. 17 The languages and districts were selected by the MoES. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 176 Table 15. SHRP Clusters, Languages, Regions and Districts Local Language Region Districts Luganda Central Wakiso, Gomba Runyankore/Rukiga South West Kiruhura, Bushenyi, Kabale Ateso Eastern Kumi, Katakwi, Serere Leblango Northern Apac, Lira, Kole Runyoro/Rutoro Mid-Western Masindi, Kyenjojo, Kabarole Acholi Mid-Northern Gulu, Pader, Kitgum Lugbara West Nile Arua Lumasaaba Mid-Eastern Mbale, Sironko, Manafwa Lugwere Mid-Eastern Budaka, Pallisa, Kibuku Ngakarimojong North East Nakapiripirit, Napak, Abim Lukhonzo South West Kasese Lusoga East Central Iganga and Kamuli With the objective of learning about the effect of this program, USAID engaged NORC at the University of Chicago, in collaboration with Panagora, to conduct the Impact and Performance Evaluation of USAID/Uganda's SHRP. NORC has been evaluating the program annually since 2013. 1.1 PROBLEM STATEMENT SHRP is regarded positively for having pioneered in Uganda what is generally viewed as a promising local language early grade reading initiative while working very collaboratively with all elements of the government education system. However, the impact evaluation undertaken by NORC team has uncovered generally disappointing results in early grade reading. More specifically: SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 177  Progress made by students in the treatment group with regard to letter sound identification, fluency, and reading comprehension subtasks of EGRA were generally modest in magnitude and the differences between the treatment and control group were often not statistically significant.  Students benefiting from SHRP generally failed to reach the oral reading fluency targets set in the SHRP Performance Management Plan. While survey data, KIIs, and FGDs conducted during the evaluation have pointed to some possible reasons for the low impacts, this study proposes to delve deeper into some possible reasons. 1.2 OBJECTIVES AND RESEARCH QUESTION NORC is working closely with USAID and the implementing organization to conduct a limited, but detailed, qualitative study to explore four possible explanations for the SHRP program’s small impacts as described in NORC’s Performance and Impact Evaluation reports. These are: 1. Teacher trainings fell short in fully training or fully motivating teachers to implement SHRP techniques in their classrooms. 2. Teacher support supervision did not provide sufficient support to ensure that teachers were able to implement techniques effectively in the classroom. 3. Community mobilization efforts were insufficient to provide the needed support to classroom implementation. 4. Absenteeism is high among learners and teachers. SHRP heavily relies on teacher capacitation, teacher coaching and learning materials for pupils and it can only produce large effects if the teachers and learners actually meet in class with high frequency. Many of these explanations for the less-than-anticipated impacts of SHRP interventions on reading outcomes emerged during the interviews conducted for the impact and performance evaluations, but they have yet to be fully explored. Thus, the research questions motivating this study are: 1. To what degree did each of the four project-based explanations identified contribute to SHRP’s overall meager impacts on early grade reading? 2. Are there explanations, either project-based or contextual, other than those identified during the performance evaluation that meaningfully contributed to SHRP’s overall meager impacts on early grade reading? 3. Do factors such as geography, languages, or variance in program implementation affect which explanation(s), whether project-based or contextual, meaningfully contributed to SHRP’s overall meager impacts on early grade reading? The study commenced with a review of the previously collected data on these three themes, aiming to focus on how and why each of these topics contributed to the projects observed impacts. Alongside the review, NORC conduct a limited round of data collection, through Focus Group Discussions (FGDs) with project teachers in order to better understand the nuances surrounding teacher trainings, teacher support supervisions, teachers and learners school attendance, and community involvement during the first four years of the project. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 178 2.0 METHODOLOGY The study adapted a qualitative research design which utilized a Focus Group Discussion (FGD) method among teachers from 3 sampled districts. We conducted a review of available data and reports, collected over the past 4 years by SHRP and the NORC/Panagora evaluation team, on the nature and efficiency of the project’s teacher trainings, teacher support supervisions, and community mobilization in an effort to understand SHRP’s limited impact on student reading skills. The review was drawn on reports and data produced by SHRP, including available reports on training and support supervision, quarterly reports, and annual reports to compile information on what did and did not work with respect to these SHRP interventions. Using the information in these reports and participant evaluations of SHRP workshops (if they took place), as well as the rich data that NORC/Panagora evaluation team collected through the continuous evaluation, we assessed, to the extent possible, the quality of the trainings, the accessibility of the training materials for teachers, the effectiveness of the teacher support supervisions, and the extent to which community mobilization was implemented. Possible resources included:  SHRP project reports.  Data from Continuous Evaluation activities.  SHRP monitoring data.  SHRP post-training evaluation sheets. 2.1 REVIEW OF RELATED LITERATURE There is mixed empirical evidence on whether teacher trainings improve student achievement. Glewwe et al. (2013) review high quality studies conducted between 1990 and 2010 on teacher training (and other types of school interventions) and find mixed results for this type of programs. As Bruns & Luque (2015) point out, given the heterogeneity of teachers’ characteristics and training programs’ content and intensity, the lack of consensus on the effects of teacher training is hardly surprising. However, some interventions that follow a comprehensive approach on teacher training show positive impacts. For example, Chay et al. (2005) use a regression discontinuity design to evaluate the Chilean P-900, an intervention targeting low performing schools that provided teacher training, infrastructure improvement, textbooks and other instruction materials, and tutoring for low performing students. The evaluation found positive effects on students’ test scores of 0.2 standard deviations. More recently, Piper & Korda (2011) evaluate a program in Liberia that provided teacher training through a combination of capacity building workshops, on￾going regular feedback, as well as other community outreach activities. Using randomization as their identification strategy, the authors find that the program improves reading scores by 0.79 standard deviations. Menendez and Dayaratna (2016) evaluate a similar intervention in Uganda using an experimental design but find small effects on reading fluency among third graders exposed to the program since the beginning of their primary education. Lucas et al. (2014) compare the effect of fielding a teacher training intervention in Uganda and Kenya. They exploit experimental designs in each country to evaluate a program that included teacher training, instruction materials and ongoing mentoring for teachers. Lucas and her coauthors find significant effects for Uganda of approximately 0.2 standard deviation, but much smaller effects for Kenya. Oliveira & Carnoy (2015) use a triple difference approach to evaluate SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 179 Pacto pela Alfabetizacao na Idade Certa, an early grade reading program in Brazil that provided teacher training and reading materials to schools, combined with monetary incentives based on student performance in standardized exams; they find effects of 0.08 and 0.14 standard deviations for Portuguese and math, respectively. What these programs have in common is a comprehensive approach to teacher professional development. Their interventions do not simply provide teacher training, but also offer, in parallel, a series of teacher support resources, including regular feedback and teaching materials. 2.2 FOCUS GROUP DISCUSSIONS WITH TEACHERS In tandem with NORC’s final Impact Evaluation and the above mentioned review, NORC conducted a limited round of Focus Group Discussions (FGDs) with teachers who benefited from the SHRP interventions. While the main evaluation already utilized quantitative and qualitative methods, namely questionnaires and key informant interviews, the FGDs were conducted to obtain an in-depth understanding and an overview of the range of different views and reasons as to why the performance of the project had limited results. These FGDs covered a variety of respondents and ensured that multiple language groups were represented. The FGDs were designed to gather in-depth information about teacher experiences in SHRP training workshops and the classroom; specifically, the positive and negative aspects of the training, challenges implementing a new curriculum and teaching methods in the classroom, barriers and successes to uptake in the classroom, parents’ involvement, and awareness about the importance of reading. 2. 2 .1 SAMPLE SELECTION AND FOCUS GROUP STRUCTURE As mentioned above, the FGDs included a diverse group of teachers from three districts.18 The districts were purposefully selected in order to represent the three different SHRP clusters, different geographical regions, and different languages. The resulting selected districts were: Kiruhura, Pader, and Kamuli. Two Coordinating Centers (CCs) that received the SHRP interventions were randomly selected from each of the three districts. Whenever possible, four schools were then randomly selected per CC. However, some of the randomly selected CCs only oversaw two to three schools, in which case a random selection was not possible and all schools were selected. Finally, two teachers per selected school were included in the FGDs, for a total of 40 teachers. The teachers were purposively selected to include both male and female teachers who participated in the SHRP training and were implementing the intervention. In creating the sample, NORC also ensured that schools from different language areas were selected. 18 The number of districts has been decided taking into account the available budget. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 180 Table 2: Districts and Schools that participated in the FGDs District School Gender F M Kiruhura Buhembe P/S 1 1 Magondo P/S 0 2 Buteraniro P/S 0 2 Kitengyeto P/S 0 2 Mirama P/S 0 2 Nyamambo P/S 0 2 Pader Laceko-cot P/S 0 2 Lacor P/S 0 2 Ogom P/S 1 1 Wang Opok P/S 1 1 Apiri P/S 0 2 Kilak Corner P/S 0 2 Pader Labongo P.S 1 1 Kamuli Bukitimbo P/S 2 0 Buwagi P/S 2 0 Buwala P/S 1 1 Luzinga COU 2 0 Nagwenyi P/S 2 0 Nalango P/S 2 0 St. Kalori Namaganda 2 0 Total 17 23 2.2.2 FOCUS GROUP CONTENT The questions asked in the FGDs were related to teacher trainings, instructional materials, teachers support supervision, classroom practices, absenteeism, and interactions with parents and the community. Each of these topics were explored in-depth within the FGDs. FGD guides are included in Appendix B of this document. 2.2.3 TEACHER TRAININGS The questions pertaining to teacher trainings were aimed at exploring the efficiency and usefulness of the instruction provided during the trainings, the accessibility and persuasiveness of the information at the trainings, and the use of training instruction in the actual classrooms. In regards to training instruction, the FGDs sought to better understand the quality of teacher trainings. For example, what did teachers find to be useful and positive about the training? What was less useful? At the end of the training, did teachers feel equipped to tackle a new curriculum in the classroom? In addition, the FGDs also aimed at understanding whether SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 181 teachers believed the training that was provided was sufficient to enable them to meet the expectations of the new curriculum. The FGDs also aimed to understand how accessible the teachers found the trainings. Teachers were asked about the amount of knowledge they actually gained during the trainings. Their responses, in conjunction with training documentation provided by SHRP, could help NORC better understand any gaps in knowledge between the instructors and the teachers during the trainings. The FGDs also attempted to gauge the persuasiveness of the instruction given at the trainings in order to understand the level of up-take seen in classrooms. For example, if a teacher was not persuaded by the training and did not think the presented methods were the best, they would not be motivated to practice these techniques in their classroom. The FGD protocols also probed deeply into how the techniques presented in the trainings were translated into classroom practice. Within this field of inquiry, the FGDs focused primarily on how often teachers were able to incorporate the techniques into their classroom instruction, and if they felt the techniques were executed to their full potential when they were used in the classroom. 2.2.4 TEACHER SUPPORT SUPERVISION A second aspect the FGDs focused on was the role and usefulness of the support supervision received by teachers when applying SHRP teaching methods and instructional materials in the classroom. The FGDs commenced by asking about the frequency of the support supervision visits and their content, and determined whether teachers received the on-the-job support they needed to apply what they learned during training. Based on the responses, the FGDs looked at the weakness and/or strengths of the support supervisions, and probed teachers on how the supervisions might be improved for future projects. 2.2.5 ABSENTEEISM The FGDs included the topic of absenteeism. Many issues faced by teachers and families resulted in absenteeism of teachers and pupils. The discussions focused on the reasons and consequences of absenteeism and the role it played on the effectiveness of the teaching techniques in the classrooms. 2.2.6 COMMUNITY MOBILIZATION The topic of community mobilization was primarily evaluated by asking teachers about parental involvement in student education. The level of parental involvement in a student’s education would help assess whether the community was properly mobilized in order to reach the intended goals of the program. Teachers were asked about which actions, and by whom, can be helpful in improving community mobilization related to reading. 2.3 DATA ANALYSIS Data from the FGDs was transcribed to identify emerging themes. The evaluation team utilized the NVivo 11- Computer-Assisted Qualitative Data Analysis Software (CAQDAS) to analyze the data that was collected during fieldwork.. Data cleaning and verification was conducted for accuracy and reliability. In order to determine the natural "meaning units" as expressed by the respondents, the research team read and reread the transcripts and then identified areas of the interviews and other tools that highlighted the participant experience in relation to the research area. Central themes emerged and were used to group findings under each of the research SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 182 questions. Data was allocated to the appropriate questions in a systematic manner and essential descriptive statements made to create meaning. Following transcription of the audios, NVIVO 11 qualitative data analysis software was used to code the information in nineteen (19) thematic nodes and 16 sub-thematic nodes, for a total of thirty five (35) thematic nodes. Nodes were selected by grouping common themes into categories. Two coding cycles were done using the Saldaña (2009) model and a designed codebook to represent the analytic scheme, which guided the data analysis process performed in NVIVO 11. Word clouds and word frequency tables were also developed. The data analysis process began with the development of a codebook, which was used as a guiding tool for the coding process. As suggested by Saldana (2009), the codebook includes the source used to develop descriptions of the variables/nodes used for coding. The table below presents the codebook with three columns. The first contains the variables used as nodes, the second column is the description of the data, which was included in each of the nodes, and the third column has examples of the excerpts that would be included in each nodes. This coding process is an example of what Glesne (2006) recommends when conducting qualitative research in order to create a method for dealing with the data collected and father description and analysis. After reading through all the transcripts, 15 nodes/codes and 16 sub-nodes/codes were registered in the codebook. Table 3: Codebook of Exploration of Possible Causes of Underperformance of the School Health and Reading Program (SHRP) NO CODE DEFINITION EXAMPLE 1 Teacher exposure to different workshops Were teachers trained, and, if so, how many times? I was trained 3 times, that is every end of term, 2 Content Defined domain of knowledge and skill teachers got from SHRP Text books, picture cards, flash cards, letter sounds Strength of the content Benefits or positive aspects of the content to children The content in the books help children learn faster since they can observe what the teacher is teaching them Weakness of the content. Gaps or negative aspects of the content. Limited numbers of text books, not in line with the national curriculum, restricted to only reading but not passing exams. 3 Methods taught in the SHRP training. Which methods of teaching where teachers trained in during the SHRP teacher trainings? The collaborative learning which is I do, We do, You do, Group Discussions, Scaffolding, guided discovery. Application of methods in class room. Did teachers apply the methods they learned in I use I do We do You do whenever I am teaching my pupils SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 183 their classes? If so, which methods and how. Strength of the methods. Benefits accrued from the methods taught. I do We do You do, helps pupils learn faster because it is participatory. It involves the pupils. Weakness of the methods. Hindrance in passing on the content using the methods learned during training. Slow learners do not benefit from group discussion method, I do We do You do wastes time. 4 Teachers’ application of the new approach. How equipped and ready where teachers to apply the new approach of teaching? Not well equipped because when teaching, it seemed like there was another method needed yet I lacked knowledge about it. I was not well equipped because the training duration was not enough and I did not understand what was taught in the trainings. 5 Contribution of teaching and learning materials/aids to children’s learning and reading. How have the teaching and learning materials helped teachers pass on their knowledge about reading to learners? Limited the performance of children since it is not in line with the national curriculum. 6 Recommendations to improve training Teachers’ suggestions on how to enable them to apply content and methods more adequately and comfortably in the classrooms. Motivation of teacher’s and close supervision. More frequency of training and duration 7 Support supervision Do teachers receive support supervision? If so from who and how frequently? Head teachers supervise 2-3 times a term while DIS, DEO, FA, RTI come once a term. Nature of support supervision The services teachers receive from their support supervisors Provide teaching and learning materials, offer advice and corrections. Strength of support supervision/ how it enhances teaching Aspects teachers liked about support supervision Confidence building, skills development, pupil motivation. Weakness of support supervision Aspects teachers did not like about support supervision Unethical words used against teachers, limited time given to teachers, SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 184 limited number of times of visits by RTI. 8 Suggestions to improvement of support supervision Teachers’ suggestions on how to make support supervision more adequate. Regular supervision, enough time with the teachers, provision of teaching and learning materials. 9 Reasons for teacher absenteeism Main aspects that keep teachers out of school for given periods or days. Long distances to school, sickness and lack of residential houses for teachers at school. Effects of teacher absenteeism on pupils’ learning to read. Outcomes of teacher absenteeism to the pupils’ learning and reading. Pupil’s poor performance and increased pupil absenteeism. Recommendations to reduce teacher absenteeism. Teachers’ suggestions on how to reduce teacher absenteeism. Construction of teachers/staff quarters at school. Increase teacher motivation and regular supervision. 10 Reasons for pupil absenteeism Main aspects that keep children out of school for given periods or days. Famine and drought, parental influence, child labor, corporal punishment. Effects of pupil absenteeism on their learning to read Outcomes of pupil absenteeism to their learning and reading. Poor performance, school dropout Recommendation to reduce pupil absenteeism Teachers’ suggestions on how to increase pupils school attendance. Adapt alternative methods of disciplining. Teachers should use attractive teaching methods to attract children to school 11 Teachers roles in engaging Parents to address absenteeism Did the teachers ever talk to parents about pupil absenteeism and what the parents say? Did it create any change? Parents say SHRP is a Local language program which will not help their children to learn. 12 Community involvement in early reading Is the community supporting/participating in SHRP? If so, when and how were they brought on board? Some community people have limited knowledge about SHRP. They have a negative attitude about it as a vernacular program. Reasons for community involvement Reasons why the community participated in SHRP Parents are the custodians of the schools and therefore they should participate in SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 185 SHRP and also ensure ownership. Ways community supports early reading Activities the community takes part in to support SHRP Parents provide their children with local language reading materials Impact of community involvement The outcomes of community participation in SHRP Pupils have improved in reading because their parents provide and help them with school reading materials. Suggestions to improve community involvement Teachers’ suggestions to increase the community’s interest and participation in SHRP. Continuous sensitization of community members about their role and benefits of SHRP. Following the construction of a codebook, data analysis began with the first and second coding cycles. After coding the information into the thirty two thematic nodes, a comparative analysis of pre and post data was performed using Nvivo 11.0 qualitative analysis software for text-based data. Finally, frequency tables and word clouds were constructed to analyze the data. These visual representations helped us understand the data and observe overall patterns in the research without getting lost in details (Glesne, 2006) The analysis process began by making meaningful connections (thematic grouping), and continued with the transformation of data by describing, analyzing and interpreting it as suggested by Glesne, (2006). The data was transformed from its original form (as it was acquired) into information which is a form that communicates findings and results (Glesne, 2006) Table 4 Summary of Study Node Count/frequency Node Name Sources References Support supervision 3 119 Methods taught in the SHRP training 3 88 Reasons for teacher absenteeism 3 82 Reasons for pupil absenteeism 3 71 Community involvement in early reading 3 66 Content 2 48 Nature of support supervision 3 37 Weakness of support supervision 3 34 Teacher exposure to different workshops 3 34 Teachers’ application of the new approach 3 34 Weakness of the content 2 29 Strength of support supervision and how it enhances teaching 3 29 Teachers roles in engaging Parents to address absenteeism 3 29 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 186 Node Name Sources References Strength of the methods 3 25 Suggestions to reduce teacher absenteeism 3 22 Suggestions to improvement of support supervision 3 21 Application of methods in class 3 20 Recommendations to reduce pupil absenteeism 3 19 Suggestions to improve community involvement 3 18 Effects of teacher absenteeism on pupils learning to read 3 14 Ways community supports early reading 2 12 Impact of community involvement 2 11 Effects of pupil absenteeism on their learning to read 4 11 Weakness of the methods 1 8 Reasons for community involvement 1 5 Strength of the content 2 5 Contribution of teaching and learning materials aids to children's learning and reading 1 5 Suggestions to improve training 2 5 Upon the completion of the data coding, it was noted that out of the three sources of data (FGDs-SHRP-Pader, FGDs-SHRP-Kamuli and FGDs-SHRP-Kiruhura), “support supervision” was the most talked about thematic node, with a node frequency of 119, followed by “methods taught in the SHRP trainings”, with a node count of 88 and “reasons for teacher absenteeism” with a node count of 82. On the other end, “contribution of teaching and learning materials” aids to children’s learning and reading” and “suggestions to improve training” were the least talked about thematic nodes with the node frequency of 5 from 1 source and 2 sources respectively. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 187 Figure 1: Flow chart of coding cycles and data analysis process. 2.4 ETHICS NORC’s IRB reviewed the research proposal, protocols, and all instruments for the study and approved this qualitative research endeavor as Non-Human Subject Research. The protocol was also submitted to the Makerere University College of Humanities and Social Sciences Research Ethics Committee ( REC) for review and approval. TEACHERS CONSENT TO PARTICIPATE IN THE STUDY AND CONFIDENTIALITY We secured a letter from the Ministry of Education and Sports introducing the study and requesting district authorities to offer their support. The letter was addressed to the Chief Administrative Officer (CAO), who endorsed it. Copies were delivered to the District Education Officer, District Inspector of schools and respective Coordinating Centre Tutors. Researchers called school teachers ahead of time to schedule the FGDs. Participants were informed beforehand that data from the FGDs would be audio-recorded and transcribed using professional transcriptionists and this was clearly stated in the consent forms. They were also informed that the audio tapes would be kept securely locked away and would be destroyed as soon as the data had been transcribed off them. No names would be recorded. Appendix A contains the consent form that was used to communicate these terms to the teachers and obtain their consent to participate. The consent forms for teachers were not be translated and were, therefore, provided in English, since all teachers spoke and understood English sufficiently well. SECURE STORAGE AND TRANSFER OF DATA As previously mentioned, an audio recording of each FGDs was made, which served as the basis for transcription. All audio recordings were securely stored via a physical lock and key prior to and during the transcription phase. Once each transcription was completed and verified, the recording was destroyed. The transcripts were then coded by the field team. Upon completion of the coding phase, both the transcripts and the coded data were transmitted to NORC researchers via a secure SFTP connection. The data was stored in a password￾protected project folder, with access restricted to include only NORC researchers working on the task. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 188 3.0 EVALUATION RESULTS The findings in this section are based on the four areas that required further exploration in regard to the possible causes of underperformance of the School Health and Reading Program (SHRP) based on the impact quantitative report. The four programme areas of investigation were teacher training, teacher support supervision, community mobilization and absenteeism among teachers and pupils 3.1 TEACHER TRAINING The questions pertaining to teacher trainings aimed at exploring the efficiency and usefulness of the instruction provided during the trainings, the accessibility and persuasiveness of the information at the trainings, and the use of training instructions within the actual classrooms. From the word cloud developed , teachers mentioned the different times that they were trained '28', they also talked about the different days '21' whereby some teachers mentioned that the training lasted for 4 to 5 days while the refresher trainings lasted for fewer days approximately two days. They went for different workshops and the period of times that the trainings '17' lasted. They further recorded the different months when these trainings took place like in January, and May as illustrated in the word cloud below. Figure 2: Word cloud on teacher’s exposure to different SHRP workshops/ trainings Teacher exposure to different workshops This study explored how exposed the teachers were to the SHRP training programme. Overall, all teachers reported to have been trained both in content and methods and majority were trained twice for the main SHRP reading and learning course. Each of the trainings lasted at least four to five days. A few teachers received additional refresher-training which lasted one to two days and were mainly conducted by the CCTs. Teachers were trained at different intervals according to the classes that they taught For example, the Primary One teachers were trained differently from Primary Two, Primary Three and Primary Four teachers. Most of the teachers were trained during holidays. The main training courses were conducted by the SHRP staff, CCTS and district education officers. Word Length Count Trained 7 28 Days 4 21 Training 8 17 January 7 16 May 3 12 Four 4 11 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 189 BOX 1: Teachers voices on exposure to training I was trained the first time in January 2016, at Bishop Willis in Iganga. The training lasted about four days. In the same year, I went back for the second time at Bishop Willis, for another four-day training session during the holidays.” FGD-SHRP-Kamuli “For P.2 class we were trained in 2016. I trained once in January at Iganga S.S. The CCTs and the District Education Officers trained us. Then later other additional workshops were done at Bupadhengo Primary School by the CCT.” FGD-SHRP-Kamuli “We Buwagi Primary School staff were trained twice in 2015 by the CCTs and the DEOs. They trained us for four days during the first training and again four days during the second training. However, the CCT continued to give us additional one-day training sessions, about four times. This also took place at Bupadhengo Primary School. We always go to Bupadhengo once every term either at the beginning of the term or during the term.” FGD￾SHRP-Kamuli “I was trained for five days first before I was taken for a refresher course for another five days. It was during holidays, I think in January 2016. One was the main training and the second was the refresher training.” FGD￾SHRP-Pader “…the trainings I remember were in May 2015, another one was January 2016 and another in May 2016.” FGD-SHRP-Kiruhura “I remember the first workshop was conducted in January 2014. Actually that was the second workshop.…I don’t remember when these workshops were conducted.” FGD-SHRP-Kiruhura. The majority of the teachers noted that they were not fully equipped to comfortably apply the methods and content they had been taught in the classroom. They felt that while the content of the course was comprehensive, the days allocated for the training were few. They also said they did not sufficiently understand all the content and methods to apply them in class. They said the facilitators rushed courses because they were also pressured to finish the entire course in the allocated timeframe. BOX 2:Teachers voices on facilitation during the training "The facilitators were a bit fast in teaching these new methods of learning, which was a challenge to the teachers. Most of them did not capture all that was taught during the main trainings and the refresher trainings." FGD￾SHRP-Pader “I was trained twice from Iganga in 2015 (in January and in May) ,each session lasting three days. Since then we have never been trained again. I feel we need to be trained again because the training was not adequate enough.” FGD-SHRP-Kamuli “I did not get all the methods they taught us, because I participated in a refresher course where we had only two days of training; because the facilitators were rushing and the content they were supposed to teach in a week was taught in two days. Therefore it is not easy to apply these methods in the classroom.” FGD-SHRP-Kiruhura “The facilitators were somewhat fast during the sessions. Some facilitators were a bit fast and I was partly left to consult my neighbor on what was being delivered at some moments.” FGD-SHRP-Pader SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 190 3.1.1 CONTENT TAUGHT DURING SHRP TRAINING Word cloud on the content taught during SHRP training The word children was highly mentioned with the word count of '67' while explaining the contribution of content to the children's learning and reading. Teachers noted that the children were provided with text books '66' to help in the learning and reading. However, the content '60' in the books was not completely aligned with the national curriculum which therefore hindered child learning and reading in some aspects. During the discussion about content, teachers also reported that when teaching English '50', they were not allowed to translate any word into the local language for the children to easily understand the content so as to enable the learn reading '36' as illustrated in the figure below. Figure 3: Word cloud on content taught during SHRP workshops/ trainings When teachers attended the SHRP trainings, they were trained on the content that enabled them to enhance the pupils’ early grade reading and learning. The content included letter names, letter sounds, and oral literacy 1 and 2. During the training teachers were also provided with the learning materials/text books and were also taught on child health, particularly on how to handle impaired children (deaf and blind) and children infected with HIV/ AIDS. All the teachers appreciated the content that was contained in the text books, although they could not comfortably apply all the content as already mentioned above. Below is a description of the content teachers received and their responses on its relevance and applicability. Letter names and sounds Teachers were taught letter names and sounds and how to pronounce each letter and sound out each letter. This was done in order for the teachers to grasp the sounds and apply this later in the classroom. This became successful when they were availed with books that guided them in the pronunciation of the words. They were also taught about the alphabetical principle and word segmentation. “I learnt to use Proper names of sounds of letters and their pronunciation. Previously, I used to pronounce letters differently but I was trained in their proper pronunciation. I learnt to use proper sounds and teach my pupils correct syllables and illustrations.” FGD-SHRP-Kamuli Word Length Count Children 8 67 Books 5 66 Content 7 60 Child 5 50 English 7 50 Reading 7 36 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 191 ”We learnt about names of letters like ‘ezinadyagyeni A’ (my name is A). We had books to guide us, most especially with the pronunciation of words. You find some words have double letters like Abaana (children), ‘Abooho’ so those books helped us very much, especially with words that had double letters and single letters.” FGD-SHRP-Kiruhura Teachers were also taught steps that they could follow when teaching the pupils, which is the alphabetical principle. This also included the way combined letters, if rhymed, can form a word. “The other thing was about reading; they taught us about five principles of Early Grade Reading in P1. The first step was the ‘alphabetical principle’, whereby we were told to go and teach children alphabetic letters.” FGD-SHRP-Kiruhura In regard to letter names and sounds, apart from the alphabetic principle teachers were taught that for a child to easily understand and pronounce a word, there is need for word segmentation. The word is subdivided into phases to help a child learn how to pronounce words, which has helped children to read words in their local languages. “We were also trained in names of letters and sounds of letters, whereby we had a book called “Kanyarurimi”, which is used in teaching local language on how to segment words. For example, in LL if one wanted to say “okutemba” (climbing), we learnt that if the word is segmented, it teaches children how to learn like o-ku-te-mba. We also teach children how to combine the segmented words to make a single word, so this helped children to read words in their local language.” FGD-SHRP-Kiruhura Oral literacy/story composition Teachers were trained in oral literacy and how they can help children in story prediction; giving a story a title, theme, sub-theme and how to conclude the stories. This helped children to plan, think and later talk about a given topic. “I was equipped in creating short stories and giving a title, theme, sub-theme for the stories. Children are able to predict how the story will end. They are able to create a story and also to predict what will happen next in a story and how it will end.” FGD-SHRP-Kamuli “What we were trained in include; Amakulu, Ekitebyo that is oral literacy, then we have Literacy One and Literacy Two; then English remains as English. In other words, English is taught in English. Makulu is news reading. Here, children take part in news reading, drawing and other things that make them to learn how to read. They read this news in front of others to gain confidence.” FGD-SHRP-Kiruhura “I was taught about four types of texts, whereby you have got narrative text, informative, poetic and then persuasive text. Under narrative text, a child narrates what he/she has seen and we have got the steps which we follow when writing a composition. Among the steps we do, one is about planning. What are you going to talk about? A child plans, thinks and talks about a given topic e.g. an old man in the village. For the first week, a child writes about planning, the 2nd week a child begins drafting the story, meaning that the child is getting words to use when writing his/her composition. When we come to the next step of revising, that is where the teacher comes in and SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 192 revises what the child has written. The child reads what she/he has written and then you correct the child after revising it. By doing this, the child’s mental ability is improved in writing, making a composition then making some simple sentences. Here I am focusing on Primary Four as a transitional class.” FGD-SHRP-Kiruhura Reading materials. Teachers were provided with textbooks/ reading materials, which contained the content that would help in Early Grade Reading. The reading materials contained words, letters and pictures for children to use in the class room to help them with reading and writing. Teachers reported that the reading and instructional materials were very relevant in instructing children in reading and learning. “The books are good in letter names and sounds to use while teaching. They are also good in helping teachers follow steps in teaching. We get letter names and sounds from the SHRP books but the vocabulary we get from the curriculum and other books like Fountain and MK. SHRP books are good for reading and writing.” FGD-SHRP-Kamuli “According to SHRP program, their main aim was based on children understanding reading their local language through Literacy One and Two. These are the areas teachers were trained to improve on the reading skills of children.” FGD-SHRP-Kiruhura Children’s health and wellbeing On the other hand, teachers were trained on how to handle different categories of children and in regard to their health, these children included those who are physically impaired (deaf and blind) and those who are HIV-positive (control and preventive measures). They were also taught skills on how to handle such children and how to teach them to read and write. During the FGD discussions, teachers did not comment on the usefulness of the health component was in contributing to Early Grade Reading and learning. “We participated in Early Grade Reading and school health. We leant about early reading among children in lower classes from P1-P3 in their local language and English. In health, we learnt about AIDS prevention and control.” FGD-SHRP-Kiruhura “We were trained by SHRP and this was in two ways: school health and [Early Grade reading. The health component tackled about AIDS - how to handle those pupils who have AIDS and teach its causes and prevention.” FGD-SHRP-Kiruhura 3.1.2 STRENGTH OF THE CONTENT TAUGHT Generally, the majority of the teachers recognized the relevancy and significance of the content after the training. Teachers felt that they were not fully equipped to apply the content in the classroom in order to empower children in Early Grade Reading and learning. There are, however, several aspects of the content and methods that teachers found helpful in guiding them in regard to pupils' reading and learning. Flexibility and easy pronunciation and use of letter names and sounds. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 193 After the training, most teachers felt that their pronunciation of letter names and the letter sounds had greatly improved. They noted that they used to pronounce the letter names and sounds poorly prior to the training. However, after the training, they noticed a positive change and were able to apply and pronounce the words and letters differently, which greatly improved the pupils’ learning. “I learnt to use Proper names of sounds of letters and pronunciation of words. Previously, I used to pronounce letters differently but after I was trained, I learnt to use proper sounds and taught my pupils correct syllables and illustrations.” FGD-SHRP-Kamuli Making of lesson plans has eased teachers’ application of the new approach by the use of textbooks Reading materials, especially the textbooks that were provided during the training, helped teachers develop lesson plans and schemes of work. In the past, they did not have books for guidance to prepare lessons and schemes of work. “SHRP books are good for reading and writing. The books are good in letter names and sounds to use while teaching. They are also good in helping teachers follow steps in teaching.” FGD￾SHRP-Kamuli “…the subjects that were trained by SHRP were Amakulu, Literacy One and Two. Makulu is news reading. Here, children take part in news reading, drawing and other things that make them to learn how to read. They read this news in front of others to gain confidence.” FGD-SHRP-Kiruhura 3.1.3. WEAKNESS IN THE CONTENT TAUGHT Teachers identified several weaknesses that hindered them in teaching the content to the children. This included missing content for Science and SST, as well as limited alignment to the government education curriculum. Missing content: Missing content was particularly identified in Literacy 1 (Science and Social Studies) as compared to the content described in the national primary school curriculum for Uganda. Teachers therefore resorted to obtaining additional content from the older thematic curriculum and other primary school textbooks like MK and Fountain books, to assist them in teaching. This was said to be time consuming and prevented them from completing the syllabus in the scheduled time. “I felt equipped but as I started implementing the work, I realized I am not fully equipped. This is because there is some missing content particularly in Literacy 1, which is composed of Science and SST. Sometimes we are forced to go back to the thematic curriculum and other science textbooks and get missing content to teach the children. The content itself does not cover all the necessary elements supposed to be covered. For example, during second term, we are starting with weather in SST and types of weather are missing in the pupil’s book for Primary One, on Page 46. In Science, characteristics of living things are missing. SHRP is limited in its content, and therefore we are forced to mix content from other sources so that our children can compete favorably with those in private schools.” FGD-SHRP-Kamuli SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 194 "When we ask the CCTs about the missing content, they just tell us to mix from other sources. This is too much work for us." FGD-SHRP-Pader Most teachers perceived SHRP reading materials as only good for teaching children how to read, not for educating children in class. Despite the fact that the SHRP books have the letter numbers and letter sounds, some teachers felt that these books are only good for children’s reading, since they lack the content for children to use when it comes to teaching in terms of literacy, that is, science and SST. Teachers further mentioned that the books that were provided to them to assist them in teaching have brief content, which does not articulate tenses and grammar. This will make the children miss out in learning grammar. “SHRP books are only good for reading and writing but the real content of Literacy 1 Science and SST is not there and if we stick to only the SHRP content, children will miss out a lot and we can lag behind, while private schools will be far ahead of us in teaching.” FGD-SHRP-Kamuli “The English books for P.1- P.3 are too brief. They are not well articulated in grammar (tenses) and do not explain the tenses. However, the books are good for reading and writing in English. But they are too brief that makes teachers take a lot of time explaining all the tenses. “You the teacher you take the trouble to consult other books like MK Literacy 1, to get all the missing content. If they write a verb in past tense, you have to look for the other correct content to teach the children because the tenses are not well explained. This makes children miss out a lot although they learn to read and write. I think the children are missing out part of the grammar, like the use of 'although' and changing sentences. They are very few like two to three.” .FGD-SHRP￾Kamuli “They have some missing content like in Literacy 1.They are too brief and not well articulated in grammar, especially English books for P.1- P.3. They are limited in vocabulary. The SHRP books are only good for reading and writing; not good for Literacy 1.They do not want us to use any other books, they want us to use their books only” FGD-SHRP-Kamuli Some teachers thought the training period was not long enough and that the content was not fully absorbed, which later on affected their implementation /application of the same content in the classroom. “I was not fully equipped because the training was not enough and the time was short. The reason why I need more time is because the RTI/SHRP content is too big and some of the SHRP content does not rhyme with the curriculum. That is why most parents tell us that we are teaching and implementing Lusoga yet for them they think that if a child speaks English it is the best thing teachers and the school should do.” FGD-SHRP-Kamuli There is some content missing. For example we have a theme like weather. You talk about the elements of weather we have sun, wind, rain and the clouds. But if you go to the SHRP books for Primary One, you do not find those elements there, you find different things completely, which makes children miss out some components. For instance, in the old thematic curriculum if a child did not understand something in English, the literacy teacher would teach the same things in SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 195 literacy that were taught in English, which is not the same thing with RTI content. I feel children need to learn and understand the elements of weather.” FGD-SHRP-Kamuli Content taught is too much and hard, particularly for Primary One pupils. There is limited time allocated for teaching content to Primary One children and yet the content is too much for them to grasp in a short time. Teachers explained that children in P1 need baby steps like knowing the environment they are in, rather than the reading materials that they are introduced to on the first day of school. This is simply because at that stage, they don’t know how to read and write. “The RTI English book is too broad. For the case of P.3, the book has two parts. Part 1 (oral and reading) and part 2 (writing) the content is too broad. Every teacher complains of not completing the work because it is too much. By the time the teacher teaches English part 2 (writing) children are already exhausted. It has a lot to be covered, like reading, pronunciation, structures, stories, word making and then editing. The time we have is not adequate enough to pass on all the content and knowledge to the children.” FGD-SHRP-Kamuli “It is difficult for term one in Primary One class because some pupils never attended nursery section and you have distributed the books and you start teaching them letter S or SSSS. The child doesn’t know the neighbor he/she is seated with and you give them a book. It is difficult for such a child to get something from that book or in teaching, since the time is limited. For me I think that for pupils in Primary One, there is no need of these textbooks. They should begin by learning class environment, know each other, know his or her teacher then from second term he/she can start with those textbooks. But immediately in first term on the first day the children are given textbooks with instructions. He/she doesn’t know anything at school and putting that book in front of him or her that he should start reading for you, I think that is not good for that child.” -SHRP-Kiruhura “If you have 120 children in class, only 20 can understand, write punctuation marks, putting in consideration the paragraphing and the rest will not understand, yet RTI told us that even those who do not know let them do it. The children who do not understand come from different backgrounds, especially those coming from private schools in P.1. They don’t know anything, but those who have come from P.1 and P.2 in our schools can understand something because they have gone through the SHRP programme.” FGD-SHRP-Kamuli Existing errors in the RTI books. Most teachers reported that there were a few errors in the textbooks, which should be corrected. The teachers have reported this to the CCTs but no appropriate action has been undertaken to revise the books. “There are some few errors in the books. For example, components of pests. The writer says a white ants is a pest and the subtheme is birds and insects. Are we really going to tell the children that white ants are pests? The time we have is not adequate enough to pass on all the content and knowledge to the children, and by the time we do all that, children are already tired.” FGD￾SHRP-Kamuli "But the challenge which is there, when you go to the books, you find that the 2nd week the books have letter names and letter sounds. Though they can see the pictures, under the pictures there SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 196 is the word abaana (children) yet the children are to be taught letter ‘A’ alone but there is the word abaana. Here the child will be challenged, since the child will not know …baa… and …na… all the child knows is the letter is the letter ‘a’ you told him/her about. There might be other words as abegi (pupils) and all a child knows is letter ‘a’ that you taught, and yet down there is ‘abegi nibashoma ekyitabo’ (learners are reading a book). You taught letter ‘a’, but with this confusion, some children feel like going back home. We should teach letters and sounds because words and sentences in first term is a great challenge. If it is a word, let it be a word and if a sentence, let it be a sentence; but not mixing letters and sounds. Even with that there is writing, which is a lot to be done in 30 minutes.” FGD-SHRP-Kiruhura Limited contextualization of reading materials. Teachers mentioned that the RTI reading materials have artificial examples and pictures that do not correspond to the surrounding and local environment where children live. “When I am teaching about things in a home like mat and saucepan, the content that you find in the book is contrary to the things that are in our homes. When you ask the supervisors, they tell you to go and teach what is in the books. So we end up teaching the right things to the children when the supervisors are not around because the content is not in the book. So we rather write on the blackboard about the actual things that are in our homes.” FGD-SHRP-Kiruhura Failure to explain the meaning of a word in the local language. Teachers are not allowed to explain words in the local language and, as a result, they leave the pupils unable to understand what the teachers are teaching them, given that some words are hard to explain in English. “When you are teaching English, you are not supposed to translate any word to Runyankole Rukiga. But for us when we are teaching English, we can use actions but there are words which have no action and so you leave that word like that and continue to the next word. When the supervisor is seated behind and you fail to explain that word or translate it, he/she will note it in his/her book. After writing such a comment in his/her book, he will go away without giving you any advice and that will tarnish your reputation as a teacher. The supervisor should have corrected me rather than noting the mistake down. You should also inform them that whenever they come to supervise us, they should advise us on where we are going wrong.” FGD-SHRP-Kiruhura “We have also got a challenge, when they are disseminating this information in workshops, it is done in English completely and the other workload is left to a teacher to interpret the content which was explained in English. We were taught in English; they don’t know the content in local language but they leave the whole load to the teacher to interpret it in local language. Even the teacher’s guide is in English. So the whole vocabulary is left for the teacher to interpret. For example, when we come to the thematic curriculum, there is where they say we should teach first aid and accidents, which is a lesson in P1, P2, P3, and P4. When I come to the transition class, the topic of accidents and first aid is taught in third term. So when it comes to those ones setting exams, they will set knowing that it was taught in P1, P2, and P3. So they will set it from Term One and yet it was taught in a local language. So how will a child know how to answer what an accident is and first aid, when they learnt them in a local language? What we have been telling our supervisors is that if it is literacy in local language, let it be taught as a subject but not all the SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 197 subjects to be taught in local language. If they were being taught in English, at least they come having some knowledge by the time they reach P4. This is also making some schools have poor performance. For example, they normally bring properties of air in P3. I looked for the word properties in Runyakole; it was not there. So how can I teach a word I can’t explain to the pupils in Runyakole?” FGD-SHRP-Kiruhura Failure to continuously supply reading materials to schools: Materials that were provided to teachers during training are worn out and torn. They also don’t correspond to the syllabus that other schools use to teach. “When our supervisors came, we told them to make flash cards for us. They asked us the challenges and we told them to make for us flash cards for each sub-theme and wall chats. However, they should also bring more books because the books that are there were given two years back and given that we have many children, you can find that all pages are missing and it starts from page 30; so I can’t know where to start from when teaching.” FGD-SHRP-Kiruhura “I heard that the books that they brought around were meant for research and later they were going to bring other more advanced learning materials but we are still using the same books that were left behind.” FGD-SHRP-Kiruhura 3.1.4 METHODS TAUGHT DURING THE TRAINING AND THEIR STRENGTH Teachers were asked what methods/techniques they were taught during the training, the weaknesses and strength of the methods and how they are being applied in class. All the teachers reported that they were taught in different methods that included: I do, we do, you do, making of lesson plans, use of different learning materials, listen say and use, play way method, observation method and play way method among others. Generally, the majority of the teachers recognized the relevancy and significance of the methods they were taught. Teachers felt that they were not fully equipped to apply these methods in class to empower children in early grade learning and reading. Word cloud on the methods taught during the SHRP training The word cloud below shows the methods that were taught during the SHRP program, the weakness and the strength that the teachers found when applying these methods in the class room. Children '111' were exposed to different methods '68' of teaching '63'. Apart from the content that was taught in the SHRP trainings, methods too were taught to enable children to know how to read and write. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 198 Figure 4: Word Cloud on the methods taught during the training 3.1.5 METHODS TAUGHT DURING THE TRAINING AND THEIR STRENGTH Making of lesson plans Teachers were taught the importance of making lesson plans. These have helped the teachers to plan and organize work for pupils, which helps them to learn. “I am able to make an outline of the weekly working plan. For instance; Day 1 is for planning, Day 2 is for drafting, Day 3 is for editing, Day 4 is for revision, Day 5 is for assessment. I also learnt that Day I planning works together with Day 3, used for editing; then Day 2 drafting works together with Day 4 that is used for revision. Day 5 is for assessing pupils by the teachers.” FGD￾SHRP-Kamuli I do, we do, you do. “I do, we do, you do” is yet another methodology that was taught in trainings, that helped the children to imitate their teachers as they teach and pronounce the words, hence enabling the children to learn by imitating their teachers’ actions and how they speak and pronounce letter words and sounds. This method is commonly used and it is applied every day in the classrooms. BOX 3: Teachers' voices on methods taught “I do, we, you do. The method is good and according to the attention span of children, it is the easiest method to use when teaching. Children become very active and participate. I apply this method daily when teaching children. Children learn from the teacher.” FGD-SHRP-Kamuli “I do, we do, you do. It is easy because children can ask and answer questions. The method is applied daily when delivering a message.” FGD-SHRP-Kamuli “In runyankole it is ‘Ninkola, Nitukola, Nokola’ that is okushoma nokuhandika; Literacy One and Two. This helps the learners to master things easily because you first read the word to them as Word Length Count Children 8 111 Method 6 68 Child 5 64 Teaching 8 63 Word 4 59 Learners 8 47 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 199 they are listening; they read with you then when they master, they read it alone. So this helps them in mastering themes easily.” FGD-SHRP-Kiruhura “In the past, there were many methods in teaching and they couldn’t be identified and as such we couldn’t realize which method we were using, so when I talk of ‘I do, we do, you do’, it is a good method that embeds all methods. This helped to make the children catch up with what the teachers were teaching them.” FGD-SHRP-Pader “These methods are good because they involve the children and control of children in class becomes easy. Learners are involved, especially English and Literacy are languages a teacher teaches and children repeat after the teacher.” FGD-SHRP-Kamuli Cooperation and group work discussion. Teachers were taught how to group children and make them cooperate and share the learning materials amongst themselves afterwards. Group work enabled slow learners to catch up with the quick learners. While forming groups, fast learners are mixed with the slow learners and this will help the latter to get a chance and pick points from the fast learners. BOX 4: Teachers' voices on cooperation and group work discussion “Cooperation is done through making my classroom environment useful and conducive to the learners by looking for enough instructional/learning materials for my learners. Now my learners are able to share those learning materials and I also encourage them to share the materials.” FGD-SHRP-Kiruhura “There is also group work discussion. In this, you put children in groups and give them a learning material to read, with different guiding questions in each group. So at the end, each group comes in front and discusses the stories in the learning material as guided by the questions they were assigned to answer.” FGD-SHRP-Kiruhura “In reading a local language, I used to give books in the local language e.g. the poem and story books, which they would read in groups and they also understand those things when you translate the poems or stories into songs. So they had to make friends, they cooperate when singing together, playing and working in groups.” FGD-SHRP-Kiruhura “Group work discussion helps slow learners to catch up because at times the learner who doesn’t get a point from the teacher finds it easy to get points from their fellow learners. So if put in a group, the slow learners will get a chance to pick points from the first learners.” FGD-SHRP￾Kamuli “A group discussion also helps the learners to be confident because some of the learners fear the teacher but when they are in the group, they express themselves.” FGD-SHRP-Kiruhura “In group discussions, learners cooperate with each other and since they cooperate, when one makes a point, others can help to build on that point since they are cooperative.” FGD-SHRP￾Pader Use of different learning materials SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 200 Teachers were taught to be creative and use the local environment to make learning materials for the pupils like balls, pots etc. When these local materials are used in teaching vocabulary. It can easily help the child to pronounce the name of the object, since the child is able to see the object she/he is speaking about. “I use the local environment to make materials like the balls, dolls, pots, ropes, mats, the counters, flash cards to mention but a few, which help the children to learn reading.” FGD-SHRP-Kiruhura “The learning situation became very easy because when you use local materials in teaching vocabulary, they read and at the same time see the picture and they know that this is a cup and it is read as this. So it makes them to easily understand.” FGD-SHRP-Kiruhura Listen say and use The listeners listen to the word the teacher is pronouncing, they repeat the word and later the children are able to use the same word in a sentence. “There is also another method called ‘Listen, say, and use’. for example, when you are teaching English vocabulary e.g. a word, like a bag, when it comes to the learners, they first listen as the teacher pronounces the word and after listening, you instruct them to say that word as learners. When they have mastered how to pronounce it, then you tell them how to use it, maybe in a sentence.” FGD-SHRP-Kiruhura Play way method Play time and ice breakers during a lesson are good because they help the child to remain active in class and stop dozing. “We have play way method; for instance when you are teaching and some pupils start dozing, you engage them in a play time/ice breaker and when they gain their strength, you continue with the lesson.” FGD-SHRP-Kiruhura Observation method. When children observe a picture, it helps them to grasp what they have observed, since it sticks in their minds. “Observation is used when using pictures in class. For example, when teaching a given word, we are encouraged to teach using three vocabularies each day. Let me say if you are teaching about things in our homes like pots, cups and chairs, you will have to draw the pictures on the flash cards and also have cards with words. So you encourage the learners to critically observe the pictures and read the words while observing what the words mean in the pictures. These pictures and words are also in the books we were given so each child can observe what you are teaching in the book. These books were given to each child. So you tell the child to observe the chair while you are pointing at it and also have to be pointing and touching that picture. So they observe the words and pictures on the blackboard, flash cards and the textbooks. So that is what they call observation” FGD-SHRP-Kiruhura SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 201 “Observation is good and promotes reading because what you see with your eyes becomes hard to get off your mind. So if your child sees something and at the end of the day he/she is able to say this is this, it will never get out of his/her mind. So it helps them to remember what was taught.” FGD-SHRP-Kiruhura Consideration of children who are physically impaired Teachers were taught to consider the learners who are physically challenged/ learners with special needs when they are teaching in the classroom. For example, the sitting arrangement should be done in such a way that children with special needs are not inconvenienced or disadvantaged. “We were also taught how to manage children with special needs, whereby we have to consider the sitting arrangement in classes. For example; you need not to put the taller child in the middle, shorter child behind considering their disabilities either eye defects or hearing impairment... Such things” FGD-SHRP-Kiruhura “We were also taught about children with special needs and how we should teach them given that some children are slow learners.” FGD-SHRP-Kiruhura Making work plans and lesson plans Making lesson plans helps teachers follow steps, which is good for a pupil to learn words systematically. “We were also taught how to make lessons and schemes of work. Here we would follow the steps e.g. let us say I am going to teach letters, you first teach them a letter and then support that letter with words and how those words rhyme in a sentence like letter “a” and from that you get abaana (children) and abantu (people). There you create a sentence like abaana nibazana (children are playing), abakazi batekyire (women have cooked) and other sentences, basing on the letter that is being taught.” FGD-SHRP-Kiruhura “There is what we call ‘phonemic awareness’ where by children were taught how to pronounce words correctly using letter words and sound. The child will then be able to pronounce the word correctly. Then after making those words, you make some few sentences like, ‘This is a cup.’ ‘The cup is on the table.’ ‘The mat is under the chair.’ After teaching them the words, you teach them how to use them in sentences. That is what I learnt in comprehension.” FGD-SHRP-Kiruhura Child assessment Assessment of learners helped in teaching the children by identifying their strengths and weaknesses. “Then another thing that I learnt about EGRA; they taught us how to assess our children. That assessment helped us very much. How do you assess? You assess them as you are teaching through giving them exercises. We also give them home work. When you are teaching, you assess five pupils a day. you are going to assess them in listening skills, in that you read a word then you ask, “What word have I said?” Those pupils who paid attention can tell you the word. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 202 Then you go to the assessment book and assess that person who has told you the correct word. Assessment helps much in teaching our pupils.” FGD-SHRP-Kiruhura Other methods of learning that the teachers were trained in and were useful in the Early Grade Reading and learning included: continuous assessment method; oral language method; multisensory method and scaffolding. 3.1.5 WEAKNESS OF THE METHODS TAUGHT Teachers from all districts indicated that they were able to apply some or most of these methods in classroom practice and teaching “from English to English”, because it was very easy to apply, owing to the fact that they were not very new. Some methods, though, were difficult to apply in English. Other teachers noted that there were problems in writing and reading when it came to pupils from P1 and a few difficulties were found in other classes. Methods require longer time Application of the methods consumes a lot of time while using the “I do, we do, you do” and “listen say and use” methodology, simply because the teacher first says a word, them the class repeats it and later every pupil says the same word. This takes longer time than the time budgeted for a single lesson. Teachers noted that they need much more time for this method to work effectively, which time they can’t get, since they are also aiming at completing the syllabus at the end of the month. “One of the challenges with that method (I do, we do, you do) is that it is time consuming because you need to say the same word with the learners also and at the end, learners have to say that word alone. So that takes time and you know our learning sessions take short periods.” FGD￾SHRP-Kamuli “A lesson period is 30 minutes and in this you have like five vocabularies and you need to teach them using the ‘I do, we do, you do’ method; so it takes time.” FGD-SHRP-Kamuli “One of the challenges in using ‘listen, say and use’ is that it also consumes time, because they have to first listen as you are saying the word, then you also listen as they are saying it, then constructing sentences using that word takes time.” FGD-SHRP-Kiruhura “Another thing is that it is tiresome. Because the period for the lesson is short, you have to fix all the items children have to learn in that short period and yet using it consumes time, so it is tiresome” FGD-SHRP-Kamuli Difficulties controlling and teaching large classes: Teaching large classes with many learners while applying these methods was found to be difficult at times, particularly when conducting group work. “Class control becomes a problem when it comes to that method because since it is a group with many pupils in one place, and you know, when you have a class of 50 pupils, you can’t put them in one group, there should be 5 groups of ten. So when you are handling this group, there is also another group there unattended so they start shouting” FGD-SHRP-Kamuli SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 203 Cross-referencing. Teachers noted that because of missing information in the RTI books, teachers were encouraged to conduct cross - referencing and yet this required a step by step process which teachers were not very familiar with “We were also taught about how we can cross-reference in the books, but when we tried to apply it, it became a problem for us in a way that you find that you are handling say the part of how to teach and then you are handling a part on transition story, you are also looking where there is at least the teacher’s guide, which is like two pages; then the pupil’s book which is like three pages. So you find you are using five pages within a lesson. The challenge we had is that how do we come about the content to be taught? Steps are many in a little time, given for a lesson that takes 40 minutes to cover a given content. When you are using cross-referencing, you find that you are using even an hour because we have got slow learners and being a transition class, it is difficult for a child. Sometimes when you tell a child look, what do you think, what type of text type is this; it takes time to understand and to interpret the information and then giving you the answer, so you find a challenge to read for each child and it takes more time. One of the challenges in using ‘listen, say and use’ is that it also consumes time, because they have to first listen as you are saying the word, then you also listen as they are saying that word, then constructing sentences using that word takes time.” FGD-SHRP-Kiruhura 3.1.6 TEACHERS’ APPLICATION OF THE NEW APPROACH Ability to apply the new approach: Most teachers felt that they were not fully equipped to apply the new methods/ approach and felt that they needed more training to effectively apply the new approach. Very few teachers mentioned that they could apply these methods comfortably. Teachers noted that the refresher course helped them to master and recap what was previously taught, which enabled children to learn faster. “I feel fully equipped in many areas which include: Articulation of words like letter names and letter sounds (words and sounds making the same meaning). I can now differentiate letters and sounds and blend letter names and letter sounds to form words that can be easy for children to read. For instance, the word (AND) and (INK), I form the onset like a-n-d and i-nk, and children are able to get the onset and the rhyme, which makes it easy for them to read a word. I am able to deliver the message easily to the children.” FGD-SHRP-Kamuli “ I was well equipped. They told us to teach in the local language and how to teach on different days; that is, how to teach Day One is the same as how to teach Day Three; how to teach Day Two is the same as how to teach Day Four. Then we read through their books, we ask them questions, whereby when you are teaching them Day One you introduce them to the letter and you make a sound of that letter. You first tell them the letter name like C and you tell them that the sound is CCC and that is Day One. Then from Day One they told us to see the picture from the books given; they asked us, ‘What are you seeing? What can you see?’ Then from there were words under that picture, including that letter then from there you read the words; you read the sentences by using those methods they taught us. When I came to my class I felt comfortable using them because of that refresher course.” FGD-SHRP-Kiruhura Inability to apply the new approach in the classroom: Most of the teachers felt that they were not well equipped in some areas and this affected pupils learning since it is very hard for a teacher to apply a method that they have less knowledge about. This also made teachers lag behind in completing the curriculum. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 204 “In some areas I felt I was equipped but in some areas I feel I need a little more time to be well equipped. I learnt to use proper names of sound which I did not know well” FGD-SHRP-Kamuli “Somehow I was equipped in some areas though not fully. Because of the missing content” FGD￾SHRP-Kamuli “I was not fully equipped. Because the content provided for day 1 to day 2 is not adequate enough which makes the teachers lag behind in curriculum delivery” FGD-SHRP-Kamuli During these trainings some teachers felt that they had mastered the skills, however when they reached in their classrooms they were not able to apply these methods effectively because these methods were not piloted to understand the gaps. “I felt equipped but along as I started implementing the work, I realized I am not fully equipped. This is because there is some missing content particularly in literacy 1 which is composed of science and SST.” Some time we are forced to go back to the thematic curriculum and other science text books and get missing content and we teach the children” The content itself does not cover all the necessary content supposed to be covered. For example during second term we are starting with weather in SST and types of weather are missing in pupil’s book for primary one in Pg 46. In science characteristics of living things are missing. ‘ SHRP is limited to only its content, so we mix for the good and sake of the children so that they can compete with those in private schools” FGD-SHRP-Kamuli “We were able to understand what we were taught during the training but when you reach in the classrooms, you find that the child is coming from the village and when you distribute the books you find the child is reading the book upside down. When you write a letter on the blackboard, the child will not be able to identify it in the book, since the book is upside down, hence a challenge. Most of these children are well conversant with pictures and when such a child is asked to explain the picture, it becomes easier for the child to tell you what the picture is about, even though he/she is not able to read the word. But with pictures a child will be able to say that I see a cow, I see teacher, etc.” FGD-SHRP-Kiruhura “I did not feel well equipped, because time was little to enable me to learn and be equipped. The first time we went for the training, we did not capture anything, not until we went for the second training in May. In fact it was a bit challenging. It needs ongoing training to be able to change parents’ attitude about the training. Most parents think we are being trained in the local language and they could take away their children from the government schools and take them to private schools, because they think we are implementing Lusoga”. FGD-SHRP-Kamuli “I took time to teach with confidence.” FGD-SHRP-Pader “I did struggle a bit because I had difficulties in learning areas and so if I used the same method, it would affect the outcome” FGD-SHRP-Pader 3.1.7 TEACHERS SUGGESTIONS TO IMPROVE TRAINING Word cloud on possible solution to improve the SHRP trainings Teachers suggested possible solutions to improving the content and the methods so as to comfortable apply them in the classrooms from the SHRP ‘3’ program. Teachers noted the need ‘6’ for more training ‘4’ to be fully equipped and able to apply these skills. They also believed that if the days ‘3’ of training ‘4’ are increased, this would help them to master the content ‘3’ and the new methods of teaching. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 205 Figure 5: Possible solutions to improve SHRP training Increase the frequency and length of the training courses The content taught during the SHRP trainings is much greater than the period of time allocated. Teachers suggested that the frequency and length of time need to be increased so as to enable the teachers fully understand the content and methods to apply in the classrooms effectively. “The other reason why I say we need more time is that the work we are given to teach is too much compared to the period we are given to train” we train for 3 days and the work your given is too much however much you try and concentrate you reach a point and despair” . Even when we went back for the second time still the three days indeed were not enough to master all the content and the methods we are supposed to teach. That means we need SHRP to give us more training so that we can continue with RTI because it is the best and easy way to teach children how to read and write.” FGD-SHRP-Kamuli “SHRP should increase the duration of the trainings to enable teachers to learn and master all the content. At least the training should last for 2 weeks not three days.” I feel we need more training to help us learn more because the 3 days are very few to deliver all the content SHRP has. The programme is good but the time is limited” FGD-SHRP-Pader “Somehow somewhere we were equipped but we need more training about those things because there some issues we never understood which we need to pass through again to be more conversant with. So if there can be more workshops/trainings, you are most welcome.” FGD￾SHRP-Kiruhura “The facilitators in EGRA should organize more workshops to help us improve on our skills and know how to use those methods” FGD-SHRP-Kiruhura 3.2 TEACHER SUPPORT SUPERVISION The section focused on the role and usefulness of the support supervision received by the teachers towards applying SHRP teaching methods and instructional materials in the classroom. Word Length Count Need 4 6 Training 8 4 Content 7 3 Days 4 3 SHRP 4 3 Teach 5 3 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 206 Teachers were asked if they had received any support supervision, the frequency of the visits, weaknesses and strengths of the support supervision and if the support enhanced their teaching techniques in class. Word Cloud Analysis. After data coding and analysis utilizing NVIVO11 software, word clouds and word frequency tables are developed to ascertain which words were highly used by the teachers when responding to questions about teacher support supervision. Figure 6: Support supervision word cloud and the table of word frequency The figure above shows that the word ‘Teacher’ with the word frequency of 55 times, was the most used during the FGDs. This word was used when identifying people who most frequently conduct support supervision. Teachers identified Head teachers and the Deputy Head teachers, with a word frequency of 44 times, as the persons who mostly provided support supervision to teachers. The overall analysis shows that Head teachers and the Deputy Head teachers visited their teachers in class at least two times and provided support like skills in lesson planning, and preparing schemes of work, among others. The word ‘Teachers’ followed, with the word count of 46. This word was used when explaining their peer-to-peer supervision within the school, which teachers liked most because they noted the process was very friendly to them and not fault-finding. The other most used word was ‘Class’ with the word count of 37. This word was mainly used when teachers were explaining what was occurring in class when the supervisors visited them.. Teachers were also noting that some of the external supervisors were not visiting their classes, or that even when they visited, they arrived when the children had already left school for home and there was, therefore, limited or no interaction between the supervisors, teachers and the pupils. The word ‘Support’ with the word count of 37, was used when identifying and explaining the kind/nature of support they received from both the internal and external supervisors and finally ‘Supervision’ with the word count of 33, was used when the teachers were explaining the frequency of visits they received from the support supervisors and what they wished should have been the frequency of the visits per term or per year. Word Length Count teacher 7 55 teachers 8 46 head 4 44 class 5 37 support 7 37 supervision 11 33 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 207 3.2.1 FREQUENCY OF TEACHER SUPERVISION AND CATEGORY OF SUPPORT SUPERVISORS All teachers reported receiving some form of support supervision by different supervisors, whom they categorized as internal and external. The internal supervisors included Head Teachers, Deputy Head teachers, director of studies and the fellow SHRP trained teachers (Peer-to-peer supervision). The external supervisors included District Education Officers, District Inspectors of Schools, Coordinating Centre Tutors (CCTs), SHRP staff and VSO (Voluntary Services Overseas) staff. “I received support supervision from the CCT, Head teacher, Deputy Head teacher and then the RTI staff.” FGDs-SHRP-Kamuli “We always get supervision from the head teachers, from the coordinating centre tutors then plus the VSO even those from SHRP, they come and assess the learners, and also assess us and the inspectors of school.” FGDs-SHRP-Kiruhura “It was from the Deputy Head teacher or the Head teacher, and sometimes from the director of studies, secretary for education and CDO.” FGDs-SHRP-Pader While all the three districts sampled mentioned the above internal and external supervisors, Pader district mentioned parents as unique external support supervisors who appeared in their schools to ascertain the progress of their children’s ability to read. “Parents could occasionally come in the classroom and see what was taking place.” FGDs￾SHRP-Pader The majority of the teachers reported that they had received support supervision from internal supervisors at least twice a term and from external supervisors once a year. However, there were divergent responses regarding the frequency of the visits conducted by the supervisors to the schools in the same district. It was found that some schools in the same district had received slightly fewer visits than others, with the exception of Pader district, where supervisors had visited all the schools equally. BOX 6: Teachers voices in regard to frequency and category of support supervisors “It was three times for peer supervision and once for the FA in a term. The CCT also appeared once while the DIS and the DEO never appeared in a term but I think once in 2016” FGDs￾SHRP-Pader “We had three times with the peers and as for parents, it was twice, and not all parents turned up”. FGDs-SHRP-Pader “CDO, DIS, CCT and FA came once in the year while Head teacher and peers was three times in a term.” FGDs-SHRP-Pader “For the case of a head teacher, he/she supervises us very many times but when it comes to those other officials from the district, RTI staff and CCTs, for sure, a term can elapse when we SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 208 have not even seen them. Sometimes they come once or even never appear. So it’s either once or never.” FGDs-SHRP-Kiruhura “RTI has come twice a year in Nagwenyi Primary school. They come to check on the children in class. They ask the children some questions and give them some small gifts to motivate them to work hard.” FGDs-SHRP-Kamuli “For us they have been coming. Sometimes twice in a term, that is the RTIs and people from the district but the head teacher, she even always marks our lesson plans and supervises us like two times in a term” FGDs-SHRP-Kiruhura Teachers attributed the difference in frequency of supervision and limited number of support visits to a lack of financial support provided to government-supported external support supervisors, a large number of schools in the district, and the small number of the supervision teams compared to the large number of schools they were responsible for covering. “We think that these people lack funding and when it comes to those district officials they have very many schools in the district that need to be supervised. So I think because the schools are very many, they can’t finish supervising all of them.” FGDs-SHRP-Kiruhura Based on the above analysis, overall, teachers were dissatisfied with the frequency of visits conducted by the support supervisors and the limited amount of time they spent with the teachers during the visits. The teachers noted that the number of times they are visited in a term or year and the hours the supervisors gave them during the visits could not help them improve their skills. “The support supervisors give us little time; they come at 2:00PM and leave by 3:00. They don’t give us enough time to help and guide us.” FGDs-SHRP-Kiruhura “Primary one pupils go home by lunch time so when these supervisors come after 1:00PM they guide you alone without seeing the pupils. Involving these children would help them not to fear people who comes around to visit us.” FGDs-SHRP-Kamuli. 3.2.2 NATURE OF SUPPORT SUPERVISION PROVIDED TO THE TEACHERS The study team wanted to find out what kind of support the teachers received from their supervisors and whether this support enhanced their teaching techniques in class. According to the discussions, each teacher had received some nature of support supervisions from either the internal or external supervisors. These included: assessment and motivation of teachers and pupils, provision of teaching and learning materials, guidance on how to use the methods of teaching, class control and how to handle children of different specialties, lesson planning, and preparing schemes of work. The teachers' responses as to whether this support positively enhanced their teaching techniques in class were mixed given the limited interaction between the teachers and the supervisors. Lesson planning and preparing work plans The teachers reported being assisted by the internal supervisors (Head teachers and external supervisors, namely CCTs) in order to improve their skills in lesson planning and preparing SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 209 schemes of work, which contributed to their ability to pass on content to their pupils in class. However, this type of activity was mainly done by the Head teachers as compared to other supervisors, like CCTs. “The support I got from my head teacher…she advised me on how to plan lesson content because it is not easy in most cases to do lesson planning and scheming. Lesson planning and scheming, actually to integrate the content since it is a new approach is becoming hard for teachers but she helped on how to always make a lesson plan and how to follow all the steps. When you follow the steps you find that all the content is taught. So you make the steps and summarise all the content which you are to teach such that the content we have can be covered in a stipulated period.” FGDs-SHRP-Kiruhura “The CCT guided me on lesson arrangement. The CCT comes in class and guides how best to arrange lessons and encourages us on our strong areas.” FGDs-SHRP-Kamuli “We get support from the head teachers, through marking scheme and lesson plans and checking the pupils’ exercise books.” FGDs-SHRP-Kiruhura Teaching and learning materials The teachers reported receiving support through the provision of teaching and learning materials and some instructions on how to utilize them. This type of support was, however, not enough and not consistent. “The Head teacher provides teaching and learning aids to the teachers. The Head teacher buys manilas, markers and text books to help the teacher do the work well. My head teacher is very supportive.” FGDs-SHRP-Kamuli “They taught us how to make instructional materials. When you come in my class, there are so many things. They taught us that when you are doing Mathematics, then look for counters like stones, bottle tops and strolls. Then when in English you get a manila, wrap it like a ball and on that day draw there different pictures like a chair, a ball or anything you feel like teaching about. So you get children to gather and throw the ball to a child and ask, ‘What can you see on that ball’ a chair, you throw it to another child and you ask them, ‘What can you see on the ball?’ a book. Those are the things in our classroom. You can also use real objects, you can make a ball, a chair in sticks for learning and this was all from this support.” FGDs-SHRP-Kiruhura Guidance on how to use the new methods of teaching A limited number of teachers reported receiving some refresher courses on how to apply new methods in order to achieve the results desired by SHRP. Teachers were advised to double-check their work as they teach to ensure that they are following the right steps in passing on the content to the children in class. This enabled the teachers to be aware on what they were teaching and to ensure that they delivered the right content using the right approach. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 210 “They also guide us on the methods of teaching and keep checking whether you are following the steps of teaching by checking our lesson plans and scheme of work and the guide in how to follow the steps in teaching. They also do continuous assessment.” FGDs-SHRP-Kiruhura “The Field Assistants from SHRP too helped us on the new ortography and how to go about it in teaching because some of us were having difficulties in teaching.” FGDs-SHRP-Pader “You know when you are handling a lesson; there is the introduction and other sessions. So if you go astray, the support supervisors came in and say ‘No please you have to do it this way'. So they help us not to go off track when teaching.” FGDs-SHRP-Kiruhura A few teachers reported receiving special tips on how to handle children with different specialties in class to allow them equally benefit from learning and reading. The teacher-pupil eye contact training enabled teachers to better deliver content to even the impaired children in class. This was particularly noted in a few schools in the Kiruhura district. So the CCT advises you according to the strong points that you have and also to the weak points that you have. If a lesson is supposed to take 40 minutes, if he sees that you are taking much time in making children to clap hands, he tells you to minimize clapping hands... that is one.… If you are reading while looking at the book and not facing the pupils, some pupils start doing other things like talking. So he tells you to read the book and the content as you are facing the children…Some children who are impaired may have got the hearing impairment, they might not hear you when you read facing the book but when you read facing them, they can hear when they see the way you utter out the words from the mouth. Sometimes the deaf need to be spoken to when you are looking at them… They also advise when you are writing on the chalkboard to write big letters to help those who have got an eye impairment to be able to view what is written.” FGDs-SHRP-Kiruhura Few teachers reported receiving support supervision in the form of video lessons, where they were reminded of how to pronounce the letter they learned from the training through a video. “They also help us with video lessons, there is a time those CCTs brought video lessons which have the letters and sounds of those letters to remind us on how to pronounce these letters in class in the way we were taught in the workshops.” FGDs-SHRP-Kiruhura Class control techniques A limited number of teachers, mainly from Kiruhura district, reported receiving class control lessons from their support supervisors. This was mainly given to teachers with large numbers of pupils in class. The teachers were reminded to always start classes with ice breakers and to enable children to relax when they appreared tired. “I got support supervision from the CCT and they advised me about class control. That is when children are making a lot of noise and so if the lesson is starting they should sing a song to make them get relaxed.” FGDs-SHRP-Kiruhura Community sensitization about the relevance of SHRP SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 211 Teachers in the Pader district reported one of the most unique supports received from the supervisors was sensitizing parents about the relevance of SHRP, which in turn improved children's class attendance. The supervisors reportedly helped change the parents’ negative attitudes about SHRP and therefore motivated parents to send their children to school. “The FAs and DOS could meet with parents and carry out advocacy on literacy because most parents were saying that the new teaching in the local language was taking their children back. The parents wanted their children to be taught in English language instead of the local language because they had undergone the same instruction while in nursery.” FGDs-SHRP-Pader “Field Assistant from RTI supported us especially with the parent since they complain about teaching their children in local language so he explained the essence of the local language before introducing the English language.” FGDs-SHRP-Pader 3.2.3 STRENGTH OF SUPPORT SUPERVISION AND HOW IT ENHANCED TEACHING This evaluation sought to find out what aspects of the support supervision the teachers liked and how these enhanced their capacity to deliver lessons on learning and reading. Some of the aspects teachers reported enjoying about support supervision include: teacher-pupil motivation, confidence building, skills enhancement, constant supervision (which eliminated laziness), friendly guidance visa-vis fault finding, and extra on job trainings. Teacher pupil motivation Majority of the teachers from Kamuli and Pader districts reported improved teacher and pupil motivation as one of the aspects they liked about the supervision. It was noted that support supervisors motivated teachers to love their work using simple words like “thank you”. This type of motivation was mainly reported in the Kamuli district and was said to have enhanced the teachers’ ability to deliver the content to children in class, since such words of appreciation made teachers feel honored for the work they had done. “In some areas, the CCT thanked us and encouraged us to work hard. After seeing you in the lesson, the CCT calls you with other RTI teachers and we sit and discuss regardless of class and guides on what we should do as teachers. The CCT insists on what the teacher should do especially following the steps while teaching.” FGD-SHRP-Kamuli “Some of the support supervisors also do encourage the teachers in their work by sending reminder messages about the application of SHRP…RTI also sends some messages on phone SMS encouraging us to work hard and use the syllabus.” FGD-SHRP-Kamuli Besides motivating teachers, what the teachers, especially in Pader district, liked most about support supervision was that it was not fault-finding but conducted in a friendly manner.. For example, during some of the support supervision sessions, the teachers felt they were dealing with peers and that it was on-job training, such that when one made mistakes, their peers were able to help them make corrections. This further equipped them for the next session. Kamuli district teachers also reported pupil motivation as one of the aspects they liked from the supervision. Motivating pupils made it easier to pass on content in class, since the SHRP staff SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 212 awarding gifts to the children not only motivated the children to read, but also helped keep children in school and reduce the rate of pupil's absenteeism. BOX 7: Teachers voices on the positive aspects of support supervision “Support supervision, is better it’s not like in the past where there were inspectors of schools, in support supervision, they come to school to share and give guidance not fault finding, in most cases, it is the teachers who give them the problem you may be facing and after sharing you discuss the action points for the next activity and decide and agree that from today we have seen a problem here and there and this is what you could do. It is a friendly sharing because you sit with the person and agree.” FGD-SHRP-Pader “SHRP staff assesses the learners. They come in class and assess the learners and motivate them with gifts like pencils to motivate them do well in class.” FGD-SHRP-Kamuli “RTI staff comes in class and assess the children…One RTI staff came in my class and identified four potential pupils and took them out of class. She sampled and tested the pupils with the writing and reading materials she had come with. She gave them some small gifts like pencils then later told them to go back to class and told me to continue with the teaching.” FGD￾SHRP-Kamuli Confidence building among the teachers The majority of teachers mentioned that support supervision enhanced their confidence delivering content to pupils. This particularly the case during peer-to-peer supervision and other constructive advice provided by other supervisors. Teachers noted that when supervisors advised them on how to apply the new methods of teaching and or corrected them, it helped them gain confidence in how to deliver content to children in class. BOX 8: Teachers’ voices about confidence building “We have liked those supervisions because it builds in us confidence since they don’t come and shout at us. When they come, they advise us on how to use some methods and skills so this gives us confidence and hope on how to make this job better in future” FGDs-SHRP-Kiruhura “Peer supervision helped us to raise our skills and confidence when going to class because at least I knew that I have what it takes to conduct reading lessons for my pupils.” FGDs-SHRP￾Pader “My head teacher told me that I need to keep my lesson plans and learning materials organized so that whenever there is inspection, I am confident and not timid.” FGDs-SHRP-Kamuli “It also helps me to gain that confidence because I know the person coming is also a human being like me so he/she comes to help me and also to tell him/her, what is good and what is bad. So we cooperate and it reduces fear as in this is a big man from the district and so on, but I know that he/she is coming to guide and also help me correct my mistakes.” FGDs-SHRP￾Kiruhura SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 213 “We have liked those supervisions because they bring confidence since they don’t come and shout at us. When they come, they advise us on how to use some methods and skills so this gives us confidence and hope on how to make this job better in future.” FGDs-SHRP-Kamuli Skills enhancement A limited number of teachers reported having enhanced their skills in content delivery to children as a result of the supervision support. Some skills that teachers reported improved their teaching capacity included teachers' handwriting and class control skills, as well as addressing children with different specialties. BOX 9: Teachers’ voices about their skills enhancement “I got support from VSO. They came to class and found out that my handwriting in Primary One class was not good. What he did he put me in a workshop and he made chalkboard lines, where I learn how to write well such that when I went back to my school I drew those lines on the blackboard and my handwriting improved and became beautiful.” FGDs-SHRP-Kiruhura “It did help a lot because in the first instance we had the first training for only five days and this was not sufficient enough to complete and understand everything. So the support supervision, especially from the peers, enabled us to have extra on-job training.” FGDs-SHRP-Pader “They also help us with video lessons. There is a time those CCTs brought video lessons which have the letters and sounds of those letters to remind us how to pronounce these letters in class in the way we were taught in the workshops.” FGDs-SHRP-Kiruhura “Me the support I got from my head teacher; one she taught me some more skills on the content to deliver to the pupils, on handling children, handling the special needs and so forth. She also helped me on how I can help the slow learners at least to learn something and that is by regrouping and then grouping the quick learners with the slow learners because the quick learners help the slow learners. Sometimes there is where you find the slow learner wants to ask a question but he/she fears a teacher and has got a quick learner who is beside him/her, then that quick learner helps the slow learner.” FGDs-SHRP-Kiruhura The teachers also mentioned that they liked teacher-learner assessments and the provision of learning and teaching materials, since this helped them to correct their mistakes and be well equipped for class with all the teaching materials they needed, leading to an, improvement in content delivery. “Assessment has helped me as a teacher to improve on the way I prepare and deliver my lessons to the pupils. In most cases I have been able to identify my areas of weakness in teaching…One SHRP staff came to my class with a story and asked my pupils to read the story but my pupils could not read well the story to her expectation, I felt bad and I had to put in a lot more effort in doing my work to teach my pupils how to read.” FGDs-SHRP-Pader “In our school we are able to provide learners with books and the picture cards. It has also helped us as teachers to prepare well our lessons.” FGDs-SHRP-Pader SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 214 Constant supervision motivated teachers Findings indicate that support supervision contributed to less “laziness” among the teachers. External supervisor support caused teachers to be more alert and available, compared to being observed by the internal supervisors whom the teachers were already familiar with. “I will talk about laziness. Sometimes you may feel reluctant to do things at the right time like drafting a lesson plan, but when you know that you are going to be supervised, you put in that extra effort to ensure that you have everything ready before supervision. Therefore it helps me to be active all the time.” FGDs-SHRP-Kiruhura “There was a lax in teaching because there was little or no supervision from an external source since teachers are used to the internal supervision and besides, the CCT is rarely available in schools. So some teachers got familiar to each yet familiarity breeds contempt!” FGDs-SHRP￾Pader 3.3.4 WEAKNESSES OF SUPPORT SUPERVISION Despite the strengths of support supervision, teachers also reported several aspects they did not like about the support supervision. These weaknesses included limited frequency of the support visits, unprofessionalism by the supervisors, humiliation, and limited motivation by the supervisors. Limited frequency of support supervision: As earlier noted, all the interviewed teachers reported that they were not pleased with the number of times they were visited in a term or year, particularly by the external support supervisors. Even when they visited, they were given limited or even no time to adequately discuss and respond to the issues raised by the supervisors. Teachers reported that the external supervisors visited them either once or never in a term or year and, if at all they visited them at all, they interfaced with them for one hour on average. Some teachers noted that sometimes when the SHRP Field Assistants came to their schools, they remained in the head teachers’ offices instead of attending to their issues as teachers. This kind of support supervision made teachers relax and gave limited time to pupils, hence affecting their delivery in class. “For the case of a head teacher, he/she supervises us very many times but when it comes to those other officials from the district, RTIs and CCTs, for sure, a term can elapse when we have not even seen them, although sometimes they come.” FGDs-SHRP-Kiruhura “The FA never stepped in my class and I think almost all classes because whenever he could visit the school on supervision duties, he stopped in only the HMs office. So if there was supervision as promised, I never got any from him.” FGDs-SHRP-Pader “The support supervisors give us little time; they come at 2:00PM and leave by 3:00. They don’t give us enough time to help and guide us.” FGDs-SHRP-Kiruhura Inadequate teacher motivation was reported by majority of the respondents as one of the leading factors that affected teacher’s implementation of the SHRP project, particularly in Kamuli district. Teachers noted that when the support supervisors came to visit, children were motivated SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 215 through awarding gifts, while the teachers were not provided with any incentives. This demoralized teachers from implementing the program. Some of the non-monetary incentives that the teachers mentioned in recognition of their efforts included provision of certificates of attendance after the SHRP course. “We were trained but we have never been given anything…They just provide pupils with some incentives like gifts but none to the teachers.” FGDs-SHRP-Kamuli Inadequate support by the support supervisors. A lack of class teaching and learning materials and a failure to provide adequate solutions to the problems presented by the teachers were mentioned as common problems among most teachers. For example, some schools did not provide class teaching and learning materials and supervisors were not well equipped to provide solutions to particular challenges, such as translating words from one learning aid to another. BOX 10: Teachers’ voices about the satisfaction support supervision “We would wish that if a supervisor comes, and you tell him/her that I have got a problem, he tells you the solution, but all he tells you is to research on that word or to ask learners the meaning of the word.” FGDs-SHRP-Kiruhura “Some head teachers do not want to buy teaching materials for the teachers and they have a negative attitude and some are just not interested…I used to buy manilas for my class using my money. Each time I went to the head teacher to ask for money to buy manilas, he could say he doesn’t budget for such”. FGDs-SHRP-Kamuli “Sometimes when they come around to supervise and you inform them about the challenges you are encountering, they promise to address them but you wait and wait for their return all in vain.” FGDs-SHRP-Kiruhura “RTI has not supported the teachers so much. They simply just come to see whether the children are performing and reading but they don’t come to support the teachers. They just come to see whether the teachers are implementing what they taught us.” FGDs-SHRP-Kamuli Findings also indicate that a few teachers were not pleased with the kind of humiliation they suffered from the supervisors. Some supervisors reportedly used rude methods to pass on information to the teachers. Teachers reported that a few supervisors used dehumanizing and inappropriate words to pass on information to the teachers when they made mistakes, and some supervisors just noted their mistakes in a book and walked away without discussing them with the teachers. This approach of interacting with the teachers was perceived to be dehumanizing and kept teachers in a tense state, which affected their confidence in class. “A teacher is not supposed to be harassed, but sometimes the support supervisor comes in such a way that demoralizes the teacher. Sometimes you find when a teacher has no lesson plan or scheme of work; the supervisors use some words which are not ethical instead of advising the teacher. Harassing a teacher with words like ‘you are not supposed to be a teacher’….. Such things are not ethical.” FGDs-SHRP-Kiruhura SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 216 “When you are teaching English, you are not supposed to translate any word to Runyankole Rukiga. But for us when we are teaching English, we can use actions but there are words which have no action and so you leave that word unexplained and you continue to the next word. When the supervisor is seated behind and you fail to explain that word or translate that word, he/she will note it in his/her book. After writing such a comment in his/her book, he will go away without giving you any advice and that will tarnish your reputation as a teacher. The supervisor should have corrected me rather than noting the mistake down. You should also inform them that whenever they come to supervise us they should advise us on where we are going wrong.” FGDs-SHRP-Kiruhura Teachers reported that some supervisors, particularly head teachers, were not trained in the SHRP methodology and could not adequately provide supervisory support from an informed position. Teachers mentioned that because a few head teachers lacked knowledge about the SHRP teaching methodology, they often overloaded teachers with over eight lessons per day. “The Head teacher and the Deputies were not trained in SHRP so they did not know what they were actually supervising. SHRP only trained P1-P3 teachers and the head teachers were not included and so they were supervising some of us from a point of ignorance” FGDs-SHRP￾Pader “Some head teachers required us to deliver a certain number of lessons per day (eight) which was not practically possible given the class setting, the number of pupils and the timing at play at any one moment. For example, I was being put under pressure to make eight lessons yet I was alone with a class size of 82 pupils, I have to teach, give lessons and mark books” FGDs￾SHRP-Pader 3.2.5 TEACHERS SUGGESTIONS TO IMPROVE SUPPORT SUPERVISION During the FGDs, teachers were asked to suggest the possible ways of improving teacher support supervision. The key suggestions included improving teacher motivation through non-monetary incentives, adequate support in terms of content, and learning materials Teachers suggested that SHRP officials should provide adequate teacher motivation by providing them with incentives like certificates of attendance. “ The SHRP team should motivate teachers by providing some small incentives just like they do for the pupils…Children are so tiring, VHTs are provided with bicycles who are just half backed but they are thought about, but us who are fully backed we have nothing”. FGDs-SHRP-Kamuli Improved frequency of support visits by the support supervisors was a key factor suggested by all the teachers to improve the school reading and learning project. The teachers suggested that external supervisors like the CCTs and the district inspectors of schools among others should visit up to 10 times a term to improve the teachers’ level of commitment and competence. Teachers suggested that the supervisors should spend at least 4 hours with the teachers to enable them to attain the required support to adequately deliver in class. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 217 “They should give us more time when they come for supervision, talk to us and guide us as well. They should spend with us at least four hours. They should also come to visit the school when both the teachers and the pupils are still around. For example for these pupils who break off at 1:00PM, supervisors should come as early as 9:00AM to have a talk with them up to their time of leaving for home” FGDs-SHRP-Kiruhura “They (supervision) should be carried out frequently using properly programmed timelines. Internal supervision could be three times a term while external could be once in term. Because it is effective when I observe as a peer, we agree to supervise each other as peers and this has an impact on improvement. Besides, the CCT and DIS, regardless of the numbers of schools in the District need to improve on their frequency of their visit in the classrooms and not in the HMs office” FGDs-SHRP-Pader They should visit us as many times as possible in a term for at least ten times because they do come like twice, once or none in a term” FGDs-SHRP-Kiruhura Provision of teaching and learning materials. The SHRP project should reprint and supply more teaching and learning materials, since the existing text books are missing pages due to the large number of children in the classrooms. Besides, these materials should also be printed in an appropriate local language used in the district. The head teachers should also provide enough books for lesson planning and scheming, while the supervisors should help in translating words from one learning area to another. BOX 11: Teachers’ voices on improving support supervision "RTI should bring more books because the books that are there were given two years back and given that we have many children, you can find that all pages are missing and it starts from page 30 and so I can’t know where to start teaching from.” FGDs-SHRP-Pader “RTI should translate teacher’s guides from English to Runyakole because it is hard to interpret.” FGDs-SHRP-Kiruhura “They should give us enough instruction materials like textbooks and charts. RTI should encourage the district stakeholders and head teachers to buy enough seats for the pupils. RTI should also print more textbooks for the pupils because there has been this basic of ‘I can read and write’; therefore these pupils in Primary One must be more targeted to enable them improve on their skills to make them experts by the time they are in upper classes like Primary Four.” FGDs-SHRP-Kiruhura “The head teachers should give us enough books to do lesson planning and scheming so as to enable us be prepared fully” FGDs-SHRP-Kiruhura “They should come during teaching hours for them to attend the teaching sessions to assess and guide us on the effectiveness of the techniques we are using” FGDs-SHRP-Kamuli Teachers also suggested introducing and facilitating team supervision (exchange learning and supervision visits between schools) as one of the ways to improve support supervision. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 218 Teachers who are excelling better than others should be identified and funded by RTI to move to different schools so that others learn from them. “The CCTs should support, organize and encourage school exchange learning visits…We should have different facilitators in different areas like teachers who do better than others to move around different schools facilitating, and SHRP should facilitate the process after the selection.” FGDs-SHRP-Kamuli “There is need to do team supervision so that corrections can be done together. This can be carried out effectively by the peers and moderated by either the DOS or Head Teacher at school level because the DIS and DEO will talk of insufficient budget.” FGDs-SHRP-Pader Organize trainings for the supervisors. Teachers also suggested that all the persons given the roles of supervision should be trained on the methods, content and leadership skills to enable them execute their duties in a professional way. “The head teachers and the Deputy Head teachers need to be taken for refresher training (preferably by the FA) to help them familiarize with the tools, Through this, they will become better supervisors from the knowledge point of view. In this way, they will be able to supervise the teachers properly and the teachers will respect them because they at least know what they are doing” FGDs-SHRP-Pader. 3.3 ABSENTEEISM This section focuses on identifying the major challenges faced by teachers and families resulting into absenteeism of teachers and pupils. To obtain this information, the teachers were asked to explain the reasons and consequences of absenteeism and the role it plays in the effectiveness of teaching techniques in the classrooms. 3.3.1 REASONS FOR TEACHER ABSENTEEISM All the teachers reported that absenteeism was prevalent among teachers. This affected the performance of the reading and learning program, since the success of the project relied on the teachers’ presence at school. The leading causes of absenteeism included: sicknesses, long distances to schools, changes in weather, lack of motivation, delayed and low salaries and personal/family responsibilities. Long distances between teachers’ homes and school was one of the most common reasons teachers reported as a contributing factor to their absenteeism from class/school. The majority of the teachers are not provided with accommodations at school, since there are no residential houses on the school campuses for the teachers. Teachers have no means to travel to school and they often feel too exhausted to move to school on a daily basis. “Absenteeism is also attributed to recruiting teachers from far, considering the distance sometimes a teacher has to cover from home to school.” FGDs-SHRP-Kiruhura “Some of us come from distant places and sometimes I fail to come to school because I lack transport.” FGDs-SHRP-Kamuli SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 219 “Some of the teachers reside in their own homes, so along their way to school in the morning they may be involved in an accident or lack transport to move to school.” FGDs-SHRP-Kiruhura Poor weather coupled with poor access roads to schools was mentioned as a leading factor for teacher absenteeism, most especially during the rainy season in Kamuli and Kiruhura districts. It was noted that when it rains, teachers who stay outside the school cannot cross to school and even if they wish to, the roads leading to their schools are impassable. Therefore, if it rains in the morning, teachers have to wait for the roads to dry up, which takes a teacher even up to midday to reach school or not reach school at all. “The unpredictable weather sometimes affects teachers coming to school. This is very common during the rainy season and since some teachers come from distant places, it affects them from attending duty It is sometimes also attributed to bad nature of the roads to the school; when it rains, they cannot access school…Sometimes when it rains very early in the morning up to like 10:00 am or midday, I cannot go to school to teach.” FGDs-SHRP-Kamuli “When you look at our school, if it rains, you can’t reach there and if you are to go you can go at around 10am and it is very normal because the school is known within the district because the road is not good.” FGDs-SHRP-Kamuli Lack of motivation was mentioned as another cause for teacher absenteeism . The teachers noted that they are not recognized for their contribution in learning and yet their counterparts in P7 and the head teachers are monetarily recognized for "just polishing" children and yet the lower class teachers who fight to ensure that children learn how to read and write are ignored. This is made worse by schools which do not provide breakfast and lunch for teachers. The teachers therefore forego some lessons and resort to additional activities to get a means of survival. “We lack motivation because in most primary schools, even if a teacher does something good, they are never motivated. For example, instead of recognizing the lower class teachers who have laboured to lay the foundation, they recognize the P7 teachers monetarily who have just done the polishing bit..again in some schools, head teachers who have done little or nothing in an excellent activity are the ones that are recognized instead of the teachers who have done the ‘donkey work’. So teachers too need to be motivated, appreciated openly for the efforts they put in.” FGDs￾SHRP-Pader “Sometimes, some schools do not provide breakfast and lunch to their staff. As a teacher, if you do not have side income then you will have to get where you will get supper so the teacher will look for other means of surviving. For example, getting a bunch of bananas around is a tug of war because it is a long distance. So you will say that ‘in the afternoon I will go and look for matooke’. So you end up missing the lessons that you would have taught after lunch.” FGDs-SHRP-Kiruhura Attending to sickness and death were cited by majority of teachers as one of the causes of absenteeism by the teachers. Sickness occurs among the teachers themselves, children, spouses or relatives during which the teacher has to take some time off to care for the patient. Also when death of a family member or relative occurs, the teachers go to the burial as demanded SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 220 by the traditional culture. During such moments, teachers tended to miss attending to their pupils in class. BOX 12: Teachers’ voices about absenteeism “In most cases when a teacher falls sick, he /she are not able to teach. This does occur so often but in circumstances where a teacher is alone in class, there is no assurance that he/she will not be absent because of sickness and other unavoidable circumstances…If you’re alone in class like for P.1 there is only one teacher, so it becomes difficulty for someone to be available in class for the whole term” FGDs-SHRP-Kamuli “I would say unplanned for eventualities like sickness of a family members or death of relatives takes teachers away from schools. To the teachers, such scenarios necessitate one to lodge in an off request to the Head Teacher to be off and attend to the occasion to fit in the society because our society will wonder what happened why you could not attend a burial” FGDs￾SHRP-Pader “You might also fall sick which is inevitable though most people take it that teachers never fall sick” FGDs-SHRP-Kiruhura “It’s always difficult for a teacher to attend class when he /she losses a close relative or parents. This is not so often but when it happens, a teacher does not attend class” FGDs-SHRP-Kamuli Delayed salaries The leading reason for absenteeism reported by all teachers was delayed payment of their salaries by government. Teachers noted that in most cases payment of their salaries delayed and they have to look for other alternatives sources of income. Some teachers resort to riding boda boda (bicycles), farming and other commercial activities in order to raise income to take care of their families. It was common that at the end of each month, most teachers do not turn up at but will be elsewhere working to raise money to support their families. “Delay of salaries is an issue because most teachers depend on their salaries. It can reach to the 15th of a new month when you have not yet been paid your salary, yet we also have expenses to meet at home, our children need school fees among others. It means I have to resort to other means of raising money to pay for my child’s fees, meaning the whole of that one week I won’t teach” FGDs-SHRP-Kiruhura “Delayed salaries make some teachers dodge schools to ride boda bodas or go to gardens to look for other means of providing for their families. Most teachers would love to be in schools to teach but when salaries delay, they are forced to think of alternative ways of sustaining their families and thus class attention becomes secondary” FGDs-SHRP-Pader “Delayed salaries make teachers to find what to do to take their children to school and take care of their families. How can I take teach other children while mine are at home seated and I don’t even have transport to take me to school? That is why some teachers are absent. So the government should pay us early on time” FGDs-SHRP-Kamuli SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 221 Personal family responsibilities contributed to teacher absenteeism in some instances. Teachers noted that they sometimes like any other responsible parent, attend their children’s school PTA or visitation days and therefore they will be absent from school. “Some of the teachers are key persons in their homes ‘an eye of the family’ so in case your wife, parent or any other relative falls sick, you are supposed to be there and attend to him or her…Personal reasons like my child may fall sick and I have to take him or her to the hospital, therefore that day pupils will miss my lessons” FGDs-SHRP-Kiruhura “We are also parents with school going children so when they call for a parents’ meeting we have to attend” FGDs-SHRP- Kamuli Teachers also reported that pregnancy, which eventually results in maternity leave among the female teachers, causes absenteeism. Teachers noted that this maternity leave always lasts over two months. Maternity days do not include weekends and public holidays and as such female teachers do not attend to their class for almost the entire term. “You get a maternity leave and go home for two months…you might find that holidays and weekend are not counted in those two months of leave, hence making it to a whole term. You will find that a school has seven classes and seven teachers and no private teacher or PTA teacher. If she is not a person to say let me go back to attend to my pupils, she will leave the class unattended for the whole term.” FGDs-SHRP-Kiruhura According to the findings, limited or irregular supervision was noted as one of the factors leading to teacher absenteeism. Teachers become reluctant to come to school whenever there is limited or no supervision from the external officials like the DIS and DEO. Since even the head teachers who are supposed to ensure that teachers are in school and performing their duties are not regular in attendance, the teachers have an opportunity to stay away from school. “Poor and /or irregular supervision by the DIS who comes either once or at times not at all in a term makes teachers reluctant to come to school because some of them may have compromised the head teacher and or even learned his/her own personal weaknesses to the effect that they can be away without any caution.” FGDs-SHRP-Pader “Lack of adequate school supervision by the DIS and DEO makes teachers keep off school freely without fear, since the head teacher could also be absent or he or she could have been compromised.” FGDs-SHRP-Pader 3.3.2 FREQUENCY OF TEACHER ABSENTEEISM Teachers noted teacher absenteeism varied depending on the reason or nature of the problem teachers encountered. On average, it was noted that a teacher can be absent for around one week (7 days) and, in case of maternity leave, a teacher could be absent for over 2 months or even the whole term. “One week, for example in case a child is sick and has been admitted you can spend like one week” FGDs-SHRP-Kiruhura SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 222 “Absenteeism has been minimized because of unannounced visits by the stakeholders and supervisors” FGDs-SHRP-Pader “I think it varies depending on the nature of the problem and I can’t specifically estimate a given time period” FGDs-SHRP-Kiruhura 3.3.3 EFFECTS OF TEACHER ABSENTEEISM TO CHILDREN’S ABILITY TO READ After discussing the reasons for teacher absenteeism, teachers were asked to share their thoughts about what could be the effects of teacher absenteeism to pupils’ learning and reading. All teachers agreed that teacher absenteeism greatly affects children’s performance, raises school dropout rate, interrupts the content or flow of information, promotes indiscipline among the pupils, and greatly promotes pupil absenteeism. Poor performance among children was reported by all teachers from all the study districts as a far reaching consequence of teacher absenteeism to pupils’ learning and reading. Teachers noted that when a teacher is absent, his/her class if not adequately attended to by someone who volunteers to offer help. Instead, the volunteer keeps the children silent by giving them text books without teaching them. Others offer to teach only English and Mathematics but subjects with reference books are ignored, leading to poor performance by the children in both reading and final term examinations because the syllabus the children were supposed to cover is incomplete. BOX 13: Teachers’ voices about effects of teacher absenteeism “You can find a school having 600 pupils with only eight teachers therefore if one teacher is not around it means some pupils will not learn hence poor performance at the end of the term. I think the government should recruit more teachers to solve that problem” FGDs-SHRP-Kiruhura “Whenever a teacher is not in class, pupils will be crowded as classes are merged so a class of 82 pupils when merged with a class of 70, will make one teacher to have 152 pupils in a class at one particular point. And this definitely has a serious impact on learning” FGDs-SHRP-Pader “The syllabus will not be completed and the pupils will lack knowledge on some of the themes not only making it difficult for them to read but also to pass exams” FGDs-SHRP-Kiruhura “There will be nobody to teach the children.“If somebody else goes to class, he/she will not teach the pupils but rather just keeps them by giving them text books but not teaching” FGDs￾SHRP-Kamuli Promotes pupil absenteeism and school dropouts were commonly attributed to teacher absenteeism. Teachers noted that when one teacher is absent, the children develop a feeling that there is no one else who can teach them except their teacher. So when the teacher misses school twice, children of that particular class will not appear in school the next morning until they learn that their teacher is back. This therefore has significantly affected the pupils’ learning and reading. “When a P1 teacher is absent for two days, on the third day, pupils don’t come back to school because they say that our madam is not there. More so, P1 pupils know their teachers so even SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 223 when an absent teacher is replaced, they cannot learn because they cannot recognize them” FGDs-SHRP-Pader Teacher absenteeism also interrupted the content or flow of information in teaching, as reported by the teachers. When the absent teacher appeared in class, they will rush the children through the syllabus so as to complete at the same time with the teachers who have been teaching. This makes them skip some topics and hence affect children’s learning and reading. “Absenteeism affects the pupil’s learning to read. It is indeed a big problem because it interrupts the content or flow of information and makes teachers to start rushing to finish the syllabus and thus rush pupils who in the end are half baked” FGDs-SHRP-Pader Teacher absenteeism was also reported to have promoted a lack of discipline among the children. It was noted that pupils, especially Primary One students, always obey their teacher more than anyone else. So when their teacher misses class, these children become unruly and such behaviors develop from that age until they grow up. Substitute teachers therefore give up offering to help teach these kind of classes. “Children become wild. They fight in class and cannot be controlled because they are used to their teacher and seeing someone else in their class makes them become unruly. Some children understand their teachers only, especially P.1 pupils. They understand their teachers better than anyone else. Sometimes they fear and do not pay attention compared to when their teacher is around.” FGDs-SHRP-Kamuli 3.3.4 TEACHER SUGGESTIONS TO REDUCE TEACHER ABSENTEEISM In order to reduce teacher’s absenteeism, teachers suggested the following actions: construction of staff quarters, recruitment of teachers from the nearby community, improving teacher motivation, adequate and timely salaries, introduction of arrival and departure books, and frequent supervision. The teachers suggested that; government and parents should construct staff quarters at school to address long distances and or lack of transport to school. When the teachers stay within the school premises, they will be able to easily attend to their classes on time. “Many teachers come from distant places which in most cases affects their regularity at school. So by building staff quarters will help teachers be at school always. This can be done by the government” FGDs-SHRP-Kamuli “Parents should construct staff quarters so that teachers can stay in school. If he or she is near they will not complain about the rain” FGDs-SHRP-Kiruhura Teachers noted the local government should recruit teachers from the nearby community to reduce teacher absenteeism. Some teachers had been transferred far away from their communities, whereby they commute long distances and yet there are nearby schools in their communities which they could teach in while commuting comfortably from their homes. This, however, has a limitation as well, since not all communities surrounding the schools have trained and qualified teachers whom the government can recruit. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 224 “I think the district officials should recruit teachers from within the nearby localities because some teachers are transferred to distant districts and other hard-to-reach areas. So you find them not performing their roles and responsibilities. Other teachers decide to sit at home after being transferred to such areas and the district officials don’t recruit teachers to fill up that gap…For example I come from Rukungiri and my family is in Lwengo, so you might have a family issue that has disrupted you like burial or any other issue so you go well knowing that you will solve the issue either today or tomorrow and you find that it has not ended. So you end up staying there another day, hence being absent…Although transfers are normal but they shouldn’t transfer us to far distant places from our localities.” FGDs-SHRP-Kiruhura Increasing teachers' salaries and paying them in a timely manner was also noted as a way to motivate and reduce diversion of teachers’ focus from teaching to looking for other alternative sources of income. Further, more teachers recommended that government should increase their salaries to allow them meet their needs and focus fully on teaching. “We should be paid enough salary on time not only to motivate us but also save us from the worries which distract us from focusing on teaching these pupils. For example one may worry about loan officers coming for him or her” FGDs-SHRP-Kiruhura Other suggested ways to increase teacher motivation included: leaders openly appreciating their work, the provision of breakfast and lunch at school, and the introduction of PTA funds to pay teachers when their salaries have delayed. “Introduce PTA funds in order to motivate teachers and use that money to facilitate teachers when salaries are not ready.” FGDs-SHRP-Pader “Teachers need to be motivated and these can come inform of preparing meals for them, taking them for outings and openly appreciating their work before the parents and visitors.” FGDs￾SHRP-Pader “Head teachers should provide breakfast and lunch for teachers.” FGDs-SHRP- Kamuli Introduction of teacher arrival and departure books was also noted by teachers as a way of reducing teacher absenteeism. Most teachers recommended the introduction of arrival and departure books at school, reasoning that this would help create fear within the teachers, as they would dread appearing absent for several times in a week/term, hence reducing absenteeism. “Head teachers should put registration books for arrival and departure. Assume a teacher is absent for three to four daysIin a week; the teacher will feel ashamed if he/she sees it is only her/him who is always absent.” FGDs-SHRP-Kiruhura Regular supervision by the district leadership: Teachers expressed that district officials, like the DIS and DEO, should conduct regular supervisions as a way of keeping teachers alert at work and cracking down on teacher absenteeism. “There is need for regular supervision by the DIS to reduce absenteeism. When DIS earmarks a program to visit particular schools, this creates positive tension that teachers have got to be on SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 225 their toes making sure that their lesson plans are in shape and that they are teaching. DIS will exert pressure on the Head Teacher who will ably monitor and supervise the teachers.” FGDs￾SHRP-Pader "Continuous and timely appraisal of teachers to check on performance and give appropriate feedback will spur teaches to re-think on unnecessary absenteeism.” FGDs-SHRP-Pader 3.3.5 REASONS FOR PUPIL ABSENTEEISM Teachers were also asked to share their thoughts about the reasons for pupil absenteeism and how pupil absenteeism contributed to pupils' under performance. During the discussions, teachers reported child labour in economic/agricultural activities, parental neglect, negative parental perception towards SHRP, famine and drought, walking long distances to school, lack of pads for female pupils during menstruation periods, and corporal punishment at school as the major factors leading to pupil absenteeism. Engaging in economic/agricultural activities by children, such as sugar cane cutting in Kamuli, cattle rearing in Kiruhura, and open markets and charcoal burning in all the three sampled districts, were reported by teachers as major contributors to pupil absenteeism. In the Kamuli district, children were reportedly working in sugar cane and rice farms to earn money to buy scholastic materials and other schools needs. It was reportedly common that parents also sent their children to engage in child labour activities to earn incomes and support their family needs. This engagement in economic activities by children sanctioned by their parents meant that pupil absenteeism was systemic and sustained by parents. In Kiruhura, during the dry season, children are reportedly absent from school because they are engaged in moving their families’ animals in search of grass and water. In the wet season, some of the children remain home to help their parents in agricultural activities, while a few get engaged in charcoal burning to raise money for basic and scholastic materials. In the Pader and Kamuli districts, teachers reported periodic markets as a major factor leading to pupil absenteeism. Teachers noted parents often send their children to the markets to either sell or buy some items for the family. This activity was reported to have affected all the schools near these periodic markets. BOX 14: Teachers’ voices on reasons for pupil absenteeism “During sugar cane cutting and rice season, children go to the rice schemes/gardens to scare away the birds. Children can take three weeks to one month without coming to school. Parents tell their children to move from shamba to shambas until the season ends because of the economic status of the families.” FGDs-SHRP-Kamuli “Children in most cases become absent especially if it is a drought period. If it is a drought period, here you don’t see children at school. Almost the enrollment of the school is cut by a half since they need to move with the animals for long distances to look for water and grass. When it is a rainy season they have to stay home and cultivate crops because that is their chance. These children when they lack the basic needs some tend to go and burn charcoal since they SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 226 are looking after themselves and depending on seasons. Growing beans is just one season and so they will harvest after three months. This means that they will have to go for charcoal burning to raise money for school necessities like books and pens.” FGDs-SHRP-Kiruhura “Periodic markets are a major concern. Most pupils are sent to markets to sell items and also buy others. Schools in areas with periodic markets are always affected by the pupil attendance on these days when markets are in operation.” FGDs-SHRP-Pader “Sugarcane cutting is affecting school attendance. Children go to cut sugar cane in peoples gardens to get money which makes them be absent from school for some good time.” FGDs￾SHRP-Kamuli Negative parental perception towards SHRP and education at large. Teachers of all the study districts noted that parents’ attitudes towards SHRP were negative and they therefore retained their children at home or sent them to private schools for better education. Teachers noted SHRP supported schools were viewed as teaching their children in the local language, which parents believed pupils already knew. Especially the earlier grades are seen as a waste of time, while those in upper classes are likely to be provided with the school requirements and necessary fees, while those in lower classed are kept at home . “Parents perception about the program is negative… Some few parents who do not understand the project do not send their children to school on a daily basis because of their perception about the project saying that teachers teach their children in Lusoga (local language).” FGDs￾SHRP-Kamuli “Some parents have seen little or no value of education and as such, they can even tell their children not to go to school but instead attend to domestic chores at home like babysitting, cooking, taking care of ill siblings, digging and running other errands.” FGDs-SHRP-Pader “Some parents’ don’t value these early childhood education levels (primary 1-3). They look at it as waste of time and resources; not knowing that it’s the foundation of their entire children’s education. And even when you go to these private schools, fees defaulters are associated with the lower classes” FGDs-SHRP-Kiruhura Famine and drought were also reported as one of the major factors for pupil absenteeism. Teachers noted that when there is famine and drought, families find it hard to provide food for their families. Children therefore abandon school and join their parents for work in other peoples’ farms to get what to eat. This therefore catalyzes pupil absenteeism. “During the dry season famine is very common in this area; some pupils come without packed food while others even fail to have supper. In some homes they may fail to have supper and they are not sure of that day’s lunch. Therefore there’s no way a parent will send such a child to school early in the morning but instead will tell the child to wait for the evening so as they can go and farm on someone’s garden in return for food. And in the long run it affects the child’s school attendance.” FGDs-SHRP-Kiruhura SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 227 “Famine and hunger at household level especially in second term during the rainy season when pupils might stay for over a week because they are working far from homes in gardens either planting, weeding or harvesting.” FGDs-SHRP-Pader Lack of sanitary pads for menstruation hygiene among the girl children was reported mainly by teachers in Pader district as a challenge that girls still face at school. Teachers noted that when girls are getting their monthly periods, they still have none to trust with such secrets, not even their parents. They therefore makeup reasons to stay away from school and hence increase pupil absenteeism. “Some mature girls fear to come to school because they are in their monthly periods. The fact that most parents have poor parent-child relations makes it hard for the girls to be open to them about their circumstances and as such they battle it silently and create reasons to be away from school.” FGDs-SHRP-Pader Findings indicate that parental negligence was also a major factor for pupil absenteeism. Teachers noted that parents had what they considered an ‘I don’t care’ attitude towards their children’s education. Some parents did not provide their children with scholastic materials even when they have been sent back home for this. There is also less effort put in by parents to encourage their children to stay at school, as children chose when to and not to go to school. “Some parents do not care about their children education. They do not provide basic needs especially the scholastic materials for their children for instance books. Some children come to school with one book or without “When we send the children home for books, they sometime take a month without coming back to school” FGDs-SHRP-Kamuli Teachers also reported corporal punishment in schools as one of the main reasons for pupil absenteeism. When a teacher inflicts pain upon a child at school, the child becomes timid and fears to come back to school the next day. “If a child reaches at school and you treat him/her harshly or punish him/her harshly, h/she will stop coming to school. Let us say it is a P.1 pupil who makes a mistake and you give her a hoe to dig around the school, the child will stop coming to school, like 10 strokes I think that child will be absent the next day.” FGDs-SHRP-Kiruhura Diseases and common illnesses such as malaria and nodding disease have contributed greatly to pupil absenteeism, particularly in Pader district. Children who get affected with nodding disease take long to recover and therefore do not report back to school for a long time. “Sicknesses; malaria cases and children with nodding diseases take long to recover. For example, in one school, nine pupils from a class of 25 pupils were affected by the nodding diseases and they were absent for a full term” FGDs-SHRP-Pader Long distances to schools. Teachers also noted that children walked long distances to school, which affected school attendance. Despite the fact that schools do exist in the districts, they are few and far away from some communities. So children walk long distances to school and when they get tired due to the long journeys, they either start missing some days or completely drop SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 228 out. This is exacerbated by poor weather, particularly during rainy seasons where most roads and paths to school are impassable. “Some children get tired of walking long distances, which has made some to drop out of school. The child has nothing to eat and yet walks a long distances. Two, three to four days the child will stop coming to school and if he is forced, he will stop on the way and if there is no intimacy between the parent and the teacher, the child will end up dropping out of school.” FGDs-SHRP￾Kiruhura “Child-headed families: This affects their attendance because there is little direction but also some of these children are studying in urban or peri-urban areas and as such they have to leave and go to the villages to pick food, which they have during their school days. During this time, chances are these children are going to miss some school days.” FGDs-SHRP-Pader Pupil absenteeism is very common and on average 10- 30 children miss out school on a daily basis during the rainy season. Every time it rains, children stay at home. They do not come to school.” FGDs-SHRP-Kamuli 3.3.6 EFFECTS OF PUPIL ABSENTEEISM LEARNING AND READING ABILITY Teachers were asked to share their views on the effects of pupil absenteeism on children’s learning and reading ability. All teachers from the three study districts reported poor performance and subsequent school dropout, as the only main outcome of pupil absenteeism. Teachers explained that their method/procedure of teaching follows a spiral curriculum, whereby themes or content is passed on from simple to complex. So if a child misses a couple of lessons, they will not be able to answer questions set about what they missed. This has led to great decrease in pupils’ performance in both learning and reading and also during the end of term exams. BOX 15: Teachers’ voices on pupil poor performance as a result of pupil absenteeism “Children miss out some concepts and therefore cannot understand. For example if I teacher a certain concept today, I build on the same concept which the child missed, the following day. By the time the child comes back, he /she has missed a lot. Much as the child comes back, he or she cannot catch up.” FGDs-SHRP-Kamuli “We have learning books and it is a spiral curriculum. It is from simple to complex, which means if a child misses, she/he will not know where to begin from when she comes back to school; she/he will not know how to answer and will not catch up. We have reading lesson 1, 2, 3 and so when a child misses one he/she can’t answer any question previously taught.” FGDs-SHRP￾Kiruhura “That child will perform poorly and will result in dropping out.” FGDs-SHRP-Pader “When a pupil is not in school, he or she is not learning and it creates a big reading gap.” FGDs￾SHRP-Pader SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 229 3.3.7 TEACHERS’ ROLES IN ENGAGING PARENTS REDUCING PUPIL ABSENTEEISM Due to the high rates of pupil absenteeism and its effects on pupils’ learning and reading, teachers were asked if they have ever taken time to talk to parents about pupil absenteeism and if so, what was the reaction of the parents and if it all changed the trend of pupil absenteeism. All teachers reported having attempted to talk to some parents about the problem. However, there were mixed reactions from parents from all the sampled districts of study. Teachers reported that some parents reacted negatively and as such teachers did not witness any change in their pupils' rate of absenteeism, while those parents who reacted positively sent their children to school. The teachers reported that they conducted face to face and phone discussions with parents about their children’s education by sensitizing them about the value of education. Parents who reacted positively to what the teachers discussed with them started giving them breakfast and lunch, followed their children to school to ascertain their attendance, and personally encouraging their children to go to school. A few children started attending school more regularly as a result of this parental support. “I have taken time to talk with parents and to some, it has made a difference because some parents understand the value of education” FGDs-SHRP-Pader “There is a positive change. We try to sensitize these parents; we tell them that Uganda is developing very fast. If you have a family of four boys and you have 20 cows if they all mature and marry the cows will be no more. But if you educated a child he or she can support your family in future upon attaining a job despite having cows at home. FGDs-SHRP-Kiruhura Some of these parents have appreciated the need to educate their children…With sensitization of parents some have improved on supporting their children’s education. Those parents have gone ahead to sensitize their fellow and at the end we have registered some success” FGDs￾SHRP-Kamuli “When a pupil has taken long to come to class, I call on the parent to ascertain the reason the pupil is away. I also have interclass meetings with parents; there is improvement but not so much” FGDs-SHRP-Pader However, parents who reacted negatively still kept their children home. Such parents used threatening words against the teachers and towards the by-law enforcers. For example, in the Kamuli district, it was reported that parents who engaged their children in sugar cane cutting and rice fields labour threatened to beat the by-law enforcers if they arrested their children for not going to school. Other parents were reportedly transferring basic needs provision responsibilities to either the teachers or the President, branding the children as ‘Museveni's children’. Other parents attributed their long distances for their children’s not going to school and famine for not packing food for their children. These mixed attitudes and actions by parents have therefore hindered the teachers’ efforts to bring a positive change regarding pupil absenteeism. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 230 3.3.8 TEACHER SUGGESTIONS ON HOW TO REDUCE PUPIL ABSENTEEISM Teachers were asked to share their thoughts on how to reduce pupil absenteeism. The following suggestions were made by the teachers for reducing pupil absenteeism: continuous parent and guardian sensitization on the value of education, construction of more schools in the communities, close monitoring of pupil attendance by teachers and other stakeholders, child motivation, and complete abolition of corporal punishment. Continuous parents and guardians sensitization on the value of education by district education officers, CCTs, teachers and SHRP staff will contribute to changing parents negative attitudes to education and will therefore send their children (back) to school. “There should be continuous sensitization about the benefits of education to help the parents change their attitude towards education” FGDs-SHRP-Kamuli “Sensitize parents through meetings and radio talk shows and this can be by head teachers and teachers during school meetings and class meeting, FA, CDO and Secretary for Education. With this, parents will see the need and take action to reduce on pupils’ absenteeism” FGDs￾SHRP-Pader “There is need to sensitize parents in this community because some don’t appreciate the need of attending school. Actually most of the rich people in this community are not educated so they ask their children ‘why are you going to school? For me I didn’t go to school, aren’t I rich? Stay home and graze our cattle’” FGDs-SHRP-Kiruhura Teachers also suggested that more schools should be constructed by government closer to children to reduce the distances children walk to get to school. This would reduce the number of children who miss classes due to long distances and those who stop on the way when forced by their parents to attend school. “Walking long distances to school is one of the causes of absenteeism, so I think the government should construct more schools in different areas.” FGDs-SHRP-Kiruhura Teachers also agreed that teachers should motivate pupils to attend school. Teachers reported that they could do this by having enough attractive teaching materials to keep children interested in coming to school the next day. They also noted that they should award academic performers gifts (like books, pens and pencils) and create good teacher-pupil relationships to make children feel that they have positive role models. This, according to the teachers, would motivate children to keep coming back to school and reduce pupil absenteeism. “As teachers we should have enough teaching materials, nice songs, poems among others such that; if this pupil whose is still at home covered under his or her blanket will be motivated to wake up and run to school so as not to miss such.” FGDs-SHRP-Pader “Children should be rewarded when they perform well by giving them books, pens and other gifts by teachers and visiting them.” FGDs-SHRP-Kamuli SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 231 “We also should have school Family Initiative which we put in schools and tried to show that we love children and told them that the head teacher is your grandfather/ mother, the teachers are your parents that is where your biological parents are not. So when children would go home they would tell them that I have got another mother at school and I have a good grandfather at school. So we would invite parents to school and some other times pupils would invite us to go to their homes to go and speak to their parents” FGDs-SHRP-Kiruhura As teachers perform their role to ensure pupil motivation to stay at school, they also suggested that government officials and other stakeholders from the community level should closely monitor children’s school attendance to reduce pupil absenteeism. The teachers further explained that the government should follow up to ensure that all children of who are school aged are at school by enforcing by-laws at community level. This would involve arresting of all parents who keep their children at home when it is a school season. “The government should implement its policy of making all children within the school going age attend school. Government officials must follow up to find out whether such children are attending school but not design a policy and leave it there… Community leaders must monitor parents within their jurisdiction, who are stopping their children from attending school, forcing them into early marriages, and housekeeping work so as to ensure that the policy is put into practice.” FGDs-SHRP-Kiruhura “Enforce bylaws at community level whereby local leaders launch an operation to arrest parents who keep children at home and this can be done by the LC1 and the secretary for education. This will instill fear in parents so that they send their children to school and children will be forced to attend school and not run to periodic markets.” FGDs-SHRP-Pader Abolishing corporal punishments at school was one of the measures teachers reported could contribute to the reduction of pupil absenteeism. Teachers confessed that corporal punishment scares children from attending school, and the only way to reduce pupil absenteeism was by teachers themselves avoiding this practice. The government has abolished corporal punishment in school, but the majority of schools are still applying it on children. “Teachers should avoid corporal punishments in order to make children like education.” FGDs￾SHRP-Kamuli 3.4 COMMUNITY MOBILIZATION/INVOLVEMENT The intervention of community mobilization was primarily evaluated by asking teachers about parental involvement in the pupil's education to assess whether the community was properly mobilized in order to reach the intended goals of the program. Teachers were asked about what actions, and by whom, can be helpful in improving community mobilization about reading. Word cloud for community mobilization Teachers reported that some parents ‘102’ took a keen interest and got involved in the SHRP program, while other parents had a negative perception about the program to the extent that they have transferred their children ’64’ to private school. The parents who supported children SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 232 provided them with scholastic materials and other reading materials. A few parents started reading with their children at home and some pupils ‘62’ invited teachers to their homes, which provided an opportunity for teachers to enlighten the parents about the SHRP ‘44’ program. PTA meetings were also held at school, which enabled parents to learn more about the SHRP program ‘35’ Figure 7: Word cloud for community mobilisation The majority of the teachers reported that there was limited involvement by the parents in the SHRP program. The reasons given for limited involvement by parents included: insufficient knowledge about the program, a negative perception that children are being taught in the local language, and an inadequate mobilization of the community to engage in the program. Lack of knowledge about the SHRP program Many parents lacked knowledge about the programme because they have never been sensitized about the programme. There was no systematic process in place to sensitize parent about their role in improving learning and reading for their children. The SHRP field staff has not adequately interfaced with the community to raise awareness about the program, which provided room for speculation and negative perception resulting in some cases parents moving their children away from the SHRP supported school to private schools. Some of the negative perceptions and myths included perceiving the program as a vernacular learning and reading program. There was no evidence that the SHRP staff had regularly moved into the community to address such a serious perception. Box 16: Teachers perceptions on community involvement “Community does not fully understand the SHRP project and the few who have ever heard about the SHRP project consider it a LUSOGA programme.” FGD-SHRP-Kamuli “Parents should conduct school tours for parents and practically see what their children are learning. Parents will know that this is not a local language program but that pupils and leaning and reading well. These can be done during class days to allow parents and interact with the teachers and the child.” FGD-SHRP-Pader Word Length Count Parents 7 102 Children 8 64 Pupils 6 62 School 6 53 SHRP 4 44 Program 7 35 SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 233 “The community does not know about the SHRP project because they have not been sensitized and informed about the project”. FGD-SHRP-Kiruhura “Parents perception about the program is negative. Some few parents who understand the project do not send their children to school on a daily basis because of their perception about the project that teachers teach their children in Lusoga (local language)” FGD-SHRP-Kamuli “Some parents were against it because they think when a child speaks English, he or she understands everything. Therefore some had a negative attitude towards teaching their children using local language and preferred their children being taught in English” FGD-SHRP-Pader “Why I said that parents didn’t like it, at our school before this program came in we used to have bigger numbers of pupils in lower primary but once they introduced SHRP the numbers dropped” FGD-SHRP-Kiruhura “When we went for these trainings, head teachers were instructed to call for meetings purposely to inform parents about how the program operates before it was implemented but this did not happen as was planned.” FGD-SHRP-Pader “Previously we would teach all pupils in English but when this program came, we started teaching pupils in lower primary (P.1 –P.3) in local languages. The Head teachers were instructed to call for parents’ meeting to inform them and deliberate on this approach.” FGD-SHRP-Kiruhura During the group discussions teachers reported that some schools that had made an attempt to sensitize the community about the programme had yielded positive results. Parents had a positive perception about the programme and the school enrolment in lower primary had improved. “I understand that the SHRP program started in 2012, upon completion of the training, we were advised to mobilize parents to inform them about the on the benefits of the programme. When we did this we received many pupils. Even when you look at these other community schools parents advise teachers to teach their children in early classes using their mother tongue because they want to rebuild it and not to lose it” FGD-SHRP-Kiruhura “Previously you would find a primary seven pupil not in position to read a Runyankore letter, therefore when SHRP program was introduced parents welcomed it” FGD-SHRP-Kiruhura “Both teachers and head teachers through PTA and management meetings involved parents in the SHRP program. However sometimes through churches the teachers would sensitize the congregation about the benefits of the program.” FGD-SHRP-Pader All the teachers agreed that it was necessary to involve the parents in the early grade reading and learning because it allowed for programme ownership by the community, parents would understand the benefits of the programme and would support their children with the basic needs to enable them adequately read and learnt both at school and at home. “For any product to sell, you must involve the customers. Therefore since parents are our custodians because they give us pupils in our respective schools where SHRP is being implemented, it was necessary to involve them prior to this program” FGD-SHRP-Kamuli SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 234 “Parents have the authority to decide for their children, a parent may choose to or not to take his or her child to a school implementing SHRP FGD-SHRP-Kiruhura “It was necessary to involve the community because parents had to be informed about the SHRP program particularly; how it works, the benefits it attributes to the learners by improving on their reading skills, as well as imparting confidence in the pupils and enhancing their interaction with others” FGD-SHRP-Kiruhura 3.4.1 IMPACT OF COMMUNITY INVOLVEMENT Teachers reported a few positive aspects of parent involvement, including improved communication between parents and children while using the mother tongue correctly and promoting positive cultures and social norms in the community. This was being enhanced through storytelling, which helped a child to learn adequately. ‘There was need to involve the community because since we started teaching pupils in their mother tongue it has eased communication between our pupils and their people at home. They can easily understand each other instead of talking to them in English.” FGD-SHRP-Kamuli “It promotes culture and ease learning for example when it comes to storytelling, you can tell the pupils to go and inquire some nice stories from their grandparents for the following day’s class exercise and the relatives find easy to teach them.” FGD-SHRP-Kiruhura “Parents have also procured translated Bibles to improve on their children’s reading skills.” FGD￾SHRP-Kiruhura Teachers reported a few parents helped their children learn how to read and pronounce some words while at home. Children who were being supported by their parents were performing better in reading and learning. “Active participation of parents in guiding their children to read, the right pronunciations of words, the accent. This has enabled the pupils to merge the two explanations from both teachers and parents hence becoming experts in reading” FGD-SHRP-Pader “A parent has become a second teacher to his or her child’. We usually give textbooks to these pupils to carry along when going home so as to practice more reading, so when they reach home parents guide them. And because of that pronunciation of words and reading Runyakore sentences has become very easy for pupils”. FGD-SHRP-Kiruhura “I would say parents are experts when it comes to the local language; they really help a lot to teach these pupils how to read.” FGD-SHRP-Kamuli A bond between teachers, parents and pupils was created. When parents or the community are involved, the concerns of the pupils are shared between the parents and the teachers and solutions are generated on how to best help the child. If what troubles a child is resolved, it helps a child to concentrate on his/her studies. “It enhanced friendship among teachers, pupils and parents. Sometimes a child may be sent to school but he decides to hide in the bush, so when you meet the parent and inquire from him or her about the child’s absenteeism. The parent will tell you that his or her child leaves home every SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 235 day for school, so you work hand in hand with that parent to address that gap and in the end you find when the child has concentrated on studying.” FGD-SHRP-Kiruhura “Some community leaders motivate pupils to read by giving best readers gifts. He or she may visit a school and ask pupils to read a given text and the best reader is awarded a prize, then he may go ahead to promise that such competitions will continue running maybe every Friday. Therefore this child who is thinking of remaining home will think of how he or she can improve on the reading to compete favorably. “FGD-SHRP- Pader Parent involvement has helped children to read perfectly and the burden is not only left on the teachers alone. When these pupils reach home, parents take the initiative or perfecting their reading skills which helps the child to learn faster. “I have liked it because ‘two heads are better than one’. When this child leaves school after learning how to read and the parent guides him or her more, that pupil becomes a perfect reader because a parent is building on what the pupil has learnt from school.” FGD-SHRP-Kiruhura 3.4.2 SUGGESTIONS TO IMPROVE COMMUNITY INVOLVEMENT Improve community sensitization. The community should systematically and regularly be sensitized on the benefits of the program in regards to their children’s learning and reading. This should be done collaboratively between teachers, SHRP staff and the district leadership. This sensitization will particularly support in improving the negative community perception about the programme. “Teachers and RTI staff should sensitize the community on the benefits of SHRP.” FGD-SHRP￾Kamuli “There is need for more sensitization which should be done by the teachers and SHRP project team to help the community understand its programs and change perceptions about the SHRP project.” FGD-SHRP-Pader Teachers, head teachers, district officials and RTIs should provide more sensitization should be done to make the community appreciate the benefits of the SHRP program to change the negative attitudes of some parents. FGD-SHRP-Kiruhura “Conduct termly tours for parents and practically see what their children are learning. These can be class days to allow parents and interact with the teachers and the child.” FGD-SHRP-Kamuli ”I think teachers from schools implementing SHRP should be more hardworking, to ensure that their pupils perform highly in the final PLE exams than the other schools, so as to make community members witness the products of SHRP. This will help in changing their negative attitude towards the program.” FGD-SHRP-Kiruhura Encourage parents and government to provide learning materials and other basic need of the children to enable children have interest in reading and learning. “The government should give more reading materials like text books, charts and flash cards to enable every pupil get his or hers. Therefore when this pupil goes back home and informs his or her parents that I was given my own reading materials, this will motivate other parents to take their children to SHRP implementing schools” FGD-SHRP-Kiruhura SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 236 Ensure a uniform curriculum in all schools: The government should also ensure that the curriculum taught is the same throughout the whole country and in all schools. This will limit transfers of children to other schools. “District leaders too need to be brought on board to increase awareness for the program because there are those who criticise the objectives of the program without knowing its impact on the pupils” FGD-SHRP-Pader “The problem we have in Uganda is love for money; the government is registering those private schools. Currently it’s like private schools operate on an independent different syllabus from that of government schools and in the end we all seat for the same final exams. And because of that when a private school gets 15 pupils in grade one, government schools are considered as serving no purpose but it is because the government has failed to come up with a clear policy on that issue” FGD-SHRP-Kiruhura Utilize PTA meeting to sensitize parents about the program. During PTA meetings and school visits, teachers and head teachers should mobilize different activities like speech and reading competitions to raise awareness on the SHRP programme. It is easy to win a parent’s heart if he/she sees a child reading comfortably. “Having speech days for all children to present before their parents, what they have learnt at school” FGD-SHRP-Kiruhura “Reading competitions can be made for pupils to read before their parents to witness and, as a result, parents will acknowledge that their children are learning at school” FGD-SHRP-Pader 4.0 RECOMMENDATIONS Recommendations have been developed based on the findings of this study and the suggestions of FGD participants. They are grouped into four areas: Teacher Training, Teaching Materials, Teacher Motivation and Support, and Combatting Pupil Absenteeism. The recommendations include both actions that could be undertaken by the SHRP or similar projects and those that may require actions by other stakeholders (such as local or national governments). Teacher Training Redesign the training course to allow for more exposure and understanding of the methods and content. Most teachers felt that they could not effectively apply the new methods and content to the children in class because they were not fully equipped to apply the new methods and content due to a short period of exposure to the course. Consideration should be given to redesigning the training course for teachers to increase the frequency and length of training. A systematic follow up plan should be put in place for conducting refresher courses with clear monitoring indicators agreed on between RTI and those conducting the training courses. Limit transfer of SHRP trained teachers. The district leadership in the SHRP implementing schools should minimize transferring teachers who are trained in the SHRP early grade reading and leaning programme. The trained teachers are often replaced with those who are not trained in the SHRP methodology and therefore, cannot adequately manage the programme. The SHRP staff in collaboration with the Centre Coordinating Tutors should identify schools whose SHRP￾trained teachers have been transferred and be replaced to fill this gap immediately to ensure effective knowledge transfer from teachers to pupils. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 237 Teaching Materials SHRP English books should be revised to include detailed grammar and literacy one. All the missing content in the SHRP books, particularly grammar and literacy one should be identified and included in the revised version of the textbooks using a participatory approach. The existing errors in the text books should be identified and corrected. A small group of teachers from the implementing schools should be involved in the revision process. Encourage cross- referencing of textbooks: RTI should identify and recommend particular textbooks for cross referencing to support the existing SHRP text books particularly to support literacy one which is limited in content in the RTI books. Linkages with other locally produced books like MK and Fountain books should be created. Align SHRP textbooks with the National primary schools curriculum: It was noted that some aspects of the SHRP text books are not fully aligned with the National primary schools curriculum. It is recommended that a small team that is composed of SHRP- trained teachers and members from the Uganda National Curriculum Development Centre be constituted to further review the SHRP books to ensure alignment. Improve supply of teaching and learning materials in a sustainable manner. Teachers reported to have received limited support through provision of teaching, learning and instruction materials. This type of support was, however, inadequate and irregular in all the schools. Some schools had not taken on full ownership of providing teaching and learning materials. It is recommended that school administrations provide the teaching and learning materials like manila papers, makers, etc in a sustainable manner, to allow the teachers apply the methods and content to the learners adequately. Teacher Motivation and Support Explore ways to motivate teachers. The teachers noted that they are not recognized for their contribution to learning and reading which has led to low teacher morale. Teachers should be motivated through non monetary incentives like provision of certificates of attendance and recognition on open days. Improve teacher support supervision: Every teacher had received some form of support supervision from either the internal or external supervisors, but this was said to have been inadequate to enable teachers apply the taught content and new methods in the classroom. It is recommended that the SHRP staff in collaboration with the CCTs and the district leadership develop a comprehensive and coordinated systematic support supervision schedule that is well communicated to the schools to support the SHRP implementing teachers in all schools. Facilitate peer to peer teachers support through the exchange programme. RTI in collaboration with the CCT and the District Education Officer should identify teachers who are excelling in the SHRP programme to provide peer support to other teachers. Introduction and facilitation of team supervision (exchange learning and supervision visits between schools) as one of the ways to enhance confidence and mentorship support among teachers was also encouraged. Improve teachers' housing on the school premises to address teacher absenteeism: The leading cause of teacher absenteeism was cited as long distances to schools between the teachers' homes and the schools where they teach. Teachers were unable to attend classes, SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 238 particularly during rainy seasons where roads and paths are inaccessible. The government may consider working in collaboration with the community to build teachers' quarters to reduce teacher absenteeism. Combatting Pupil Absenteeism Increase alternative sources of household incomes to address pupil absenteeism. Pupils were involved in agricultural and other economic sources of income to raise money for their families, for example cattle rearing, sugarcane cutting and chasing of birds in rice fields. Parents should not only be sensitized about the dangers of child labour but also be linked to alternative sources of income that can free children to attend classes. RTI should work with the district leadership to enact bylaws on child labour and absenteeism in every district Adapt alternative methods of disciplining children. Corporal punishment in schools was cited as one of the main causes of pupil absenteeism. Corporal punishment inflicts pain and fear upon the children. Teachers in the SHRP schools need to be monitored to ensure that they are adopt alternative methods of disciplining children, as stipulated in the national guidelines on discipline. Improve menstruation hygiene management for the girl child in schools. Lack of sanitary pads for menstruation hygiene was reported as one of the leading causes of absenteeism among girls. Through the health component of the SHRP project, RTI should design a long term strategy in collaboration with other key stakeholders to ensure girls’ access to sanitary pads to retain them in school during their monthly menstruation periods. Work to change the negative parental perception towards SHRP. Teachers from the study districts noted that parents’ attitude towards SHRP was negative. The community considered SHRP's Early Grading Reading and Learning a vernacular school programme. The RTI leadership should develop a comprehensive intervention so as to change this negative perception in the community. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 239 Appendix A – Consent form for teacher participation INFORMED CONSENT FORM TEACHERS PARTICIPATING IN THE FOCUS GROUP DISCUSSIONS Name of Principal Investigators: Richard Wamimbi and Alicia Menendez Name of Organization: NORC at the University of Chicago Contact Telephone: 0776880446 Title of Protocol: Exploration of Possible Causes of Underperformance of the school Health and Reading Programme (SHRP) in Uganda PURPOSE OF STUDY Hello to you. My name is . I have come here with my colleague(s) to conduct some research about the USAID-funded School Health and Reading Program activities which is being implemented in your school. We are here on behalf of NORC at the University of Chicago, a US-based research agency that is conducting an evaluation of the project. SELECTION You were purposively selected to participate in this study. We shall be conducting a focus group discussion to get your thoughts about the performance of the School Health and Reading Program (SHRP). Your responses will be treated with utmost respect. There are no “right” or “wrong” answers, since the information needed is based on experiences, perceptions, and opinions. All your answers shall be completely confidential and your name shall not be mentioned in the report. (Do you know why we are asking you to take part in this study? Do you know what the study is about?) VOLUNTARY PARTICIPATION Your participation in this study is voluntary. It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked to sign a consent form. After you sign the consent form, you are still free to withdraw at any time and without giving a reason. Withdrawing from this study will not affect the relationship you have, if any, with the researcher. You are free not to answer any question if you do not wish to. If you withdraw from the study before data collection is completed, your data will be returned to you or destroyed SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 240 BENEFITS There is no immediate and direct benefits associated with your participation in the research. The information we collect will provide information that will be used to improve reading and health programming for children in Uganda (If you decide not to take part in this research study, do you know what your options are? Do you know that you do not have to take part in this research study, if you do not wish to? Do you have any questions?) CONFIDENTIALITY We shall ensure that we do not disclose you name or use your name to anybody and it will not appear anywhere in this report. Your responses to this study will be anonymous. The information obtained from you will be stored in a safe place where it cannot be accessed by unauthorized people. The audio tapes we shall use during the study will be kept securely locked away and will be destroyed (by burning) as soon as the data has been transcribed off them. DURATION If you decide to take part in this study, your participation in this research will last for about 2 hours You are however free to pull out or stop at any time if you so wish. (If you decide to take part in the study, do you know how much time will the interview take? Where will it take place? Do you know that we will be sending you transport to pick you up from your home? Do you know how much time will the discussion with other people take? If you agree to take part, do you know if you can stop participating? Do you know that you may not respond to the questions that you do not wish to respond to? Etc. Do you have any more questions? REIMBURSEMENTS/COMPENSATION As mentioned earlier there is no immediate and direct benefits associated with your participation in the research; however you will be provided with some refreshments ( one soda and a snack) during the group discussion and transport reimbursement of USHS 20,000/=. (Can you tell me if you have understood correctly the benefits that you will have if you take part in the study? Do you know if the study will pay for your travel costs and refreshments? Do you have any other questions? SHARING THE RESULTS The information you will provide will be shared among stakeholders to inform activities to improve reading and health programmes for children in Uganda. The research findings will therefore be shared through workshops, conferences and publications. PHOGRAPHY, VIDEO AND TAPE RECORDING SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 241 I understand that I will be photographed, tape recorded or videotaped by the researcher. These tapes will be kept by the researcher in a locked filing cabinet. I understand that only the researcher will have access to these tapes and that they will destroy by 30th October 2017 Video recording of study activities Interviews may be recorded using video devices to assist with the accuracy of your responses and they can be published in the reports for others to learn from. You have the right to refuse the video recording. Please select one of the following options: I consent to video recording: Yes _______ No_______ Audio Recording of Study Activities Interviews may be recording using audio recording to assist with the accuracy of your responses. You have the right to refuse the audio recording. Please select one of the following options: I consent to audio recording: Yes _______ No_______ Photographing of study activities / participants Photographs of participants may be taken to preserve an image related to the research and they can be published in the reports for others to learn from. You have the right to refuse to allow photographs to be taken. Please select one of the following options: I consent to photographs: Yes _______ No_______ WHO TO CONTACT This research has been approved by the Makerere University School of Social Sciences Research Ethics Committee (MAKSS REC) and Uganda National Council for Science and Technology (UNCST). If you would like to talk to someone other than the researcher(s) about; (1) concerns regarding this study, (2) research participant rights, (3) research-related injuries, or (4) other human subjects’ issues, please contact: Dr. Stella Neema, The Chair, Makerere School of Social Sciences Research Ethics Committee Telephone: +256- 772 457576, E-mail: sheisim@yahoo.com Or Dr. Peter Ndemere, The Executive Secretary, Uganda National Council of Science and Technology, Kimera Road. Ntinda P. O. Box 6884 Kampala, Uganda, Telephone: (256) 414 705500, Fax: +256-414-234579, Email: info@uncst.go.ug Or SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 242 Richard Wamimbi, Principle Investigator, NORC at Chicago University, Telephone: Richard Wamimbi, Email: richardwamimbi@gmail.com Certificate of Consent I understand the provided information and have had the opportunity to ask questions. I understand that my participation is voluntary and that I am free to withdraw at any time, without giving a reason and without cost. I understand that I will be given a copy of this consent form. I voluntarily agree to take part in this study. Participant_____________________ Signature of Participant ___________________ Date ___________________________ Day/Month/Year Statement by witness I have witnessed the accurate reading of the consent form to the potential participant, and the individual has had the opportunity to ask questions. I confirm that the individual has given consent freely. Name of witness____________ Signature of witness _____________ Date ________________________ Statement by the researcher/person taking consent I confirm that the participant was given an opportunity to ask questions about the study, and all the questions asked by the participant have been answered correctly and to the best of my ability. I confirm that the individual has not been coerced into giving consent, and the consent has been given freely and voluntarily. A copy of this ICF has been provided to the participant. Name of Researcher/person taking the consent________________________ Signature of Researcher /person taking the consent__________________________ SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 243 Date ___________________________ Day/month/year SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 244 Appendix B – Focus Group Discussion Guide FOCUS GROUP GUIDE FOR TEACHERS 1. Identification 101 Write the name of the District 102 Write the name of the moderator 103 Write the name of the note taker. 104 Write the number of the participants in the discussion (Male or Female) 105 Note the start and end time of the discussion 2. Introduction [After obtaining consent from each participant, start the discussion by introducing yourself again] The interviewer or the interviewing team greets participant and introduces itself – mention your name, where you are from and the purpose of the visit. Greetings to you! Thank you very much for agreeing to speak with us today. My name is . I have come here with my colleague(s) . We are all here on behalf of NORC in the University of Chicago, U.S.-based research organization, to ask you some questions about the performance of the School Health and Reading Program (SHRP), which has been implemented in your school for the past 3-4 years. You were randomly selected to participate in this study. The information you provide today will be used by stakeholders to improve the School Health and Reading Program (SHRP) to ensure the well- being of children. Your responses will be treated with the utmost respect. There are no “right” or “wrong” answers since the information being solicited is based on your experiences, observations, opinions, and perceptions. All your answers shall be completely confidential and your name shall not be mentioned in the report. This discussion will last approximately 2 hours. We shall have some refreshments. Request participants to introduce themselves. Get permission from the respondent to use a tape recorder, explaining that it would help to capture the whole discussion and ensure that nothing is missed out. Do you have any questions before we begin the discussion? (Give time to take and address questions thoroughly and then start with the discussion) Request to continue with the discussion. SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 245 3. Issues for discussion A. TEACHER TRAINING Some teachers from your school received training provided by the School Health and Reading Programme. 1. Did you receive training from USAID/SHRP on early grade reading instruction? When? More than once? For how many days? (we want to have an idea about how exposed to training they are) 2. After the training, did you feel equipped to tackle this new approach to teaching how to read in the classroom (If YES - probe for WHY and HOW. If NO probe for why) 3. Please share with us the methods and techniques that instructors taught during the training (Probe for strength, gaps, persuasiveness). 4. Did you apply these methods in your classroom practice? (If YES probe for how often and which methods were used. If No probe why these methods were not applied) 5. Do you feel you were able to competently use these methods to their full potential in the classroom? (Probe for reasons if YES and NO). B. TEACHERS SUPPORT 1. Did you receive any support supervision after the training? From whom? (Probe reasons for YES and NO) 2. Please share with us what kind of support you received? 3. Please share with us what aspects of the support supervision you liked (strengths) and what you did not like (Weaknesses)? 4. Please share with us how often you received the support supervision? Did this support supervision enhance your teaching approach/techniques in the classroom? If yes, how did it help? If no, why was it not useful to you in your classroom practices? 5. Please share with us your thoughts about how the support supervision visits can be improved so that they are more useful for you as a teacher in the classroom. Please mention what can be done (action), who you think should do it (persons responsible), and how that action will be helpful (outcome) in improving the teacher's support C. ABSENTEEISM 1. Sometimes teachers have difficulties coming to class. What are the most common reasons for teachers being absent? a. How frequently would you say it happens? b. Do you think this might affect the pupils’ learning to read? (Why yes, why no) c. What can be done to reduce teacher absenteeism? What could be done (action), who would be responsible for doing it, how will it help reducing absenteeism? 2. How often are pupils absent? What are the reasons? a. Do you think this affects the pupils learning to read? (Why?) 3. Do you ever talk with the parents about pupil absenteeism? If so, does it make a difference? What do parents say? SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 246 a. What could be done to reduce pupils’ absenteeism? Who should do it? D. COMMUNITY MOBILISATION 1. To what extent did the community get involved in early reading initiatives/activities or any type of support? a. Why do you think they got involved? b. Please share with us in what ways the community got involved in promoting early reading or with the School Health and Reading Program c. What did you like about community involvement in the school reading activities / SHRP? 2. Please mention what can be done (action), who you think should do it (persons responsible), and how that action will be helpful (outcome) in improving community mobilization about reading? Do you have anything else to say? Thank you very much for taking time to share with us your views SCHOOL HEALTH AND READING PROGRAM IMPACT EVALUATION REPORT 247 References Bruns, Barbara; Luque, Javier. 2014. Great teachers: how to raise student learning in Latin America and the Caribbean. Washington, DC: World Bank Group. Chay, K. Y., McEwan, P. J., & Urquiola, M. (2005). The central role of noise in evaluating interventions that use test scores to rank schools. American Economic Review, 95, 1237–1258. Glewwe, P., E. Hanushek, S. Humpage, and R. Ravina (2013). School Resources and Educational Outcomes in Developing Countries: A Review of the Literature from 1990 to 2010. In Education Policy in Developing Countries, edited by P. Glewwe, 13–64. Chicago: University of Chicago Press. Lucas, Adrienne M., Patrick J. McEwan, Moses Ngware, and Moses Oketch (2014). Improving early-grade literacy in east Africa: experimental evidence from Kenya and Uganda. Journal of Policy Analysis and Management, Vol. 33, No. 4, 950–976 Menendez, Alicia and Varuni Dayaratna (2016). Uganda: Impact Evaluation of SHRP Reading Program, Year 3. NORC at the University of Chicago Oliveira, Leandro & Martin Carnoy (2015). The Effectiveness of an Early-Grade Literacy Intervention on the Cognitive Achievement of Brazilian Students. Educational Evaluation and Policy Analysis, vol. 37, 4: pp. 567-590. Piper, B. & Korda, M. (2011). EGRA Plus: Liberia, program evaluation report. Research Triangle Park, NC: RTI International.