1 MIDLINE REPORT Impact Evaluation of the Early Grade Reading Activity (EGRA) October 2015 This publication was made possible by the support of the American people through the United States Agency for International Development (USAID). It was reviewed by the USAID/Malawi Education team and prepared by Social Impact, Inc. through Dr. Geetha Nagarajan, Dr. Pedro Carneiro, Melissa Chiappetta, Brian Fuller, Emily Gonzales, Brenda Sinclair, and Aaron Mapondera. ii TABLE OF CONTENTS Acronyms......................................................................................................................................vii Executive Summary....................................................................................................................viii I. Introduction............................................................................................................................ 1 II. Evaluation Purpose and Questions....................................................................................... 2 Purpose................................................................................................................................................................................................. 2 Evaluation Questions ........................................................................................................................................................................ 2 III. Program Implementation ..................................................................................................... 4 Early Grade Reading Activity.......................................................................................................................................................... 4 INVC, a Feed the Future Initiative................................................................................................................................................. 6 SSDI....................................................................................................................................................................................................... 7 Other Early Grade Literacy Programs.......................................................................................................................................... 8 IV. Methodology and Limitations ............................................................................................. 10 Impact Evaluation Overview........................................................................................................................................................10 Treatment Assignment..................................................................................................................................................................11 Sample Size......................................................................................................................................................................................12 Data Collection Instruments........................................................................................................................................................14 Midline Survey Implementation Activities................................................................................................................................15 Limitations.........................................................................................................................................................................................24 V. Sample Characteristics........................................................................................................ 27 School Sample Background Features........................................................................................................................................27 Sample Household and Learner Features...............................................................................................................................39 Characteristics of Sample Teachers Observed at Midline ..................................................................................................43 VI. Midline Findings: Reading Assessment............................................................................... 45 Reading Assessment Findings by Subtasks..............................................................................................................................48 Factors Predicting Learning Scores.............................................................................................................................................58 Effects of Intermediate EGRA Outcomes on Learner Scores.............................................................................................68 EGRA Program Impacts on Learner Scores: Difference in Differences Results............................................................70 VII. Cost Effectiveness of EGRA on Learner Scores................................................................ 75 Unit Costs of EGRA Intervention for 2014-15.......................................................................................................................75 Cost Effectiveness of EGRA..........................................................................................................................................................76 VIII. Midline Findings: Factors Predicting Dropouts and Repetition.................................. 78 Dropouts............................................................................................................................................................................................78 Repetition Rates..............................................................................................................................................................................79 IX. Summary and Conclusions.................................................................................................. 82 X. Recommendations ............................................................................................................... 86 Annexes........................................................................................................................................ 87 Annex 1: Statement of Work......................................................................................................................................................87 Annex 2: A Brief Explanation of the Stages of Learning to Read and the Corresponding Reading Assessment Tasks............................................................................................................................................................................................... 100 Annex 3: Pilot of Two EGRA Tools: Results.......................................................................................................................... 101 Annex 4: Midline Benchmarks................................................................................................................................................. 105 Annex 5. EGRA Program Effects on Oral Reading Fluency: Results of Difference in Difference Estimates (Ordinary Least Squares Estimates) .......................................................................................................................................................... 106 Annex 6: Dropout at Base and Midline, by Treatment Status, Standard and Gender ........................................... 109 Annex 7: Regression Analysis Tables...................................................................................................................................... 110 iii Annex 8: Balance Tables for Baseline.................................................................................................................................... 115 Annex 9: Literature Review....................................................................................................................................................... 118 Annex 10: Early Grade Reading Assessment – Baseline.................................................................................................. 122 Annex 11: Early Grade Reading Assessment – Midline ................................................................................................... 135 Annex 12: Learner Questionnaire........................................................................................................................................... 149 Annex 13: Teacher Questionnaire.......................................................................................................................................... 158 Annex 14: Head Teacher Questionnaire.............................................................................................................................. 170 Annex 15: Classroom Observation Instrument ................................................................................................................... 189 Annex 16: School Climate Instrument................................................................................................................................... 195 Annex 17: Household Survey................................................................................................................................................... 199 iv TABLE OF FIGURES Figure 1: Standard 2 Reading Assessment Scores, by Subtasks at Midline and Baseline ................................ xv Figure 2: Standard 4 Reading Assessment Scores, by Subtasks at Midline and Baseline ................................ xv Figure 3: Average Dropout Rates................................................................................................................................ 30 Figure 4: Average Age of Learners.............................................................................................................................. 31 Figure 5: Has Anyone Added an Hour to School Day?.......................................................................................... 32 Figure 6: Frequency of PTA Meetings......................................................................................................................... 34 Figure 7: Frequency That Teachers Meet with Parents.......................................................................................... 35 Figure 8: What Type of Support Has EGRA Provided?.......................................................................................... 36 Figure 9: Effect of EGRA................................................................................................................................................ 37 Figure 10: Has EGRA Provided You with Coaching?.............................................................................................. 37 Figure 11: Have Other Individuals, Organizations, or Businesses Provided Support?.................................... 38 Figure 12: Other Donor Support ................................................................................................................................ 38 Figure 13: Does Learner Have Access to Reading Materials at Home?............................................................. 39 Figure 14: Do Learners Take Books Home? If Yes, Do They Use Them? ........................................................ 40 Figure 15: Household Member Reads to Learner................................................................................................... 40 Figure 16: Does a Household Member Help with Homework?........................................................................... 41 Figure 17: How Household Member Helps Learner............................................................................................... 42 Figure 18: How Important is it for Learner to Go to School? ............................................................................. 42 Figure 19: Is Household Member Involved in Learner's School?.......................................................................... 43 Figure 20: Teacher Behavior......................................................................................................................................... 44 Figure 21: Percent Zero Scores at Midline................................................................................................................ 48 Figure 22: Percentage of Learners who Take Books Home ................................................................................. 63 Figure 23: School Resources PCA Scores by District ............................................................................................ 66 Figure 24: Average Length of the School Day by District ..................................................................................... 67 Figure 25: Total Average Annual Standard 1-4 Dropouts by District at Midline............................................. 79 v TABLE OF TABLES Table 1: Midline Impact Evaluation Results............................................................................................................... xiv Table 7: Number of Zones, Schools and Learners at Midline (2015) by Treatment Group......................... 14 Table 8: Values to Equate Midline and Baseline EGRA tools................................................................................ 18 Table 9: 2014 Benchmarks for Reading Comprehension, Oral Reading Fluency, Familiar Word Reading and Syllable Reading ........................................................................................................................................................ 19 Table 10: Average Student Enrollment and Teachers in Comparison Schools – Base and Midline Results for Standard 2 and 4. ...................................................................................................................................................... 28 Table 11: Average School Enrollment and Teachers in Treatment Schools - Base and Midline Results for Standard 2 and 4. ............................................................................................................................................................. 28 Table 12: Average School Enrollment and Teachers in All Schools - Base and Midline Results for Standard 2 and 4................................................................................................................................................................................ 29 Table 13: Average Repetition by Standard and Sex - Base and Midline Results for Standard 2 and 4........ 30 Table 14: Percentage of Students Overage for Their Grade ................................................................................ 32 Table 15: Community Involvement............................................................................................................................. 34 Table 16: Percentage of Students with Zero Scores, Baseline and Midline....................................................... 47 Table 17: Listening Comprehension Midline Results Compared to Baseline.................................................... 49 Table 18: Percentage Reaching Benchmarks — Listening Comprehension ...................................................... 50 Table 19: Syllable Segmentation Midline Results Compared to Baseline ........................................................... 50 Table 20: Percentage Reaching Benchmarks — Syllable Segmentation.............................................................. 50 Table 21: Initial Sound Identification Midline Results Compared to Baseline ................................................... 51 Table 22: Percentage Reaching Benchmarks — Initial Sound Identification...................................................... 51 Table 23: Letter Name Recognition Midline Results Compared to Baseline.................................................... 52 Table 24: Percentage Reaching Benchmarks — Letter Name Knowledge........................................................ 53 Table 25: Syllable Reading Midline Results Compared to Baseline...................................................................... 53 Table 26: Percentage Reaching Benchmarks — Syllable Reading......................................................................... 53 Table 27: Familiar Word Reading Midline Results Compared to Baseline ........................................................ 54 Table 28: Percentage Reaching Benchmarks — Familiar Word Reading........................................................... 54 Table 29: Non-Word Reading Midline Results Compared to Baseline.............................................................. 55 Table 30: Percentage Reaching Benchmarks — Non-Word Reading................................................................. 55 Table 31: Oral Reading Fluency Midline Results Compared to Baseline............................................................ 56 Table 32: Percentage Reaching Benchmarks — Oral Reading Fluency .............................................................. 57 Table 33: Reading Comprehension Midline Results Compared to Baseline ..................................................... 57 Table 34: Percentage Meeting Benchmarks — Oral Reading Comprehension................................................ 58 Table 35: Factors Predicting Standard 2 Boys’ Oral Reading Fluency Scores (Measured by correct words read per minute).............................................................................................................................................................. 59 Table 36: Factors Predicting Standard 2 Girls’ Oral Reading Fluency Scores (Measured by correct words read per minute).............................................................................................................................................................. 60 Table 37: Factors Predicting Standard 4 Boys’ Oral Reading Fluency Scores (Measured by correct words read per minute).............................................................................................................................................................. 61 Table 38: Factors Predicting Standard 4 Girls’ Oral Reading Fluency Scores (Measured by correct words read per minute).............................................................................................................................................................. 62 Table 39: Factors Predicting Intermediate EGRA Result (Measured by correct words read per minute) 69 Table 40: EGRA Program Effects on Oral Reading Fluency for Standard 2 Learners: Results of Difference in Difference Estimates, by Gender............................................................................................................................. 71 Table 41: EGRA Program Effects on Oral Reading Fluency for Standard 4 Learners: Results of Difference in Difference Estimates, by Gender............................................................................................................................. 71 Table 42. EGRA Program Effects on Oral Reading Fluency for Standard 2 Learners: Results of Difference in Difference Estimates, by Gender and Treatment Groups (Ordinary Least Squares Estimates).............. 73 vi Table 43. EGRA Program Effects on Oral Reading Fluency for Standard 4 Learners: Results of Difference in Difference Estimates, by Gender and Treatment Groups (Ordinary Least Squares Estimates).............. 74 Table 44: Direct Costs Incurred by RTI for EGRA Implementation during School Year July 2014–June 2015 .............................................................................................................................................................................................. 76 Table 45: Coverage of RTI Intervention in School Year 2014–15 ....................................................................... 76 Table 46: Cost Effectiveness of EGRA per Learner for Unit Change in Oral Reading Fluency.................... 76 Table 47: Factors that Predict Total Annual Standard 1-4 Dropouts (at the school level)........................... 78 Table 48: Factors Predicting Whether Learners Repeat a Standard (Logistic Regression)........................... 80 vii ACRONYMS CCM Community Case Management CDCS Country Development Cooperation Strategy CLPM Correct Letters Per Minute CNWPM Correct Non-Words Per Minute CSPM Correct Syllables Per Minute CWPM Correct Words Per Minute DAI Development Alternatives, Incorporated DiD Difference-in-Differences EGRA Early Grade Reading Activity EP Evaluation Policy FEWEMA Forum for African Women Educationists in Malawi FtF Feed the Future FUM Farmers Union of Malawi GHI Global Health Initiative GoM Government of Malawi HHS Household Survey HIV Human Immunodeficiency Virus HSA Health Surveillance Assistants ICC Inter-Class Correlation IE Impact Evaluation IP Implementing Partner IKI Invest in Knowledge, Incorporated INVC Integrating Nutrition in Value Chains MDG Millennium Development Goal MDES Minimum Detectable Effect Size MoEST Ministry of Education, Science, and Technology MOH Ministry of Health MOU Memorandum of Understanding MTPDS Malawi Teacher Professional Development Support NASFAM National Smallholder Farmers’ Association of Malawi NRA National Reading Assessment OLS Ordinary Least Squares ORPM Oral Reading Per Minute PCA Principal Component Analysis PMTCT Prevention of Mother-to-Child Transmission PTA Parent Teacher Association RA Reading Assessment RTI Research Triangle Institute, International SI Social Impact, Incorporated SMC School Management Committee SSDI Support for Service Delivery Integration SOW Statement of Work UNICEF United Nations Children’s Fund US United States USAID United States Agency for International Development viii EXECUTIVE SUMMARY USAID and the Ministry of Education, Science, and Technology (MoEST) in Malawi are working to improve the quality of education through the Early Grade Reading Activity (EGRA) and other activities. The Early Grade Reading Activity (EGRA), awarded to RTI (Research Triangle Institute, International) in July 2013, is a multifaceted educational development approach intended to enable sustained literacy among children, promote a literate community, and help the country of Malawi improve its economic growth and reduce poverty. RTI began implementing EGRA in August 2013. In April 2013, USAID/Malawi contracted with Social Impact (SI), an Arlington, Virginia-based development consulting firm to conduct four tasks: Task 1. A baseline assessment for an impact evaluation (IE) of the USAID/Malawi EGRA Project on addressing learner reading outcomes in Standards 2 and 4. Task 2. A survey of the households of the Standards 2 and 4 learners selected for the IE sample. Task 3. A national reading assessment (NRA) of Standards 1 and 3 learners.1 Task 4. A final IE of the USAID/Malawi EGRA Project and the Country Development Cooperation Strategy (CDCS) hypothesis that learner reading outcomes will improve even more from the implementation of EGRA combined with the Integrating Nutrition in Value Chains (INVC) Activity, a Feed the Future (FtF) Activity, and Support for Service Delivery Integration Services (SSDI) Activity than they will from EGRA alone. Together, USAID/Malawi expects Tasks 1, 2, and 4 to provide a rich, multifaceted IE of EGRA. USAID and SI designed data collection for the IE to be carried out in three rounds: a 2013 baseline, a 2015 midline, and a 2017 endline. The IE will allow USAID and the Government of Malawi (GoM) (as well as other donors) to track progress toward improved quality of education and improved success in meeting early grade reading benchmarks. In addition, the IE will help track the progress of EGRA alongside other complementary USAID projects such as INVC and SSDI. The evaluation findings will also help inform school- and policy-level decisions through individualized school report cards and district-specific policy briefs. At the midline in 2015, the evaluation team addressed the following: • For Tasks 1 and 2: What proportion of Standards 2 and 4 learners have attained MoEST-established learning benchmarks; and what are the household, school, and other predictors of learning scores? Do they differ by gender and treatment status? • For Task 4: Have learning scores for Standards 2 and 4 changed since the EGRA intervention, and what is the EGRA program effect? What other predictable factors besides EGRA treatment affect changes in outcomes (i.e. other USAID activities in agriculture and health)? • For Task 4: What is the cost effectiveness of the EGRA intervention? In addition, the evaluation team provides midline data for the indicators below for use by USAID and MoEST: • Proportion of learners receiving one hour extra time-on-task reading instruction per day • Proportion of learners who take home and use a book or other reading materials at home • Proportion of teachers receiving at least one coaching/support visit per term from anyone (a head teacher, Primary Education Advisor, District Education Manager, etc.) 1 NRAs for Task 3 are carried out in 2014 and 2016, and the purpose is to allow MoEST, USAID, and other stakeholders to monitor education in Malawi over time using nationally representative data. Because the NRA data are nationally representative, they are not comparable with the IE data, which only examines 10 of Malawi’s 28 districts. ix • Proportion of teachers demonstrating “essential” skills in teaching reading IMPACT EVALUATION METHODOLOGY The IE is designed as an experiment and is being implemented in and evaluated for four distinct treatment levels: Treatment Level 1. Zones from three focus districts that provide an opportunity to evaluate the impact of a fully-integrated development approach, with multiple projects across sectors, including EGRA, SSDI, and INVC. Treatment Level 2. The district where EGRA overlaps only with USAID’s health intervention (SSDI). This serves as a test ground for the development hypothesis that synergies between education and health initiatives catalyze changes that are greater than the sum of their parts. Treatment Level 3. Zones from the district where EGRA overlaps only with the INVC intervention. This serves as a test ground for the development hypothesis that synergies between education and agricultural livelihood and nutrition initiatives catalyze changes that are greater than the sum of their parts. Treatment Level 4. Zones from districts that only receive the EGRA initiative. These districts are used to test the EGRA theory of change that educational support leads to improved literacy and general education outcomes. In order to infer impacts, SI designed the evaluation to compare learner reading scores from each of these four groups with a comparison group of schools and zones from the same districts to determine the effectiveness of each type of treatment. By design, the evaluation team followed the same schools longitudinally at baseline and midline, but not the same set of learners longitudinally, due to USAID’s request that the evaluation assess the same standards — Standards 2 and 4 — at baseline, midline and endline. SI partnered with the MoEST and Invest in Knowledge, Incorporated (IKI), a Malawian data collection firm, to gather data for the evaluation. MAJOR FINDINGS AND CONCLUSIONS Summary of results at midline are shown in Table 1, Figure 1 and Figure 2 and are discussed below. AT MIDLINE, WHAT PROPORTION OF STANDARDS 2 AND 4 LEARNERS HAVE ATTAINED MOEST ESTABLISHED BENCHMARKS FOR READING SKILLS? At midline, by the end of Standard 2, about one percent of learners were able to read grade-level text, and zero percent of learners were able to read with comprehension according to the MoEST benchmark established in 2014. The results were similar in both comparison and treatment groups in the proportion attaining the benchmarks.2 In Standard 4 treatment schools at midline, nearly 8 percent of learners met the Oral Reading Fluency benchmark, and 5 percent of learners were able to read with comprehension. But, in comparison schools, it was 7 and 4 percent of learners attaining Oral Reading Fluency and Reading Comprehension benchmarks, respectively. 2 It is to be noted that these benchmarks are set to be reached in five years (in 2018; year five of EGRA implementation). At midline, EGRA has been implemented only for two years. SI did not apply any prorated benchmarks for 2015 since no intermediate targets or benchmarks were provided in the MoEST benchmarks issued in Dec. 2014. SI considers that approximate allocation of benchmark results by years using the final benchmark assumes a linear change that may not occur in reality, as shown in some EGRA evaluations. Therefore, benchmarking midline learner scores for these tasks at five year benchmark level could likely underestimate the progress made at midline. x Baseline results in 2013, however, were slightly higher for Oral Reading Fluency in that 1.2 percent of learners in Standard 2 and 10 percent of learners in Standard 4 reached the benchmark. But, baseline results were similar in both standards for proportion of learners attaining the 2014 benchmark for Reading Comprehension (0 percent in Standard 2 and 5 percent in Standard 4). At midline, both the comparison and treatment groups fell below the benchmarks across all nine subtasks under the pre-reading, initial reading, and reading fluency and comprehension skills, although the treatment group performed slightly better than the comparison group. Among pre-reading subtasks, benchmarks were met by more than half the learners in both standards in listening comprehension, but, by less than five percent on the initial sound identification subtask. On all initial reading subtasks except letter name knowledge, less than ten percent of learners attained the benchmarks in both standards suggesting a lack of pre-reading and initial reading skills, such as phonemic awareness and decoding. Since learners were able to read a similar number of words on the Familiar Word Reading subtask and the Oral Reading Fluency subtask (about three words and nine words per minute, respectively, among Standard 2 and 4 learners), the results implied that they can read sight words from memory; however, they are not yet able to decode words in isolation or infer meaning from connected text in a simple reading passage. HAVE LEARNING SCORES FOR STANDARDS 2 AND 4 CHANGED SINCE THE EGRA INTERVENTION, AND WHAT IS THE EGRA PROGRAM EFFECT? When midline average scores were compared between treatment and comparison schools, performance across nearly all subtasks and standards was higher in treatment than in comparison schools. When midline average scores were compared with baseline, however, performance across nearly all subtasks and standards dropped considerably in both treatment and comparison groups of schools. The exception to this trend was in the syllable segmentation and syllable reading subtasks, in which the treatment group slightly improved from the baseline to midline, while the comparison group showed a decrease in performance. Regression results using Difference in Differences (DiD) approach indicated that, overall, learners in EGRA treatment schools in both Standards 2 and 4 have shown improvements in oral reading fluency relative to comparison schools at midline, albeit they were less than one cwpm and were not significant. There were differences noticed by gender in that girls appear to perform better than boys in both standards. The program effects were better for Standard 2 than for Standard 4 (0.99 cwpm in Standard 2; 0.84 cwpm in Standard 4) likely due to EGRA’s explicit focus on this during the first year of the intervention (2013-2014 academic year), focusing on Standard 1 learners and teachers, some support for Standard 1-3 teachers, and again more intense support to Standard 2 teachers and provision of materials specifically for Standard 2 learners in the 2014-2015 academic year. Most Standard 4 learners were not exposed to EGRA with the exception that Standard 3 teachers received one five-day training in 2013, and learners could have likely received the benefits of the extended school day and reading lessons, reduced class sizes, more parental and community involvement in the schools, and at least two reading fairs. Also, some Standard 4 teachers might have been more motivated due to the distribution of the grants under contract in the 2014-2015 academic year. But, notable impacts have not yet occurred in terms of positive and larger effect sizes due to the decline in midline scores from baseline, although the reduction has been lower in treatment schools than in comparison schools. WHAT ARE THE HOUSEHOLD, SCHOOL, AND OTHER PREDICTORS OF LEARNING SCORES AT MIDLINE? DO THEY DIFFER BY GENDER AND TREATMENT STATUS? The evaluation team found numerous predictors of oral reading fluency scores at midline. These factors do differ by sex and standard, as described in greater detail in the body of the report. For Standard 2, xi factors such as learners reporting that they take books home from school, are being read to at home, and getting tired at school were found to be significantly correlated with oral reading fluency scores. Also, learner-to-teacher ratio, teachers reporting adequate teaching resources, and overage learners were correlated with reading scores. For Standard 4, learners reporting that they take books home from school appears to be significantly correlated with midline reading scores. Additionally, overage learners, school resources, and essential teaching skills were correlated with reading scores. All the above factors were similar to those found as predictors of learning scores at baseline. There were two factors, however, that were found to be correlated only at midline for both standards: length of school day and speaking Chichewa either at home or with friends. Further, at baseline, no notable correlations between a school feeding program and learner outcomes were found, though in the 2014 NRA it was found to be significantly and positively correlated with Standard 1 learner oral reading fluency scores; at midline, the team found that whether a school had a school-feeding program predicted statistically significant results for Standard 4 only. The results appear to indicate that the declining trend in scores could plausibly be linked to factors such as an increase from baseline to midline in learner-to-teacher ratio, and decreases from base to midline in learners receiving help with their homework from a household member and in households encouraging the child to read, although there were improvements in teacher practices and length of school day at lower standards.3 DO OTHER USAID ACTIVITIES IN AGRICULTURE AND HEALTH BESIDES EGRA TREATMENT AFFECT CHANGES IN LEARNER SCORES? Treatment schools in areas that received just the EGRA treatment (Level 4) had the highest reading outcomes as shown by analysis of only the midline scores and through DiD that compared changes in average scores between base and midline across treatment and comparison schools. The results indicated that EGRA has had a clear program effect in EGRA-only areas. EGRA effects in Treatment Level 1 - EGRA + SSDI + INVC, USAID/Malawi’s Country Development Cooperation Strategy (CDCS) focus area - were mixed. Analysis of midline scores of learners in treatment and comparison schools showed negative correlations across standards and gender, and the correlation was statistically significant for Standard 4 girls. However, analysis using DiD that compared changes in average scores between base and midline in treatment and comparison panel schools showed that Level 1 treatment schools were more likely to have higher reading scores than Level 1 comparison schools, especially for Standard 2 learners where results were significant. The difference in magnitude of changes in average scores from base to midline in comparison and treatment schools likely explain the difference in results seen between the two methods of analysis. Interestingly, DiD results also indicated differences in effect sizes between boys and girls in Level 1in that better effect sizes were noted for girls relative to boys. This requires further inquiry to understand reasons behind the trend since the study areas are located within the USAID/Malawi’s CDCS focus districts. It is likely that INVC and SSDI programs in addition to EGRA were more favorable to girls than boys in treatment schools located in Level 1. Analysis of midline learner scores showed that Treatment Level 2 - EGRA + SSDI - was consistently correlated with higher predicted oral reading fluency scores, and those differences were statistically significant for girls in both Standards 2 and 4. However, DiD analysis for Standard 2 showed that Level 2 treatment was associated with higher but insignificant average change in scores for boys, but lower and 3 Across base and midline, the samples were balanced and tools used for assessment were equated to be similar. Further, no changes were made in the curriculum or language of instruction. Thus, these factors can be ruled out as reasons for the observed trend. xii significant change for girls. But, the results were reversed in Standard 4 where Level 2 treatment, although results were insignificant, was associated with lower average change in scores for boys, but higher change in scores for girls. Analysis of midline learner scores showed that Treatment Level 3 - EGRA + INVC - was correlated with higher predicted learner reading scores for Standard 2 boys and Standard 4 girls, and results were statistically significant for Standard 4 girls. Results were negative but insignificant for Standard 2 girls and Standard 4 boys. While DiD results were similar for Standard 2 girls, they were slightly different for others. While none of the DiD results were significant, effects were positive for boys in Standard 2, and were positive for both genders in Standard 4. WHAT ARE THE INTERMEDIATE EFFECTS OF EGRA ON LEARNER SCORES? The number of coaching visits per term appears to be negatively correlated with oral reading fluency scores and may indicate a diminishing effect. SI will explore this further if this result continues at endline. Also, the EGRA MOUs appear to have been very successful, with the exception of the MOU that encouraged parents to read to learners. The latter may have been less successful because the MOU is fairly new (having just been signed in the 2013-2014 academic year for Cohort A schools and the 2014- 2015 academic year for Cohort B schools). Further, it likely targeted parents who were not already reading to their learners; thus, based on earlier findings, these learners were probably scoring lower on reading tests prior to the 2013-2014 academic year. By 2017, this MOU may likely prove to have been beneficial to learner reading scores. On the other hand, the MOU that worked to reduce class size appears to have been very successful, improving reading scores by an average of 4.4 cwpm. The MOUs to extend the school day and the length of the reading lesson also appear to have had a large effect, increasing learner reading scores by an average of 2.9 cwpm and 2.1 cwpm, respectively. Finally, the number of reading fairs the school hosted in the past two years also appears to be a good predictor of learning reading scores, with each additional fair increasing scores by almost 1 cwpm. WHAT IS THE COST EFFECTIVENESS OF THE EGRA INTERVENTION ON LEARNER SCORES? Using the program effects calculated by SI using DiD and total direct costs obtained from RTI for year 2014-15, the evaluation team calculated the cost effectiveness of the EGRA intervention. The results showed that it would cost around $6.10 per learner in Standard 2 to improve by one unit in oral reading fluency in correct words per minute. For Standard 4, based on the share of costs approximately allocated by SI to have been incurred by RTI in 2014 academic year, it would cost about $1.80 per learner to improve by one unit in oral reading fluency in correct words per minute. The cost effectiveness estimates, however, should be interpreted with caution, especially for scaling up, because it is early to measure impacts since intervention focus standards may change in coming years, and because direct costs were aggregated under all components (excluding labor) implemented by RTI and were approximately allocated across the two standards. At this stage, the estimates only provide some insights into cost effectiveness at early stages and also of phased implementation of EGRA by standard. As the project matures and is expanded to more standards, economies of scale and scope may occur, leading to reduction in some costs, and also effects may improve, thus altering the cost effectiveness estimates of EGRA. WHAT ARE THE FACTORS PREDICTING DROPOUTS AND REPETITION AT MIDLINE? Factors predicting dropouts at midline included average household wealth, household head’s level of education, learner attendance at preschool, teacher use of best practices, and number of school reading fairs. They were all negatively correlated with learner dropouts meaning these factors led to a reduced xiii number of dropouts, though household wealth was the only variable that had a statistically significant correlation. Also, the average learner-to-teacher ratio for sampled classrooms predicted an increase in dropouts, meaning the more learners in a class, the more dropout. None of the four treatment levels were statistically significantly correlated with dropouts. Student repetition was found to be correlated with learner access to reading materials at home, household wealth, highest level of household head’s education, whether the learner attended preschool, whether the learner was in Treatment Level 2 (learners from Treatment Level 2 were 22 percent less likely to be repeating a standard at midline), and whether the learner was in Treatment Level 1 (learners from Treatment Level 1 were 48 percent more likely to be repeating a standard this year). RECOMMENDATIONS Based on the above findings and conclusions, the study recommends USAID and MoEST do the following: • Build up community programs that work to get parents and household members involved in learner reading and ensure that these programs encourage households to read to learners and explain the benefits of doing so. • Consider other ways of ensuring learners are read to more often, possibly by creating after￾school peer-mentoring programs. This method has been tried in many other education interventions and proved beneficial both for the mentors and mentees. • Work with schools to ensure they have enough textbooks or a system of protecting textbooks to allow learners to take books home from school with them, and encourage learners to do so—possibly through reading incentive programs such as those often used in the U.S., which provide small rewards for learners who read multiple books over school break periods (or even throughout the academic year). • Continue to work with teachers through targeted capacity-building and coaching interventions to improve teacher use of “essential” reading practices. • Train additional teachers and identify additional resources to allow schools to reduce the average learner-to-teacher ratio. This might also be accomplished by simply not pairing teachers together, but instead having them teach their own classes or expanding EGRA MOUs to other schools. • Work with RTI to ensure all EGRA schools actually adopt the provisions of the MOUs. This means ensuring all standards are extended by an hour—not just the lower standards. This may require USAID and the MoEST working together to discuss the larger policy implications of this extended day in terms of financial costs for keeping teachers at schools longer. This also means ensuring more schools sign the MOU to reduce class sizes or split up classes between more teachers. • Identify ways to better integrate EGRA, SSDI, and INVC activities. • It is also important for EGRA and USAID/Malawi to examine the reasons behind the declining trend in learner scores and find ways to stop the decline in treatment schools in both standards, and also further increase learner skills from midline. Also, the differences in scores by gender need further inquiry to confirm the results and understand how and why EGRA activities contribute to such differences. xiv Table 1: Midline Impact Evaluation Results Reading Subtask Standard 2 Standard 4 Mean Score Percent of Learners Reaching Proposed Benchmark (%) Percent of Learners Scoring Zero (%) Mean Score Percent of Learners Reaching Proposed Benchmark (%) Percent of Learners Scoring Zero (%) T C T C T C T C T C T C Listening Comprehension 60% 53% 58 54 8.4 11.9 77% 73% 56 52 2.1 2.6 Syllable Segmentation 51% 44% 47 43 30.9 38.8 55% 63% 55 49 13.7 16.4 Initial Sound Identification 13% 6% 4 3 64.5 78.5 13% 6% 5 3 58.1 71.0 Letter Name Knowledge 11.4 clpm 6.1 clpm 15 9 33.2 48.1 30.2 clpm 27.8 clpm 26 21 8.1 9.7 Syllable Reading 6.0 cspm 3.1 cspm 1 0 66.0 77.9 29.7 cspm 27.1 cspm 10 9 16.4 21.0 Familiar Word Reading 3.7 cwpm 2.0 cwpm 1 1 69.6 79.9 23.1 cwpm 20.9 cwpm 11 9 15.6 21.5 Non-Word Reading 2.6 cwpm 1.4 cwpm 7 3 74.0 83.5 14.1 cwpm 12.9 cwpm 2 2 19.3 25.4 Reading Fluency 3.5 cwpm 1.8 cwpm 1 1 73.6 84.4 22.3 cwpm 20.3 cwpm 8 7 19.3 25.7 Reading Comprehension 1% 1% 0 0 95.6 98.4 14% 12% 5 4 55.4 62.8 Indicators Percent Comparison Treatment Proportion of learners in Standard 2 receiving extra 1-hour time-on-task reading instruction per day 32% 97% Proportion of learners in Standard 4 receiving extra 1-hour time-on-task reading instruction per day 36% 97% Proportion of learners in targeted grades that take home and use a book or other reading materials 43% 59% Proportion of schools receiving at least one coaching/support visit per term 78% 96% Proportion of teachers demonstrating “essential” skills in teaching reading 47% 42% Head Teacher Reported Basic School Statistics Average Comparison Treatment Enrollment in Primary Schools (1-4 standards) 198.6 220.8 Number of Learners per Teacher in Standards 2 and 4 112.2 110.4 Number of teachers per Standard (Standards 1-4) 6.8 7.7 Length of School Day for Standard 2 4.5 hours 4.5 hours Length of School Day for Standard 4 5.5 hours 5.5 hours Drop-out Rate in Standard 2: Girls 8.9 8.4 Drop-out Rate in Standard 2: Boys 8.8 8.2 Drop-out Rate in Standard 4: Girls 6.0 6.2 Drop-out Rate in Standard 4: Boys 5.9 6.2 Repeat Rate in Standard 2 23.6 22.6 Repeat Rate in Standard 4 16.7 16.8 Number of Years of Experience as Head Teacher 16.4 7.6 Note: ‘T’ refers to Treatment schools and ‘C’ refers to Comparison schools. xv Figure 1: Standard 2 Reading Assessment Scores (Mean) by Subtasks at Midline and Baseline Figure 2: Standard 4 Reading Assessment Scores (Mean) by Subtasks at Midline and Baseline 0 5 10 15 Listening comprehension Syllable segmentation Initial sound identification Letter naming Syllable reading Familiar word reading Non-word reading Oral reading fluency Reading comprehension Baseline Midline comparison Midline treatment 0 5 10 15 20 25 30 35 40 Listening comprehension Syllable segmentation Initial sound identification Letter naming Syllable reading Familiar word reading Non-word reading Oral reading fluency Reading comprehension Baseline Midline comparison Midline treatment 1 I. INTRODUCTION USAID and the Ministry of Education, Science, and Technology (MoEST) in Malawi are working to improve the quality of education through the Early Grade Reading Activity (EGRA) and other activities. EGRA, awarded to RTI (Research Triangle Institute, International) in July 2013, is a multifaceted educational development approach intended to enable sustained literacy among children, promote a literate community, and help the country of Malawi improve economic growth and reduce poverty. RTI began implementing EGRA activities in August 2013. In April 2013, USAID/Malawi contracted with Social Impact (SI), an Arlington, Virginia-based development consulting firm, to conduct four tasks: Task 1. A baseline assessment for an impact evaluation (IE) of the USAID/Malawi EGRA Project on addressing learner reading outcomes in Standards 2 and 4. Task 2. A survey of the households of the Standards 2 and 4 learners selected for the IE sample. Task 3. A national reading assessment (NRA) of Standards 1 and 3 learners.4 Task 4. A final IE of the USAID/Malawi EGRA Project and the Country Development Cooperation Strategy (CDCS) hypothesis that learner reading outcomes will improve even more from the implementation of EGRA combined with the Integrating Nutrition in Value Chains (INVC) intervention (a Feed the Future (FtF) Activity) and Support for Service Delivery Integration Services (SSDI) than they will from EGRA alone. Together, USAID/Malawi expect Tasks 1, 2, and 4 to provide a rich, multifaceted IE of EGRA. SI and USAID designed data collection for the IE to be carried out in three rounds: a 2013 baseline, a 2015 midline, and a 2017 endline. The IE will allow USAID and the Government of Malawi (GoM) (as well as other donors) to track progress toward improved quality of education and improved success in meeting early grade reading benchmarks. In addition, the IE will help track the progress of EGRA alongside complementary USAID projects such as INVC and SSDI. The evaluation findings will also help inform school- and policy-level decisions through individualized school report cards and district-specific policy briefs. At midline in 2015, we addressed the following: • For Tasks 1 and 2: What proportion of Standards 2 and 4 learners have attained MoEST-established benchmarks; and what are the household, school, and other predictors of learning scores? Do they differ by gender and treatment status? • For Task 4: Have learning scores for Standards 2 and 4 changed since the EGRA intervention, and what is the EGRA program effect? What other predictable factors besides EGRA treatment affect changes in outcomes (for example, other USAID activities in agriculture and health in the areas studied)? • For Task 4: What is the cost effectiveness of the EGRA intervention? 4 NRAs for Task 3 are carried out in 2014 and 2016, provide valuable data on learner reading performance for all stakeholders, and contribute to the trend within Malawi’s education system toward greater accountability and evidence-based decision making. The purpose of the NRAs is to allow MoEST, USAID, and other stakeholders to monitor education in Malawi over time using nationally representative data. The results from both the IE and the NRA will be used by all of these parties to improve the effectiveness and efficiency of future education interventions. 2 II. EVALUATION PURPOSE AND QUESTIONS PURPOSE The purpose of the IE is to provide USAID/Malawi and the MoEST with information to inform: 1) improvements to EGRA, 2) the possible scale-up of EGRA and other USAID interventions, 3) the types of interventions that are the most cost-effective at increasing learner reading outcomes in Malawi. In order to do this, this evaluation seeks: 1. To measure the effect of EGRA on student reading outcomes (versus a comparison group); and 2. To test the hypothesis that integrating USAID interventions in education, agriculture, health, and community strengthening in the same communities results in increased student learning, as described in the USAID/Malawi CDCS. This includes measuring: How integration of USAID programing across sectors (education, health, and agriculture) in the same geographic areas impacts student reading outcomes. See Annex 1 for the full scope of work for the IE across five years. EVALUATION QUESTIONS Evaluation questions are presented here by task. TASK 1. EVALUATION OF THE EFFECTS OF EGRA ON STANDARDS 2 AND 4 LEARNER READING OUTCOMES 1. What proportion of primary school learners are able to read with comprehension, according to Malawi’s curricular goals, by the end of lower primary school (Standard 4)? a. What is the proportion of learners who by the end of Standard 4 are able to read standard-level text, as measured by the number of correct words per minute (cwpm)? b. What is the proportion of learners who by the end of Standard 4 are able to answer comprehension questions after reading standard-level text, as measured by the number of comprehension questions answered correctly? 2. What is the proportion of learners who by the end of Standard 2 demonstrate that they can read and understand the meaning of standard-level text? a. What is the proportion of learners who by the end of Standard 2 are able to read standard-level text, as measured by the number cwpm? b. What is the proportion of learners who by the end of Standard 2 are able to answer comprehension questions after reading standard-level text, as measured by the number of comprehension questions answered correctly? TASK 2. HOUSEHOLD SURVEY OF STANDARDS 2 AND 4 LEARNERS 1. What household and community factors correlate with learner reading outcomes? 2. What level of household and community resources is dedicated to schooling overall and reading specifically? 3. How have health and agricultural interventions at the household and community levels influenced schooling and reading outcomes? 4. What factors at the household and community levels correlate with changes in the rates of learner standard repetition and early departure from school, and are girls and boys treated differently in the household in ways that influence academic achievement? 3 TASK 4. FINAL IMPACT EVALUATION OF EGRA AND CDCS HYPOTHESES 1. What is EGRA’s impact on children’s reading abilities (disaggregated by sex) in terms of the following: a. Impact of level of effort of reading instructor on learners’ reading abilities? b. Effect of extracurricular reading activities? c. Effect of time-on-task on improving reading outcomes? 2. Which components have the largest effect, and what is the relative cost effectiveness of these various components? 3. How do teachers’ classroom behavior and practices impact the ability of children to read? a. How did the level of coaching impact teacher behavior and learner reading outcomes? 4. How does the level of integration with INVC and SSDI and other related development program interventions in the target districts impact the reading outcomes of learners? 5. What are the secondary effects that can be attributed to EGRA for the following: a. Impact on repetition rate? b. Impact on dropout rate? c. Impact on school completion, particularly for girls and learners with disabilities? Social Impact also provides the data at midline for the indicators below for use by USAID and MoEST: • Proportion of learners receiving extra one hour time-on-task reading instruction per day • Proportion of learners who take home and use a book or other reading materials at home • Proportion of teachers receiving at least one coaching/support visit per term from anyone (a head teacher, Primary Education Advisor, District Education Manager, etc.) • Proportion of teachers demonstrating “essential” skills in teaching reading 4 III. PROGRAM IMPLEMENTATION EARLY GRADE READING ACTIVITY EGRA derives from a strong baseline of educational research that has revealed the importance of developing learners’ fluent reading abilities by the end of Standard 3.5 The research showed that learners who are not fluent readers by that time are unlikely to ever catch up, not only in reading but also in all other learning areas that require reading facility. Several factors are critical to ensuring that all learners develop reading fluency in Malawi, and EGRA is designed to address each of these factors, including teacher training, reading instruction, parental and community involvement, and political perceptions related to education policies. Specifically, the USAID/Malawi EGRA Project objectives are to: • Improve the capacity of Standards 1 to 3 teachers to provide quality reading instruction to learners • Improve the learning outcomes of Standards 1 to 3 learners • Increase parental and community engagement to support learner reading • Reduce repetition and dropout rates in the early grades by providing a quality learning environment To accomplish these objectives, USAID/Malawi awarded a contract to RTI International in July 2013. RTI began implementing EGRA in August 2013. Since then, according to RTI monitoring data and staff, the following activities have been undertaken under the activity: • Standard 1 Chichewa reading books and learner booklets for all Standard 1-3 classrooms, with the idea that higher standards could use the books to remediate weaker readers • One training in 2013 or 2014 (depending on the cohort6) for teachers from Standards 1-3 • Two additional teacher trainings and practicums on Chichewa reading lessons for Standards 1 and 2 teachers only • Two additional teacher trainings and practicums on English reading lessons for Standard 1 teachers • Chichewa scripted lesson plans provided to Standards 1-2 teachers, and English scripted lesson plans provided to Standard 1 teachers • In-service teacher support and mentoring (or coaching) for Standards 1-2 teachers (Chichewa and English for Standard 1 and Chichewa-only for Standard 2) • Rewards (Grants Under Contract) for high-performing teachers and schools (these were distributed in the 2014-2015 academic year) • Development and distribution of books, story cards, and letter cards for Standards 1 and 2 classrooms • Encouragement of schools to hold reading fairs and other events at least once per school term to engage parents, caregivers, and the community in learners’ reading • Invitations for parents to participate in their learners’ classrooms and/or become engaged in extra￾curricular activities 5 USAID Malawi (2010), “Early Grade Reading Assessment: National Baseline Report,” www.eddataglobal.org/reading/index.cfm/Malawi%20National%20Baseline%20EGRA%202010.pdf?fuseaction=throwpub&ID=354. USAID Malawi (June 2012), “Malawi National Early Grade Reading Midterm Assessment, 2011.” 6 EGRA is being implemented in two cohorts—one cohort of 275 schools across all levels that began receiving benefits in the 2013-2014 academic year (Cohort A) and one cohort of 45 schools from Level 4 only that began receiving benefits in the 2014- 2015 academic year (Cohort B). 5 • EGRA staff attendance at the Basic, Standards, and Teacher Education Technical Working Groups and signed Memoranda of Understanding (MOUs) between EGRA and MoEST to help ensure a supportive policy environment and gain MoEST support and buy-in • MOUs with community leaders to encourage local ownership and foster local efforts promoting reading • MOUs with schools to extend Chichewa reading classes and school instructional time by one hour • MOUs with schools to reduce class size (or split large classes into two streams) • MOUs with parents, for them to commit to reading to learners • Special Group trainings to promote community support of the school, which included theater for development activities and sensitization meetings with traditional authorities and other community leadership structures, among other things. These activities were implemented under the schedule discussed below: EGRA IMPLEMENTATION SCHEDULE Since inception in 2013, RTI has rolled out various EGRA components in treatment schools according to a schedule based on learner standard and treatment cohort.7 Table shows a simplified overview of the training and materials provided for each academic year, according to RTI staff. In 2013–14, RTI trained all Cohort A Standards 1-3 teachers on Standard 1 EGRA methodologies in a five-day training session at the beginning of the school year in August or November (some zones were added slightly later). Cohort A Standard 1 teachers received 2 further five-day trainings on the EGRA methodologies and use of the new EGRA materials in Chichewa in December 2013 and April 2014. The same teachers (Cohort A, Standards 1-3) also received Standard 1 reading textbooks and readers for their learners in 2013. The following academic year, 2014–15, RTI emphasized Chichewa reading training for Standard 2 teachers and distributed corresponding Standard 2 materials to all Cohort A schools. They also emphasized English reading training for all Standard 1 teachers and distributed corresponding materials to all Cohort A schools. Then, in August of the 2014-2015 academic year, Standards 1-3 teachers from both Cohort A and Cohort B received a seven-day Standard 1 English reading training; Standard 2 teachers from Cohort A received a seven-day Chichewa reading training; and Standards 1-3 teachers from Cohort B received a seven-day training on Standard 1 Chichewa reading. All of the same teachers received follow-up trainings on the same topics for five days in December 2014 and three days in April 2015. Table 2: General Schedule of Chichewa Language Training Emphasis and Materials Rollout COHORT ACADEMIC YEAR 2013–14 2014–15 2015–16 A Stds 1-3 Teachers Trained in Standard 1 reading Std 2 Teachers Trained in Standard 2 reading Std 3 Teachers Trained in Standard 3 reading Std 1 Textbooks and Readers distributed Std 2 Textbooks and Readers distributed Std 3 Textbooks and Readers distributed B Stds 1-3 Teachers Trained in Standard 1 reading Stds 2 & 3 Teachers Trained Std 1 Textbooks and Readers distributed Stds 2 & 3 Textbooks and Readers distributed 7 Some schools (45 total) originally categorized as comparison schools in 2013 were rolled into treatment in 2014 at the request of USAID, MoEST, and RTI. Those schools originally designated as treatment in 2013 are called Cohort A, and those that shifted from comparison to treatment in 2014 are called Cohort B. More details are available in the Methodology Section. 6 Table shows only the Chichewa-language-based activities. RTI is following a similar schedule for Cohorts A and B in English language instruction and materials, exactly one year behind rollout of the Chichewa activities in Cohort A schools (in other words, Cohort B schools get the Chichewa and English trainings at the same time). Accordingly, Cohort A and B Standards 1-3 teachers received Standard 1 English language materials and training during the 2014–15 academic year, and Standards 2 and 3 teachers will receive the Standard 2 and 3 English language materials and training during the 2015-2016 academic year. In addition to training and materials, RTI signed MOUs with schools, parents, and community leaders to encourage local ownership and foster local efforts to promote reading (as described above). RTI finalized MOUs with Cohort A schools, parents, and communities during the 2013–14 academic year and Cohort B communities during the 2014–15 academic year. According to RTI monitoring data shared with SI in October 2015, more than 90 percent of EGRA communities reported holding reading fairs in the 2014-2015 academic year across both cohorts. In 98 percent of communities, residents, including adults, reported engaging in at least some community-level reading activities, which included mentoring, book lending, book duplication, storytelling, and working on reading skills. Since this assessment examines scores for Standard 2 and 4 learners, readers can expect that: • Cohort A Standard 2 learners were exposed to EGRA materials, lesson plans, and trained teachers for two years in Chichewa and one year in English. They also received the benefits of two years of the extended school day and reading lessons, reduced class sizes, more parental and community involvement in the schools, and at least two reading fairs. Their teachers might have also been more motivated due to the distribution of the grants under contract in the 2014-2015 academic year. • Cohort B Standard 2 learners were exposed to EGRA materials, lesson plans, and trained teachers for one year for both Chichewa and English reading lessons. They also received the benefits of one year of the extended school day and reading lessons, reduced class sizes, more parental and community involvement in the schools, and at least one reading fair. Their teachers might have also been more motivated due to the distribution of the grants under contract in the 2014-2015 academic year. • Cohort A Standard 4 learners were not likely exposed to EGRA materials, lesson plans, or trained teachers at all, with the exception of the fact that their Standard 3 teachers did receive one, five-day training in 2013 on Standard 1 reading and lessons, and also received those Standard 1 materials. But, they could have likely received the benefits of two years of the extended school day and reading lessons, reduced class sizes, more parental and community involvement in the schools, and at least two reading fairs. Their teachers might have also been more motivated due to the distribution of the grants under contract in the 2014-2015 academic year. • Cohort B Standard 4 learners were not likely exposed to EGRA materials, lesson plans, or trained teachers at all. But they did, likely, receive the benefits of one year of the extended school day and reading lessons, reduced class sizes, more parental and community involvement in the schools, and at least one reading fair. Their teachers might have also been more motivated due to the distribution of the grants under contract in the 2014-2015 academic year. INVC, A FEED THE FUTURE INITIATIVE The INVC Activity is USAID/Malawi’s flagship agriculture activity under the FtF global initiative to reduce poverty and hunger, and is implemented by Development Alternatives, Inc. (DAI). The activity began in 2013 to create and strengthen soy and groundnut value chains, improve the nutritional status of women and children, and build the capacity of Malawian agriculture and nutrition organizations. INVC takes a dual approach to accomplishing its goals. First, value-chain innovations are encouraged under this activity in 7 order to increase productivity and help farmers earn higher incomes. At the same time, the activity also promotes consumption of highly nutritious soy and groundnuts, and builds community capacity to prevent undernutrition. The INVC Activity is now active in seven districts, including the treatment zones of four EGRA IE districts—Lilongwe Rural, Machinga, Balaka, and Ntcheu. Specific INVC activities related to nutrition include the creation and strengthening of community-level “Care Groups,” wherein a lead mother/father facilitates local meetings to share trainings, videos, theater performances, and information about nutritional best practices at the family level. These nutrition topics include dietary diversity, breastfeeding, and maternal nutrition. Each group is made up of parents who have children under the age of five or are expecting a child. As the prime implementer, DAI works through a network of local partners to train and maintain these groups. The “Care Groups” complement messaging in radio broadcasts to reinforce key information. INVC agricultural activities utilize a similar approach with local Farm Clubs that share effective farming practices, approaches to selling and saving, and new tools to widen access to markets. In addition to information sharing, local farm clubs distribute seeds of newer varieties to help crop diversification and thereby lower farmers’ overall crop risk. Seed distribution, along with some production inputs, are carried out through a network of preexisting agricultural professional groups such as the National Smallholder Farmers’ Association of Malawi (NASFAM) and the Farmers Union of Malawi (FUM). INVC targets smallholder farmers that own/operate 0.5–1.2 hectares of land. As per USAID/Malawi’s CDCS, EGRA and INVC are currently planning on future collaborations that include local radio shows, producing low-literacy reading materials, and a community-garden activity that may also encourage attendance at adult literacy classes. At the time of midline data collection, the plans had not been implemented or scheduled for the year, but partners expected to move forward with their integration plans soon. These cross-sectoral integrated activities are expected to benefit households, including EGRA-beneficiary households, in the implementation districts. To date, INVC-beneficiary households with Standard 2 and 4 learners may have benefitted from improved dietary diversity, crop diversification and use of improved seed inputs, and improved farming and marketing knowledge, all of which may have led to higher levels of household income and/or improved nutrition for beneficiary learners. Learners who come from households with higher incomes or who have improved nutrition often do better in school because they often have more resources available to them to assist with their learning and they are also usually better able to focus (due to adequate nutrition). SSDI USAID Malawi’s five-year flagship health activity, the Support for Service Delivery Integration Services (SSDI), began in October 2012 in close collaboration with the Ministry of Health (MoH) to support the GoM in the areas of health communications, service delivery, and systems strengthening. Jhpiego, the prime implementing partner, works closely with a network of local and international partners to implement activities in 15 of Malawi’s 28 districts. Four of the ten EGRA IE study districts are currently covered by SSDI activities, and the SSDI activities are spread completely across those four districts, which include Lilongwe Rural, Machinga, Balaka, and Salima. SSDI aims to support the Malawian MoH in its service provision of the following types: maternal/neonatal care, nutrition, Human Immunodeficiency Virus Prevention of Mother-to-Child Transmission (HIV/PMTCT), family planning, and malaria. The activity includes both facility-based and community-based activities. At health clinics, SSDI provides medical training and equipment to more effectively address medical issues related to new or expectant mothers and their children. At the community level, SSDI also trains and equips MoH Health Surveillance Assistants (HSAs) in community case management (CCM) of 8 conditions such as pneumonia, diarrhea, and malaria. Community volunteers are also recruited to further local knowledge on topics such as vaccinations and nutrition for children. At both the facilities and in the community, SSDI activities are focused on new and expectant mothers and children aged 0–5. SSDI implementation in Malawi is scheduled to scale down by December 2015, and implementation will conclude in 2016. As part of USAID/Malawi’s CDCS, the EGRA and SSDI implementers are currently planning on future collaborations that include an information dissemination strategy to improve their results. This includes collaborating on development of a comic book featuring malaria, which will be distributed in EGRA reading centers and schools across many Malawian districts. Also, SSDI and INVC have identified potential opportunities to disseminate HIV information through self-help groups organized for livelihood improvements (see Social Impact’s 2015 CDCS Baseline Report). At the time of midline data collection, however, the plans had not been implemented or scheduled for the year, but both partners expected to move forward with their integration plans soon. These cross-sectoral integrated activities are expected to benefit households, including EGRA-beneficiary households, in the implementation districts. To date, SSDI-beneficiary households with Standard 2 and 4 learners may have benefitted from improved medical treatment by HSAs or through improved vaccinations or nutrition, all of which may have led to healthier beneficiary learners and/or improved nutrition for beneficiary learners. Learners who get sick less often and have better nutrition often do better in school because they are better able to focus and also simply do not miss as many lessons. OTHER EARLY GRADE LITERACY PROGRAMS Though not included in the evaluation questions of the IE, SI notes the presence of other implementers focusing on early grade reading in the sampled area. These include: • TIANA, or Tiwerenge Ndi Ana Athu, is funded by USAID and implemented by Save the Children as part of the All-Children-Reading Initiative. Since February 2013, the initiative has used both teacher- and learner-focused activities to strengthen literacy on a community level. These activities include literacy camps and youth movements to promote reading skills. All TIANA schools are found in the Zomba district. • Literacy Boost is a multi-country project implemented and funded by Save the Children. Similar to TIANA, Literacy Boost uses a community-level approach to engage teachers, families, and learners with activities that include reading camps, teacher trainings, establishing book banks, arranging “reading buddies,” and providing books. Activities began in January 2009 and continue with a presence in both the Zomba and Balaka districts. • Strengthening Early Grade Reading in Malawi (SEGREM) began in October 2014 using similar approaches to EGRA. The implementing partner, the Malawi Institute of Education (MIE), is a parastatal organization that replicates EGRA activities in Standards 1-4 for 519 schools in the Chiladzulu, Dedza, and Mchinji districts. The intervention is scheduled to conclude in October 2017 and is funded by USAID. This activity does not affect this IE. • Girls’ Empowerment through Education and Health Activity (ASPIRE) is funded by USAID and implemented by Save the Children. This intervention focuses on upper primary girls and builds off of the work done in EGRA schools in Standards 1-3. In addition to developing literacy skills, ASPIRE will encourage girls ages 10-19 to adopt positive sexual and healthcare seeking behaviors. It will also aim to decrease structural and cultural barriers for girls’ access to education. The activity began in December 2014 and is set to conclude in December 2018. 9 ASPIRE is in all secondary schools in the Balaka and Machinga districts. While this activity does have the potential to affect this IE - especially for Standard 4 learners, since its implementation was still beginning and yet to roll-out at the time of data collection - SI assumed (and also found through examining the data) that it was not affecting the reading outcomes of Standard 4 learners in these districts. However, SI will examine this activity further for the endline report, when results may be affected—not only for Standard 4 learners but for Standard 2 learners with parents who also have Standard 4 learners—since many of the activities are focused on parents and the community. • FAWEMA or the Forum for African Women Educationalists – Malawi Chapter – is a non-profit organization founded in Malawi in 1993. Since its founding, the organization has been working to further girls’ education and training through a formalized partnership with MoEST and other partners. According to its website, FAWEMA works through the following five channels: influencing education policies, demonstrative interventions, advocacy, replication and scaling-up of best practices, and capacity building. • Other School-Level Education Interventions, including activities implemented by the following donors: Concern Universal, the United Kingdom’s Department for International Development (DfID), Mary’s Meals, Plan Malawi, the United Nation’s Children Fund (UNICEF), World Vision, and Yoneco. Since several of the above activities overlap with the IE sample, SI documented the coverage and overlap in both the baseline and midline during data collection using the household and head teacher surveys. 10 IV. METHODOLOGY AND LIMITATIONS IMPACT EVALUATION OVERVIEW Answering causal questions such as “What is USAID/Malawi EGRA’s impact on children’s reading abilities?” and attributing the impact to a specific program requires ruling out alternative possible causes for impacts or changes in outcomes. Recognizing the multitude of possible alternative causes for changes in EGRA’s key outcomes (learner reading ability, dropouts, and repetition rates) between government programs, other donor initiatives, and political and/or economic development, the 2011 USAID Evaluation Policy (EP) requires that IEs use a carefully-selected comparison group to rule out possible alternative causes for key outcomes through estimating the counterfactual, or the level of change in project participant outcomes expected in the absence of the project. By comparing project participants with a comparison group, it is possible to “subtract away” the contextual changes (or those caused by other interventions or natural changes such as time) that affect both activity participants and non-activity participants (the comparison group). If activity participation is the only substantive difference between participants and the comparison group, then any differences in outcomes between the two groups can be attributed to the activity. In order to test possible complementary or multiplier effects of EGRA and the INVC and SSDI activities, EGRA and the associated IE are being implemented in and evaluated for four distinct treatment levels: Treatment Level 1. Three focus districts (Balaka, Machinga, and Lilongwe Rural) that provide an opportunity to evaluate the impact of a fully-integrated development approach with multiple activities across sectors, including EGRA, INVC, and SSDI, on early grade reading outcomes. Treatment Level 2. The district (Salima) where EGRA overlaps with only the SSDI intervention. This serves as a test ground for the hypothesis that synergies between education and health initiatives catalyze changes that are greater than the sum of their parts. Treatment Level 3. The district (Ntcheu) where EGRA overlaps with only the INVC intervention. This serves as a test ground for the development hypothesis that synergies between education and agricultural livelihood and nutrition initiatives catalyze changes that are greater than the sum of their parts. Treatment Level 4. Five districts (Blantyre Rural, Mzimba North, Ntchisi, Thyolo, and Zomba Rural) that only receive the EGRA initiative. These districts are used to test the EGRA theory of change that education support leads to improved literacy and general education outcomes. In order to infer impacts, SI compared learner reading scores from each of these four groups with a comparison group to determine the effectiveness of each type of treatment. SI included comparison schools at each level within these same districts (with the exception of Level 2, where a comparison group was not possible since SSDI is implemented across the entire district rather than just in treatment areas) rather than having some districts as treatment districts and some as comparison districts. This was done to increase the likelihood that the evaluation will be able to attribute identified changes to EGRA, SSDI, and/or INVC, rather than to non-project-related differences between districts. The study includes comparisons of the four types of treatment with one another, although it is more difficult to isolate the effects of the different types of treatment from district-level effects in such a comparison, since each of the treatment types is being rolled out in separate districts (this is explained in more detail in the treatment assignment section of this report). 11 The evaluation team follows the same schools longitudinally at baseline, midline, and endline, but not the same set of learners longitudinally, due to the evaluation need to assess the same standards — Standards 2 and 4 — at baseline, midline and endline, but not the same students in different years.8 TREATMENT ASSIGNMENT BASELINE ASSIGNMENT While SI expected to include both treatment and comparison schools in each district at each level, at baseline, it was unable to because the SSDI and INVC Activities were already underway. As such, SI found that in the Level 2 district, Salima, it was impossible to select comparison zones that were not already contaminated with the SSDI intervention because SSDI was already working across the entire district. Therefore, no comparison schools were assigned at Level 2. SSDI was also already working across all of the Level 1 districts—Balaka, Machinga, and Blantyre Rural. So, despite still including comparison schools at Level 1, SI’s estimation of treatment effects at this level are likely somewhat underestimated because SI is unable to measure the effects of the SSDI Activity in addition to the EGRA and INVC Activity. For that reason, in addition to comparing treatment schools in one level against comparison schools in the same level, the evaluation team also compares the treatment schools for each level against the comparison schools in Levels 3 and 4 only, since these comparison schools represent the only “true comparisons.” Since EGRA is implemented at the zonal level, at baseline in 2013, SI randomly selected zones in each of the four levels described above to implement the EGRA intervention (taking into account areas where INVC and SSDI were already working). However, since INVC and SSDI were not randomly assigned at baseline, the evaluation team is only able to determine whether EGRA is better than no EGRA and whether EGRA plus INVC and SSDI is better than no treatment. For more details on sample selection at the district level, see the 2014 IE baseline report prepared by SI and approved by USAID. PHASED TREATMENT DESIGN For at least one year after baseline, the comparison zones did not receive an intervention of any type. However, RTI, MoEST, and USAID wanted to offer the EGRA intervention to as many learners as possible. Thus, beginning in the 2014-2015 academic year, RTI began to phase in support in some of the comparison zones in Level 4, essentially turning those zones into treatment zones to ensure more wide-spread access to support through the project. SI worked with RTI and USAID in 2014 to ensure the zones that were phased into EGRA treatment were randomly selected from the comparison zones from Level 4. RTI used a public lottery to select 20 out of the 41comparison zones in the five Level 4 districts to be phased into EGRA treatment and called Cohort B, as shown in Table 3. 8 The differences found between baseline and midline could be affected by the quality of students assessed each year. Therefore, we checked for similarity/differences between the two samples, and included some factors that help explain the quality of students as controls in the regressions. Further, by comparing treatment against comparison schools - both with different sets of learners - we minimize any effects on students that might come from trends in changes in learners. Also, we have included analysis of implementation factors to look at treatment fidelity in the regressions. 12 Table 3: Number of Zones and Schools Converted from Comparison to Treatment in 2014 (Year 2 of EGRA Project, All in Level 4 Treatment Zones) DISTRICT CONVERTED ZONES CONVERTED SCHOOLS Blantyre Rural 3 6 Mzimba North 5 13 Ntchisi 3 6 Thyolo 5 12 Zomba Rural 4 8 Total 20 45 No zones in the other three levels (Levels 1, 2, or 3) were converted from comparison to treatment in 2014. Also, USAID decided the conversion in 2014 was the only phasing in that would occur until the evaluation ends in 2017. Therefore, RTI will not phase in other zones or schools within any other zones before the end of this IE. The rest remain as comparison zones and schools, as per the baseline assignment. Phasing in the treatment only for some comparison schools, rather than providing all comparison schools with treatment in the second year of the project, allows the evaluation team to compare the effects of two years of the EGRA intervention with the effects of one year of the intervention. However, the phased design does limit the evaluation team’s ability to measure the effects of more than two years of EGRA treatment for Cohort B schools from Level 4. SI followed the same set of schools selected at baseline for midline, to the extent possible. The phasing in of some schools in 2014, however, affected the treatment assignment and sample size for midline, as discussed below. SAMPLE SIZE BASELINE SAMPLE SIZE SI conducted power calculations prior to the baseline, based on clustering at the zonal level, using data from the Malawi Teacher Professional Development Support (MTPDS) Activity (the predecessor to the EGRA intervention) to establish an adequate sample size to measure the level of changes in key outcomes that USAID expected to see. SI estimated that a total of 320 schools with 9,600 learners would be required (see the 2014 baseline report for details on the power calculations). However, during the baseline, the sample size had to be adjusted slightly due to field conditions.As such, the sample realized at baseline was slightly smaller than the intended sample size (based on the power calculations done prior to baseline) due to the following: (i) during baseline sampling, the evaluation team realized that there were only 28 possible comparison schools within Level 3. Thus, the remaining 13 comparison schools for this level were moved to Level 4 to allow for more pure comparison schools, as described later; (ii) the team found at baseline that the SSDI Activity had already spread across the districts it was working in before baseline data could be collected, contaminating possible comparison zones and schools (as described above). Because of these issues, rather than including 40 schools per treatment and 40 schools per comparison level, the evaluation team decided to move the comparison schools from Level 2 treatment to Level 4, since Levels 3 and 4 were the only levels where pure comparison schools were feasible (given contamination by the SSDI Activity). This was done to avoid comparing EGRA and SSDI to SSDI alone, which would essentially be answering the same question as that of Level 4. Finally, (iii) several challenges arose during baseline data collection (i.e., the reading assessment took longer than expected, school let out earlier than normal or started late, and data lost on some schools due to data collection supervisors leaving their teams, etc.). 13 As such, the final sample used at baseline included 8,910 learners selected from 310 schools across the ten districts. Table 4 shows the number of schools and the average number of learners per school per treatment type. Table 4: Number of Schools and Learners at Baseline (2013) by Treatment Group LEVEL TREATMENT (NUMBER OF ZONES IN PARENTHESIS) NUMBER OF SCHOOLS AVERAGE LEARNERS PER SCHOOL TOTAL LEARNERS L1 Treatment (18) 40 27.5 1,101 L1 Comparison (13) 40 30 1,198 L2 Treatment (7) 32 28.4 908 L3 Treatment (11) 39 30 1,171 L3 Comparison (4) 27 27.7 748 L4 Treatment (27) 40 26.8 1,071 L4 Comparison (41) 92 29.5 2,713 Total Treatment 151 - 4,251 Total Comparison 159 - 4,659 Grand Total All 310 - 8,910 MIDLINE SAMPLE SIZE Prior to midline, SI estimated that the change in sample size due to the phasing in of some comparison schools to treatment schools in Level 4 would still allow for a minimal detectable effect size (MDES) of 0.20 at Level 4, which was even smaller than the MDES estimated prior to baseline data collection. The reason for this is that SI’s power calculations at baseline were very conservative, and when actual data were collected at baseline, SI found it had more power than expected. Also, the conversion of some Level 4 schools to treatment did not have a major effect on power, as SI was left with more than 40 schools still in the comparison group for Level 4 (due to the shifting of some sample to Level 4 comparison, as described above). In order to maintain the power in the other three levels at midline, SI sought to ensure collection of the full 9,600 learners in 320 schools planned at baseline. SI made all efforts to minimize any drop in the number of schools or learners surveyed due to challenges that arose during baseline data collection (for example, longer time to administer reading assessments, shorter school days due to schools letting out early or starting late, and schools lost due to data collection supervisors leaving their teams). As a result, at midline SI collected data from 320 schools and all but one of the sampled schools included in the baseline. Prior to the midline, one school was disbanded due to a land-rights issue and replaced with another school in the sample. However, as described above and planned for in the design, the same learners at baseline were not tested at midline. A new set of learners from the same Standards 2 and 4 were tested, according to USAID’s interest in tracking results over years across standards rather than following cohorts of learners. As a result, SI was able to reach the sample size at midline shown in Table 5. 14 Table 5: Number of Zones, Schools, and Learners at Midline (2015) by Treatment Group LEVEL TREATMENT (# ZONES IN PARENTHESES) NUMBER SCHOOLS LEARNERS PER SCHOOL TOTAL LEARNERS L1 Treatment (18) 42 30 1,264 L1 Comparison (13) 38 30 1,141 L2 Treatment (10) 40 30 1,199 L3 Treatment (10) 41 30 1,229 L3 Comparison (3) 27 30 811 L4 Treatment (59) 84 30 2,519 L4 Comparison (21) 48 30 1,426 Total Treatment (97) 210 30 6,211 Total Comparison (37) 110 30 3,378 Grand Total All 320 - 9,589 In working to answer evaluation questions and to statistically test the hypotheses of EGRA program impacts, however, SI used a panel of schools that remained with the same treatment status at both baseline and midline. This helped to examine changes across time by treatment status using average gains in treatment and comparison for each school in the panel. That is, when comparing baseline average outcomes to those of midline averages for each school, SI excluded some schools, as appropriate for the analysis. These exclusions included: (i) 46 comparison schools in Level 4 at baseline that were converted into treatment in 2014 (Cohort B schools or “New Treatment” schools), since these schools only received treatment for a very short period at midline and essentially could be similar to comparison schools in terms of length of exposure; (ii) schools that were planned for baseline but could not be surveyed, but were surveyed at midline (12 schools); and (iii) one school that was surveyed at baseline but was disbanded, leading SI to drop and replace the school in the midline sample. Using the exclusion criteria, 56 schools were excluded (certain schools satisfied more than one criterion), which resulted in a sample size of 264 schools. Since these schools were randomly selected, there was no attrition bias that could affect power of the evaluation. But, SI intends to use the data from new treatment schools and the 320 schools that were surveyed in midline for endline impact analysis when midline and endline results will be compared for inferring EGRA effects. More details are presented later in this report, in the midline analysis methods section. DATA COLLECTION INSTRUMENTS Although English and Chichewa are the official languages of Malawi, and English is now the primary language of instruction for all learners beginning in Standard 1 (MoEST passed a new policy in 2014 that changed the language of instruction), learners in the early grades are still being taught to read in Chichewa in addition to English. The baseline included testing in Tumbuka, Yao, English, and Chichewa; however, USAID decided to shift its focus away from languages other than Chichewa to avoid politicizing these assessments (given the heated debate about language of instruction happening in the country). Therefore, USAID requested that the midline assessment only assess Chichewa reading skills. 15 SI used the following data collection tools, which were adapted from baseline in coordination with USAID and the MoEST for the midline EGRA IE: • Chichewa Early Grade Reading Assessment Tool • Learner Survey • Teacher Survey • Head Teacher Survey • Classroom Observation Protocol • School Climate Protocol • Household Survey CHANGES MADE AT MIDLINE ON SURVEY INSTRUMENTS SI’s Institutional Review Board approved the tools presented above, which SI then presented to USAID for feedback and approval prior to the 2015 midline assessment. Since the baseline, SI made some revisions to the instruments for the midline assessment. These include the following: EGRA Tool. At midline, in order to avoid bias in scores if a learner had taken the same test version at baseline or the teachers in the sampled schools were using the baseline tool for teaching purposes, SI used a version of the Chichewa language EGRA that was not used at baseline. RTI developed both of those tools in 2010 in Chichewa to use in MTPDS assessments in 2010 and 2011. At baseline, SI used the EGRA tool from the MTPDS program, used in 2011. At midline, SI used the tool used in the 2010 MTPDS program. Head Teacher Survey. SI included some new questions at midline in the school survey to gather data on whether the school has been reached by EGRA since implementation started, and a more detailed section to examine which other reading initiatives through other projects are being implemented and what activities are included in those projects. Household Survey. The household survey was expanded to include more questions on the INVC and SSDI interventions. The new questions, added by DAI and Jhpeigo, delve deeper into the program contributions and respondents’ participation in the programs. The expansion enhances SI’s ability to evaluate the CDCS hypothesis. All Tools. During the Baseline Assessment, only some of the tools (including the EGRA tool, the learner survey, and the household data collection tool) were programmed into the tablets, while others were implemented using paper surveys due to time constraints. At midline, enumerators used electronic data collection for all of the instruments. This helped to reduce data loss and data entry errors. In order to reduce interviewee fatigue and reporting errors, certain information from baseline was also pre-populated in relevant places, and the respondents were only asked to confirm whether the information had changed. MIDLINE SURVEY IMPLEMENTATION ACTIVITIES MIDLINE DATA COLLECTION: SCHOOL LEVEL DATA The 2015 midline began with a training of the Survey and Logistics Managers between April 7 and 9 in Zomba. Invest in Knowledge (IKI), SI’s local data collection partner, SI, and MoEST hosted the event, which was attended by IKI’s Survey and Logistics Managers and Senior officials from MoEST. Training focused on the schedule, the purpose of the study, the role of the Survey and Logistics Managers, selection of survey and assessment participants, assignment of unique identifiers for all surveys/assessments, general best practices in data collection and working with youth subjects, a detailed technical review of all of the data collection instruments, a review of data collection using tablets, and guidance on preparing and submitting weekly progress reports by IKI to SI. 16 The Survey Manager training was followed closely by an enumerator and Technical Manager training in Lilongwe from April 13 to 18. Hosted by USAID, MoEST, IKI, and SI, this event trained MoEST staff to contribute to the IE as enumerators and Technical Managers and further expanded upon the training of the Survey Managers. The training included a two-day field test to pilot the instruments and protocols in the field. Following this field test, SI and IKI revised and updated the instruments and reprogrammed the tablets to ensure ease of use. School data collection began on April 20 and concluded on June 15. MIDLINE DATA COLLECTION: HOUSEHOLD SURVEY SI and IKI began household survey data collection shortly after completion of the school visits. From May 19 to 23, IKI supervisors and enumerators attended a training in Zomba focused on the household survey and protocol. IKI and SI hosted the training, which featured a full-day pilot test of the midline instrument. IKI survey teams departed for their field assignments on Sunday, May 24, 2015, and visited the 320 communities previously visited by the school visit teams. Evaluators divided the enumerators into 24 teams of five staff each including a single supervisor. Survey teams completed the household visits on June 15. DATA CLEANING AND DATA QUALITY ASSURANCE SI and IKI collaborated to review the data sets for their completeness. SI did not find incomplete assessments or surveys; so, there was no need to follow-up on incomplete data. But, in a limited number of schools, SI and IKI identified missing surveys and assessments and initiated a second round of school visits, seeking out just the missing surveys and assessments from approximately 20 schools. These data were added to the original data, and the midline data were then considered complete as of August 20, 2015. SI also checked the data for accuracy, with an emphasis on student and school identifiers that are essential to connect a learner’s information with that of his/her school and household. In addition, SI and IKI identified and corrected other errors such as suspicious/outlier responses and violations in a survey’s skip logic. Reviewers prioritized these errors by analytic importance and collaboratively addressed them between IKI and SI. SI took rigorous quality assurance steps throughout the process to ensure that the data were complete and accurate. This began with using programmable tablets to avoid data loss and preclude a wide variety of user errors. The process continued with the cleaning and completeness checks described above. During field work, a separate team of enumerators revisited respondents with a sub-sample of the original questions to ensure responses matched the earlier, original survey. IKI and SI conducted these traditional back checks as well as random audio audits for more than 10 percent of the schools and survey respondents. These back checks and audits involved verifying responses by either revisiting respondents and asking a subset of questions or randomly activating the microphones on the tablets while surveys were in progress. SI then compared these back checks and audits with actual data collected to reconcile results. MIDLINE ANALYSIS METHODS At midline, in order to address the evaluation questions by the three relevant tasks for IE, SI analyzed the data to discuss the following: 17 Task 1 • At midline, using midline data, what proportion of Standards 2 and 4 learners have attained established benchmarks? • Using both baseline and midline data, what has been the change (gains) in learning scores for Standards 2 and 4 since the EGRA intervention began by comparing baseline and midline data? Task 2 • At midline, using midline data, what are the household, school, and other predictors of learning scores? Do they differ by gender and treatment status? – What is the link between midline reading achievement data of learners in Standards 2 and 4, by treatment status (comparison and treatment), and midline information on teaching practices, school environment, head teachers information, teacher information, learner information, and household data? • At midline, using midline data, what factors at the household and community levels correlate with learner grade repetition and early departure from school? Do they differ by gender and treatment status? – Impact on repetition rate – Impact on dropout rate – Impact on school completion, particularly for girls and learners with disabilities • At midline, using midline data, are girls and boys treated differently in the household in ways that influence academic achievement? • At midline, using midline data, how do teachers’ classroom behavior and practices affect the ability of children to read? – How did the level of coaching impact teacher behavior and learner reading outcomes? These factors will help USAID, MoEST, and RTI understand best practices in fostering reading skills in early grades in the Malawian context Task 4 • Using both baseline and midline data, what is the EGRA effect on learning scores for Standards 2 and 4? – Comparisons of changes from baseline to midline are made in comparison and treatment schools on reading achievement to measure EGRA impacts over a period of two years of EGRA implementation in order to test the hypothesis that learner reading outcomes will improve with the EGRA intervention. • Using both baseline and midline data, what predictable factors besides EGRA affect changes in learning scores? – Do household, school, and other factors predict changes in learning scores besides EGRA intervention? Do they differ by gender of the learner? • What is the impact of level of effort of reading instructor on learners’ reading abilities? • What is the effect of extracurricular reading activities? • What is the effect of time-on-task in improving reading outcomes? • Using midline data, do other USAID activities in agriculture and health sectors in study areas affect outcomes besides EGRA? – This tests the hypothesis that integrating USAID interventions in education, agriculture, health, and community strengthening in the same communities results in increased student learning, as described in the USAID/Malawi CDCS. This includes measuring how integration of USAID programing across sectors (education, health, and agriculture) in the same geographic areas impacts student reading outcomes. • Using midline data, what secondary effects can be attributed to EGRA? – Impact on repetition rate 18 – Impact on dropout rate – Impact on school completion, particularly for girls and learners with disabilities • Using baseline and midline data on program impact and total direct costs in implementing EGRA by RTI, what is the cost effectiveness of the EGRA intervention? SI began analysis of the midline data incrementally, as it became available and cleared quality assurance and cleaning checks. As discussed below, SI included key indicators from the contract in its analysis, chief amongst which were scores on the nine reading subtasks as well as the percentage of students meeting MoEST or EGRA Coordinating Committee benchmarks (described in the next section), depending on the subtasks. SI used a Difference in Differences (DiD) approach to examine treatment effects attributable to the EGRA intervention. In addition, SI repeated the analytic approach used in the 2013 baseline to identify characteristics and factors at midline that predict reading scores. SI used Principal Component Analysis (PCA) to summarize related characteristics (wealth/assets, school characteristics, etc.) into single indices. EQUATING EGRA TOOLS As discussed above, SI used an EGRA tool at midline that was different from baseline. In order to ensure that the results obtained from the two tools used at base and midline are comparable, SI conducted a pilot at midline with 603 learners in 21 schools that were not part of the IE sample using both baseline and midline EGRA tools. This helped to test for similarity of the two EGRA versions in their difficulty levels such that any differences noticed in scores between baseline and midline were only due to changes in learner reading abilities and not due to the change in the tool used to conduct the assessments. The pilot data, after removing zero scores, were analyzed for reliability, correlation, and differences in scores by subtasks to arrive at a conversion factor that can help equate the two tools (see Annex 3 for more details). Means equating was used to assign conversion values for the timed sub-tasks based on observed difficulty in midline tool in comparison to the baseline tool. Table 6: Values to Equate Midline and Baseline EGRA Tools Sub Tasks Conversion Factor Used on Midline Scores Letter Name Knowledge 1.015 Syllable Reading 0.864 Familiar Word Reading 1.021 Oral Reading Fluency 1.032 Non-Word Reading 1.046 Source: Non zero pilot data collected by SI, 2015. The conversion factors, shown in Table 6, were used to multiply midline scores to form new scores - equated scores. These equated scores were used throughout this report as midline scores and were also used to compare with baseline scores to assess changes across time to infer program effects. WEIGHTING THE DATA The evaluation design used stratified sampling method. The learners tested for the evaluation were randomly selected and were clustered within schools, and the selected schools were located within districts. Since every school and learner did not have an equal chance of selection, statistical procedure was needed to adjust for design effects. Weights were constructed based on the probability of selection of each school and learner in the sample. Sampling weights were constructed by SI at both the district 19 level and the school/standard level and used in the analysis in this report. The weights were applied to the dataset as probability weights, or pweights, using STATA version 14’s set of survey commands. BENCHMARKS SI used benchmarks developed by USAID, RTI, and MoEST for each subtask to compare learner reading scores. Table 7: 2014 Benchmarks for Reading Comprehension, Oral Reading Fluency, Familiar Word Reading, and Syllable Reading Items Std. 3 Std. 2 Std. 1 Reading Comprehension Recommended benchmark 80% 80% 60% Recommended objective: % at benchmark in 5 years 50% 40% 35% Recommended objective: % of zero scores in 5 years 10% 20% 30% Oral Reading Fluency Recommended benchmark 50 40 30 Recommended objective: % at benchmark in 5 years 50% 50% 40% Recommended objective: % of zero scores in 5 years 5% 10% 20% Familiar Word Reading Recommended benchmark 45 40 30 Recommended objective: % at benchmark in 5 years 50% 50% 40% Recommended objective: % of zero scores in 5 years 5% 10% 20% Syllable Reading Recommended benchmark 65 60 50 Recommended objective: % at benchmark in 5 years 60% 55% 50% Recommended objective: % of zero scores in 5 years 5% 10% 15% Source: MoEST and USAID/Malawi, December 2014: Proposing Benchmarks for EGRA in Malawi. For oral reading fluency, reading comprehension, familiar word reading, and syllable reading, SI used the new benchmarks developed by MoEST, USAID and RTI in December 2014 (Table 7). It is to be noted that these benchmarks are set to be reached in five years (in 2018; year five of EGRA implementation). At midline, EGRA has been implemented for only two years. Ideally, intermediate targets should have been set for years 2, 3 and 4 to indicate whether students are on their way to reaching the benchmarks set for year 5. Since no intermediate targets were provided in the MoEST benchmarks issued in Dec. 2014, SI did not apply any prorated benchmarks for 2015. Approximate allocation of benchmark result by years using the final benchmark assumes a linear change and SI considers that such linear change may not occur in 20 reality, as shown in some EGRA evaluations. Therefore, benchmarking midline learner scores for these tasks at 5 year benchmark levels could underestimate the progress made at midline. The new benchmarks in 2014, however, were only developed for four of the nine sub-tasks. Therefore, SI used EGRA-Coordinating-Committee-recommended benchmarks from 2011 for listening comprehension, syllable segmentation, and initial sound identification; and MoEST benchmarks established in 2011 for letter name knowledge and non-word reading when comparing learner reading scores against benchmarks (see Annex 4 for more details). These benchmarks from 2011 are shown below: • Listening Comprehension: According to EGRA-Coordinating-Committee-recommended benchmarks, Standard 1 should be able to answer 3 out of the 5 questions correctly (60 percent) and Standard 3 should be able to answer 4 (80 percent). • Syllable Segmentation: In order to meet the EGRA-Coordinating-Committee-recommended benchmarks, learners must correctly segment 7 of the 10 words in Standard 1 (or 70 percent) and 8 of the 10 words in Standard 3 (or 80 percent). • Initial Sound Identification: The EGRA-Coordinating-Committee-recommended benchmarks for this subtask are 80 percent and 90 percent for Standards 1 and 3, respectively. • Letter Name Knowledge: The MoEST benchmarks are 24 clpm for Standard 1 and 50 clpm for Standard 3. • Non-Word Reading: The MoEST benchmarks are 15 cwpm in Standard 1 and 40 cwpm in Standard 3. Since no benchmarks were available for Standard 4, SI compared all Standard 4 learner scores against benchmarks set for learner achievement by the end of Standard 3. In the case of Standard 2, SI used Standard 2 benchmarks for the four subtasks for which MoEST benchmarks were available and Standard 1 benchmarks for the other subtasks, as the EGRA Coordinating Committee only set benchmarks for Standards 1 and 3. The above benchmarks were used for both the baseline and midline scores presented in this report. FACTORS AFFECTING READING OUTCOMES To answer Task 2 evaluation questions about the factors that predict reading outcomes, the study used measures of statistical correlation to examine the relationship between oral reading fluency and potential prediction variables from the head teacher, teacher, and learner questionnaires as well as the school climate and classroom observation protocols. Specifically, the assessment team specified multiple Tobit regression models, which allow results to be examined even when there is clustering around the lower and/or upper score bounds (ceiling and flooring effects). The team found strong flooring effects when using Ordinary Least Squares (OLS) regression models because there were so many zero scores. However, it is not necessarily true that all zero scores are the same, meaning that learners who scored zero may have differing levels of capability that the assessment tool (the EGRA) simply cannot pick up. Tobit works to correct for this challenge by predicting the change in oral reading scores for learners whose scores fall above zero, and weighting for the probability of scoring higher than zero. It then reveals the isolated effects of various factors on predicted values of reading scores while controlling for other factors. The team also used a Tobit regression model at the school level to examine factors that help predict dropout rates. However, to determine the factors that best predict repetition, the team used a logistic regression model since the outcome of interest—whether a learner has repeated a standard—is a binary variable. Finally, the evaluation team examined the effects of various EGRA components to assess implementation fidelity and determine which activity interventions have made the greatest difference. 21 PROGRAM IMPACTS USING THE DIFFERENCE-IN-DIFFERENCES MODEL In order to analyze data to address Task 4a and 4b questions, SI used difference-in-differences (DiD) approach where program impacts were examined through differences between treatment and comparison groups in oral reading fluency measured by correct words per minute read by the learner. In using the DiD model, SI used the standard assumption in DiD that while treatment and comparison schools may have been somewhat different at baseline (since selection was not completely random for any levels except Level 4), the trajectories/trends (slope) of learning in the treatment and the comparison groups over time would have been the same in the absence of the intervention.9 When this assumption holds, the basic DiD model allows evaluators to determine that the observed achievement is the result of EGRA and not due to chance or measurement error. Therefore, SI estimated the impact of the treatment assignment by examining the changes in scores between midline and baseline, and not the actual level of scores observed at baseline or midline. The basic DiD model under this approach can be written as: 𝑦𝑦𝑦𝑦 = 𝛼𝛼 + 𝛽𝛽 ∗ 𝑇𝑇 + 𝑢𝑢 Where, • 𝑦𝑦 is the changes in learning scores for each school between baseline and midline, • 𝛼𝛼 is an intercept, • 𝛽𝛽 is the treatment effect, • 𝑇𝑇 is a ‘dummy’ variable taking value 0 for comparison group and taking value 1 for the treatment group, and • 𝑢𝑢 is a residual term. SI ran regressions on the changes in average test score between baseline and midline for each school - that was included in both baseline and midline with similar treatment assignment - using ordinary least squares (OLS) against the treatment assignment. The panel of schools were matched using unique school IDs. The OLS model first assumed that the treatment and comparison groups did not differ systematically in characteristics that affected learning and that no other factor other than EGRA affected outcomes. However, such pure scenarios do not exist due to many interacting and facilitating factors in addition to EGRA. In order to account for such cases and obtain precise estimates, SI added relevant covariates from baseline to the DiD model. COST EFFECTIVENESS ANALYSIS Question 4d above is addressed using a cost effectiveness analysis where a cost effectiveness ratio of EGRA treatment is obtained by dividing direct costs incurred by a quantifiable outcome that is the focus of the evaluation, i.e. reading skills measured by EGRA. 9 The DiD methods (compared to propensity score matching) assume that unobserved heterogeneity in participation is present, but is time invariant. The assumption of time invariance can at times pose as a limitation. If projects are targeted in selected areas or schools using specific criteria, there could be dynamic response in both observed and unobserved ways in comparison and treated areas. In practice, while designing the evaluation, the ex-ante, time-varying unobserved heterogeneity could be accounted for by ensuring that treatment and comparison schools are drawn at baseline from similar districts or within districts. SI, as noted before, randomly drew comparison and treatment schools from the same district. If selection bias still remains an issue, combining PSM with DiD can help resolve it. Alternatively, controlling for initial conditions can also resolve nonrandom fixed events that might bias the program effect. Therefore, SI will apply DiD with controls in such cases for initial conditions (found at baseline) to infer program effects. 22 SI first calculated the program impacts due to EGRA. Then, the evaluation team looked at the costs of implementing EGRA in treatment schools, using data provided by RTI from their financial reporting of direct costs for school year 2014. The team calculated unit costs for treatment schools by dividing the total direct costs for EGRA by the 2014 school year enrollment in the EGRA project schools to obtain costs per learner. Using the impact and cost data above, SI calculated the cost effectiveness ratio (US dollar cost per unit effect) by dividing the costs by the EGRA effect size. In other words, the cost effectiveness captures the costs incurred to cause one additional unit effect due to the treatment. PRINCIPAL COMPONENT ANALYSIS The midline IE produced a large dataset, including hundreds of variables. Having a large number of variables was necessary in order to capture complex concepts such as school resources or quality of teaching practices. However, it was not practical to use all these variables in an unrestricted way during data analysis, for many reasons (see footnote).10 When a regression model incorporates several correlated variables, the problem of multi-collinearity could emerge. In such cases above, it is usually much more informative to aggregate these variables into indices, which then convey the main information contained in a group of variables. One way to construct these indices is to use a method called “principal component analysis” (PCA). This method decomposes a set of correlated variables into another set of linearly unrelated components. The single component that is found through statistical analysis to have the most explanatory power, the one that explains the highest amount of variance of the index as a whole, is chosen as the principal component. In a sense, it is then taken to represent all of the other components of the index, but using it in place of the others avoids the problems outlined above related to large numbers of correlated variables. One advantage of using this method over other ways of constructing an index (such as adding or averaging all variables in a group) is that it allows the data itself to guide the construction of the index rather than some external determinant. In selecting the principal component, PCA also produces a number by which learners, or schools, can be ranked, allowing for classification of units according to an independent variable of interest. This study used PCA to group some variables together and used it in regression analysis along with other variables. The grouping of variables conducted using PCA includes the following: Household wealth. A number of studies have been conducted on the topic of measuring wealth in developing countries, in large part because wealth is difficult to measure in poor populations due to the population’s tendency to rely on unconventional and inconsistent methods of generating income, acquiring and trading goods, and supporting the needs of their families. The majority of experts agree that assets are usually the best indicator of household wealth among impoverished families in developing countries. Income is difficult to measure because families tend to make their living farming — often only at the subsistence level — doing day labor, selling small goods, or selling crops grown in excess of subsistence, all of which usually produce inconsistent and unpredictable income. Cash is often scarce among the poor, so households often rely on trade to obtain goods they do not produce themselves. Because of this, household expenditures also are not a particularly reliable indicator of wealth for this population. Consumption is also an unreliable indicator, because families tend to vary their consumption patterns throughout the calendar year because of natural variations in their annual wealth arising from harvest 10 Multi-collinearity can cause large standard errors for the coefficients on the correlated variables, sometimes even resulting in a situation where two variables that are correlated and that should have the same signs actually end up with opposite signs. It can also cause two different but related independent variables that have been shown to have an effect on a dependent variable appear to have no significant effect whatsoever. This is because each one diminishes the effects of the other. These kinds of unanticipated results also contribute to a second problem, which is that regression models with large numbers of variables are difficult to interpret. The sheer number of variables leads to complex and unwieldy findings statements, and if the standard errors are large, the regression results often become more confusing to explain. 23 versus “lean seasons” and tourist seasons (for those who sell small goods or rely on tourists for day labor). Thus, assets tend to be the most reliable indicator of long-term wealth, with the caveat that assets do not indicate recent changes in wealth: It takes time for families to acquire assets after they have gained more wealth (either in the form of money or goods to sell or trade). The evaluation team created the PCA for wealth at the EGRA midline IE from each of the following factors: • Whether the household had access to piped water on its land, • Whether the household had a toilet, • Whether the household shares its toilet with any other households, • Whether the household has a paraffin lamp, cell phone, bicycle, table, bed with a mattress, sofa, radio, television, ox plow, jewelry, motorcycle/scooter, refrigerator, car/truck, and/or tractor, • Whether the household owns any cows, pigs, sheep, goats, chickens, horses, donkeys, and/or oxen, • Whether the household has electricity, • What materials the house is made of, including the walls, floor, and room, • And, how many rooms the household has. The team obtained the data from household surveys. Ownership of each of the assets listed above correlated with higher PCA scores, indicating that all of these assets were individually indicators of higher wealth and that the PCA score effectively captures relative wealth within the sampled population. The factors the team found to best predict household wealth were whether the household had access to electricity and whether the ceiling was made of metal.11 School resources. The assessment team created a PCA score for school resources using data from the school climate protocol and head teacher questionnaire. All of the following factors were included in that PCA score: • Whether the school has a library, • Whether the library is well stocked, • Whether the school has electricity, • Whether the school has plantings to make the grounds more attractive, • Whether there are any broken windows (really whether there is an absence of broken windows), • Whether the classrooms have functioning locks, • Whether there is space for the teacher and learners to move around, • Whether there is a teachers’ lounge available, • Whether classrooms have sufficient ventilation, • Whether the classrooms have sufficient light, • Whether the school has clean water, • Whether there are a variety of posters and resources on the classroom walls, • Whether latrines are available, • Whether latrines are available specifically for teachers, • And, whether most classrooms have desks. Outside of this PCA score, the study also considered the learner-to-teacher ratio, availability of girls’ latrines, whether the school had a school feeding program, and whether teachers felt they had sufficient 11 These variables were also found to be best predictors of household wealth and poverty status in poverty assessment tools (PAT and PPI) developed for Malawi. See http://www.progressoutofpoverty.org/country/malawi and http://www.microfinance.com/English/Papers/Scoring_Poverty_Malawi_2010_EN.pdf for PPI tool, and http://www.povertytools.org/countries/Malawi/Malawi.html developed by USAID for PAT. 24 access to school resources as indicators of the level of school resources. These factors were not included in the PCA score due to their perceived importance and explanatory capability. Teacher use of best practices in teaching reading. The final PCAs used by SI were teacher use of best practices in teaching reading and teacher use of essential practices in teaching reading. The use of best practices PCA was developed through analysis of all of the variables included in in the classroom observation protocol, which can be found in Annex 15. These are distinguished from the essential teaching practices, which include only 13 teaching practices from the classroom observations tool that, based on a review of literature and the USAID-approved RTI-International EGRA curriculum and classroom observation protocol, the team’s education specialist found embody what should be considered “essential” teaching practices. These practices include: • Whether the teacher assesses learner learning, • Whether the teacher introduces the lesson by connecting to what learners have learned previously, • Whether the teacher uses a lesson plan, • Whether the teacher has individual learners read aloud, • Whether the teacher engages learners in reading activities or games appropriate to reading level, • Whether the teacher asks learners questions to assess their understanding of something the learner(s) or teacher have/has read, • Whether the teacher provides learners with structured opportunities to apply understanding and skills to everyday life and problems, • Whether the teacher encourages learners to “sound it out” when they don’t know a word, • Whether the teacher asks learners questions to assess their understanding of stories they hear, • Whether the teacher asks learners to recognize letters and say letter names and/or sound, • Whether the teacher provides instructions on how to decode syllables and words, • Whether the teacher applies multiple methods to support comprehension, including games, group work, etc., • And, whether the teacher asks learners pre-reading questions. LIMITATIONS ABSENTEEISM AND DROPOUTS Factors such as absenteeism and dropouts can affect the internal validity of a study. The nature of this IE study and the fact that learning assessments are conducted at schools means that some learners will not be present at school when assessments are conducted. This is acceptable if the absent population is random and does not represent learners who do statistically better or worse on reading tests. However, the absentee population may not be the same as the population present at schools, since the absentee population may be over-representative of the lower-performing population of learners. The reason for this is that the GoM reports absenteeism rates averaging more than 25 percent of the learner population in the lower standard levels. And, SI found through qualitative data collection after the baseline in 2013 that some of the population tend to miss school more often than others. These learners tend to perform worse on tests and in school because of this, and they often drop out. This means that the results of the midline study are likely skewed slightly toward the positive when considering all enrolled learners in the sampled schools and likely even more positive when considering all school-aged children in Malawi. COMPARABILITY OF CONTEXTS: EXTERNAL VALIDITY The conclusions in this report were designed to be valid for the sampled districts and treatment and comparison populations only. As such, readers should not assume that conclusions described herein hold 25 true for all of Malawi or outside Malawi. Similarly, there may be smaller regions or sub-populations within the sampled districts that could differ significantly from the general norms and trends of sampled areas. Therefore, IE midline conclusions should only be taken to hold true in contexts that resemble the characteristics of the sample. This limitation was not unique to this assessment. It is a weakness that generally exists for all IEs and assessments. Having extensive data about the learners, their schools, their households, and their communities will help users of this study to assess how similar the context of the learners sampled is to other contexts into which these results may be extrapolated. This will allow USAID and other stakeholders to make an informed determination about how appropriate it would be to apply the findings of this assessment to other contexts. SAMPLE SIZE SI determined the sample size based on power calculations conducted prior to the IE baseline and again verified and confirmed those calculations after the IE baseline in 2013. While it was possible to detect differences in reading performance by sex of the learner and standard, some subgroups such as districts do not have large enough sample sizes to allow evaluators to be confident in differences identified between the districts. Such analysis would require relatively large real-world differences between groups or over time in order for the assessment team to be able to identify statistically significant differences over time. USE OF GOVERNMENT EMPLOYEES AS SUPERVISORS AND ENUMERATORS Another potential threat to the accuracy and reliability of the data was the use of MoEST staff as enumerators. The study recognized the value of involving the MoEST in this process: it capitalized on existing experience and expertise, especially of those individuals who were involved in the EGRA baseline study; it increased ownership of the MoEST for study results; and it built the capacity of the MoEST. However, there is always a risk when the same actors who are responsible for overseeing or implementing a project are asked to evaluate the project. It may have been in the interest of some individuals or groups within the MoEST to show improved reading outcomes over time or to show no change or a negative trend. In any case, when individuals who may have conflicting interests are involved directly in evaluation activities, there is always a risk that they may somehow inappropriately influence the results of the evaluation. On the other hand, GoM personnel, and in particular, MoEST staff, had been involved in data collection activities for evaluations of other activities in the past and conducted themselves in a professional and objective manner. Also, these data served an important purpose for the GoM and especially the MoEST. As such, to help inform their decisions related to reading, teaching, and learning, MoEST had a vested interest in obtaining accurate information from these evaluations. Finally, to help avoid issues of potential enumerator bias, SI made sure that no enumerators were assigned to gather data in the region in which they worked. Therefore, there is reason to believe that the risk of MoEST enumerator bias was very low. GENERALIZABILITY AT SCHOOL LEVEL During school visits, enumerators sampled learners from only one class per standard (implementing the Reading Assessment (RA) tool and learner questionnaire with those learners) along with the main teacher for that class (who was observed up to three times using the class observation protocol and was interviewed using the teacher survey). Since this approach was limited to one class per standard, and two classes per school, the responses and results have a reduced ability to be generalized or reach conclusions for the school as a whole. However, to mitigate any potential bias from this approach, enumerators chose each class at random, so no particular profile of class or teacher was sought (other than targeting classes where the teacher was present that day and had been at the school for at least a year). Further, it is likely that learners are randomly assigned to classes such that the selection of learners from one class does not bias average reading outcomes. 26 The advantage to sampling one class per standard at each school was that it gave the study team the ability to establish links between teaching practices and learner reading outcomes. Measuring these links would not be feasible between a given teacher and a wider, representative sample of learners in his/her standard as a whole. Limiting the survey group to one class per standard ensured that a teacher’s choices and behaviors were direct inputs into learner outcomes. As one important goal of this study was to identify and report on effective teaching practices, the study team and USAID decided that this advantage outweighed the limitations related to overall school generalizability. RESPONSE BIAS Response bias is a common issue with in-person surveys. This bias includes several types of false or adjusted responses where respondents react to stimuli other than that of the question itself (e.g., environment, presence of others nearby, etc.). Among these is a bias that occurs when interviewees favor responses they judge to be more pleasing or acceptable to the interviewer. In the context of the IE, this may skew the reported school data to suggest better teaching practices, more diligent study habits, or higher attendance rates than are actually the case. Further, it is difficult to measure the extent of this bias at work in this situation without more costly follow-up procedures. Fortunately, in this study, there was no reason to suspect that any response biases would not be uniform across respondents, so comparisons between subgroups (including treatment and comparison groups) should remain valid even if a bias were detected. Further, the study took several precautions to reduce such biases by carefully training enumerators on appropriate reactions to learner correct/incorrect answers and general attitudes when interviewing respondents. The SI team also made sure not to notify schools too far in advance (just calling the head teacher the night before the visit) of the team’s visit to avoid them only sending the best teachers to school that day or changing lesson plans/practices. 27 V. SAMPLE CHARACTERISTICS This section discusses midline school and household sample characteristics gathered through midline data collection conducted in April–June 2015. School-level data were gathered from surveying head teachers, teachers, and Standards 2 and 4 learners, and household data were gathered from households of Standards 2 and 4 learners selected for the learning assessments. Wherever relevant, the midline sample characteristics are compared with baseline sample to discuss similarities and divergences. SCHOOL SAMPLE BACKGROUND FEATURES The evaluation team presents the background characteristics of the school-level survey respondents at midline (learners, schools, head and class teachers, households) by treatment status. Although data were gathered from 320 schools at midline, in order to establish a panel of schools and compare with baseline characteristics, the evaluation team only included the schools that were part of both the baseline and midline and held the same treatment status at both times for the discussion below.12 SCHOOL CHARACTERISTICS Enumerators gathered school-specific data using the learner, teacher, and the head teacher questionnaires. Results from these surveys are presented below at the baseline and midline by treatment assignment and are further disaggregated by standard and gender of the learner. Learner Enrollment Learner enrollment was higher in treatment schools than in comparison schools, but the average enrollment from baseline to midline increased across both groups, albeit with greater magnitude in comparison schools. When disaggregated by gender, boys’ enrollment increased from baseline to midline in both comparison and treatment schools, but decreased in treatment schools for girls (Table 8 and Table 9). Overall, enrollment in treatment schools increased from an average of 214 learners at baseline to an average of 221 learners at midline; but in comparison schools, enrollment went from an average of 171 learners at baseline to an average of 199 learners at midline. The changes were similar when disaggregated by standard and gender. The average enrollment in treatment schools increased from 215 learners at baseline to 222 learners at midline for Standard 2 and from 166 learners at baseline to 173 learners at midline in Standard 4. In comparison schools, average enrollment was 175 at baseline and 198 at midline for Standard 2; and 128 at baseline and 147 at midline for Standard 4, indicating a larger increase than in treatment schools. Number of Teachers and Learner-to-Teacher Ratio The number of teachers decreased from baseline to midline in both treatment and comparison schools (Table 8 and Table 9). In comparison schools, the average number of teachers dropped from about 8 teachers across all standards at baseline to about 7 teachers at midline. In treatment schools, there were about 9 teachers at baseline across all standards and about 8 teachers at midline. The learner-to-teacher ratio increased from baseline to midline, indicating either fewer teachers, more learners, or a combination of the two. Considering that the average number of teachers decreased from baseline to midline while enrollment tended to increase, the increased ratio was likely driven by a combination of the two. 12 There were 320 schools sampled at midline when taken alone, but 264 schools sampled at both baseline and midline due to 1 school closing from baseline, 8 schools being newcomers, and 47 schools converting from comparison to treatment; thus, these 56 schools are not included when comparing baseline to midline. Since the schools were randomly selected, there was no attrition bias that could affect power of the evaluation. Also, SI used a conservative approach for sample size and schools were oversampled in Level 4 where the conversion occurred. See earlier section on sampling for more details. However, these 320 schools will be surveyed at endline in 2017 and will be used to compare with midline results from 320 schools. 28 Table 8: Average Student Enrollment and Teachers in Comparison Schools – Base and Midline Results for Standards 2 and 4. COMPARISON AVERAGE ENROLLMENT SE AVERAGE NUMBER OF TEACHERS SE AVERAGE LEARNER￾TO-TEACHER RATIO SE Baseline Standard 2 175.2 9.1 7.7 0.1 97.9 7.2 Girls 176.9 9.8 Boys 173.7 9.2 Standard 4 127.6 7.5 7.8 0.1 67.2 3.6 Girls 130.8 8.3 Boys 124.6 7.2 Overall 171.4 7.8 7.7 0.1 96.7 6.5 Girls 174.7 8.4 Boys 168.3 7.7 Midline Standard 2 198.3 10.6 6.8 0.1 114.8 6.5 Girls 197.8 10.7 Boys 198.7 10.5 Standard 4 146.6 8.9 6.8 0.1 83.1 4.6 Girls 146.7 9.0 Boys 146.6 8.8 Overall 198.6 9.4 6.8 0.1 112.2 4.7 Girls 198.4 9.5 Boys 198.8 9.4 Baseline Head Teacher Questionnaire 2013 and Midline Head Teacher Questionnaire 2015. SE = Standard Errors. Table 9: Average School Enrollment and Teachers in Treatment Schools - Base and Midline Results for Standards 2 and 4. TREATMENT AVERAGE ENROLLMENT SE AVERAGE NUMBER OF TEACHERS SE AVERAGE LEARNER￾TO-TEACHER RATIO SE Baseline Standard 2 215.1 17.3 8.6 0.1 106.8 6.4 Girls 222.9 21.2 Boys 208.1 17.2 Standard 4 165.7 14.6 8.5 0.1 77.2 4.5 Girls 174.9 15.9 Boys 157.3 15.3 Overall 213.7 15.5 8.6 0.1 102.0 5.6 Girls 220.9 18.7 Boys 207.3 15.2 Midline Standard 2 221.7 13.1 7.7 0.1 113.5 6.1 Girls 221.0 13.0 Boys 222.5 13.4 Standard 4 172.7 10.4 7.7 0.1 94.6 6.3 Girls 172.9 10.4 Boys 172.4 10.5 Overall 220.8 10.3 7.7 0.1 110.4 5.0 Girls 220.7 10.2 Boys 221.0 10.4 Baseline Head Teacher Questionnaire 2013 and Midline Head Teacher Questionnaire 2015. SE = Standard Errors. 29 Table 10: Average School Enrollment and Teachers in All Schools - Base and Midline Results for Standard 2 and 4. OVERALL AVERAGE ENROLLMENT SE AVERAGE NUMBER OF TEACHERS S E AVERAGE LEARNER-TO￾TEACHER RATIO SE Baseline Standard 2 197.1 10.6 8.2 0.1 102.9 4.8 Girls 202.1 12.9 Boys 192.7 10.5 Standard 4 148.4 9.1 8.2 0.1 72.6 3.0 Girls 154.9 10.1 Boys 142.5 9.3 Overall 194.6 9.5 8.2 0.1 99.6 4.2 Girls 199.9 11.5 Boys 189.7 9.4 Midline Standard 2 211.9 8.9 7.3 0.1 114.0 4.5 Girls 211.3 8.8 Boys 212.5 9.0 Standard 4 161.9 7.2 7.3 0.1 89.8 4.2 Girls 162.1 7.3 Boys 161.7 7.2 Overall 211.6 7.2 7.3 0.0 111.2 3.5 Girls 211.4 7.2 Boys 211.8 7.2 Baseline Head Teacher Questionnaire 2013 and Midline Head Teacher Questionnaire 2015. SE = Standard Errors. Learner Repetition Overall, average percentages of learners repeating the same standard were similar across treatment status, but increased from 18 at baseline to 22 at midline in both comparison and treatment schools (Table 11). Girls and boys repeated classes at similar numbers in both treatment and comparison schools. However, repetition steadily decreased from Standard 2 to Standard 4 at both baseline and midline, with greater magnitude in treatment schools than in comparison schools. 30 Table 11: Average Repetition by Standard and Sex - Base and Midline Results for Standard 2 and 4. AVERAGE REPETITION: COMPARISON SE AVERAGE REPETITION: TREATMENT SE AVERAGE REPETITION SE Baseline Standard 2 18.0 1.2 17.7 1.4 17.8 0.9 Girls 18.1 1.3 17.1 1.4 17.6 1.0 Boys 17.8 1.2 18.2 1.7 18.0 1.1 Standard 4 14.5 1.5 12.7 1.1 13.5 0.9 Girls 14.7 1.7 12.3 0.9 13.5 0.9 Boys 14.3 1.5 13.0 1.4 13.6 1.0 Overall 17.8 1.2 17.3 1.2 17.5 0.9 Girls 18.0 1.3 16.8 1.2 17.3 0.9 Boys 17.6 1.3 17.8 1.5 17.7 1.0 Midline Standard 2 23.6 1.8 22.6 1.2 23.0 1.0 Girls 23.8 1.8 22.4 1.2 23.0 1.0 Boys 23.5 1.8 22.8 1.2 23.1 1.0 Standard 4 16.7 1.4 16.8 1.0 16.8 0.8 Girls 16.8 1.4 16.8 1.0 16.8 0.8 Boys 16.7 1.4 16.8 1.0 16.8 0.8 Overall 22.0 1.4 22.0 1.0 22.0 0.8 Girls 22.1 1.4 21.8 1.0 21.9 0.8 Boys 21.9 1.4 22.1 0.9 22.0 0.8 Baseline Head Teacher Questionnaire 2013 and Midline Head Teacher Questionnaire 2015. Learner Dropouts Learner dropouts, from baseline to midline, tended to decrease in comparison schools, but increase in treatment schools for reasons unknown to SI (Figure 3). At baseline, dropouts were lower in treatment schools than in comparison. But, dropouts decreased from baseline to midline in comparison schools, while they increased in treatment schools. Further, dropouts were lower in treatment schools than in comparison schools in Standard 2, but higher than comparison schools in Standard 4. These trends need to be examined further through qualitative inquiry for reasons and to find out if EGRA had a role to play in minimizing this. Figure 3: Average Dropout Rates Baseline Head Teacher Questionnaire 2013 and Midline Head Teacher Questionnaire 2015. 0.0 2.0 4.0 6.0 8.0 10.0 12.0 Standard 2 Girls Standard 2 Boys Standard 4 Girls Standard 4 Boys Baseline Comparison Treatment 0.0 2.0 4.0 6.0 8.0 10.0 12.0 Standard 2 Girls Standard 2 Boys Standard 4 Girls Standard 4 Boys Midline Comparison Treatment 31 Learner Age There are no notable differences in average age of learners from baseline to midline, across comparison and treatment schools. Also, evaluators did not find a notable difference when disaggregating average age by gender. In Standard 2, the average age of learners was about 9; and in Standard 4, the average age was about 11. Figure 4: Average Age of Learners Baseline Learner Questionnaire 2013 and Midline Learner Questionnaire 2015. Overage Learners Learners are considered overage if they are two years older than the expected age for their standard. Based on national norms, the assessment team expected Standard 2 learners to be between 7 and 8, and Standard 4 learners to be between 9 and 10. As such, Standard 2 learners were overage if they were 10 or older, and Standard 4 learners were overage if they were 12 or older. Overall, the percentage of learners above the average age for their standard decreased from baseline to midline in both Standard 2 and Standard 4. As shown in Table 12, however, when disaggregated by standard and gender, the percentage of learners who were overage slightly increased for girls from baseline to midline in both standards, but decreased for boys. 0% 5% 10% 15% 20% 25% 30% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Standard 2 Baseline Standard 2 Midline Standard 4 Baseline Standard 4 Midline 32 Table 12: Percentage of Students Overage for Their Grade COMPARISON TREATMENT OVERALL Baseline Standard 2 24.3 22.2 23.1 Girls 10.7 8.5 9.4 Boys 13.6 13.8 13.7 Standard 4 21.1 21.7 21.5 Girls 9.0 8.6 8.8 Boys 12.0 13.2 12.7 Midline Standard 2 24.0 21.5 22.5 Girls 10.5 9.0 9.6 Boys 13.5 12.6 12.9 Standard 4 21.0 20.0 20.4 Girls 8.7 9.1 8.9 Boys 12.3 11.0 11.5 Baseline Learner Questionnaire 2013 and Midline Learner Questionnaire 2015. Extra Length of School Day Less than 40 percent of comparison schools at midline and more than 95 percent of treatment schools at midline reported that their school day had been extended by an hour. The extensions were implemented by EGRA, MTPDS, or another organization.13 Figure 5: Has Anyone Added an Hour to School Day? Midline Head Teacher Questionnaire 2015. 13 This question was not asked explicitly at baseline; thus, this figure only captures midline schools. However, schools at baseline were asked what type of support MTPDS provided, with an extended school day being an option. Of the 310 head teachers interviewed at baseline, 26% reported that their schools’ day lengths were extended by MTPDS. As this is not a direct comparison to the way the midline question is asked, this baseline figure is not presented alongside the midline. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Yes No COMPARISON Standard 2 Standard 4 Overall 33 Midline Head Teacher Questionnaire 2015. Community Involvement in Schools Across baseline and midline, nearly all teachers reported that their schools have a PTA across treatment and comparison schools (Table 13). Frequency of PTA meetings increased from baseline to midline across both groups (Figure 6). Treatment schools that have PTA meetings at least monthly increased from 39 percent to 50 percent from baseline to midline; and PTA meetings at least monthly in comparison schools increased from 40 percent to 49 percent from baseline to midline. Further, more schools at baseline had PTAs that never met or met on an as-needed basis than schools at midline; in 31 percent of baseline comparison schools and in 29 percent of baseline treatment schools’ PTAs never met or met as needed, while only 18 percent and 14 percent, respectively, of midline comparison and treatment schools’ PTAs never met or met as needed. While head teachers at baseline were not asked whether they had a school committee, they were asked at midline and all the teachers reported that their schools have School Management Committees (SMCs). When they were asked about frequency of SMC meetings, treatment school head teachers reported that SMCs met only slightly more often than SMCs in comparison schools’, with 48 percent and 44 percent in treatment and comparison schools, respectively, reporting that their SMCs met monthly or more. Only less than 2 percent of treatment and comparison schools reported that SMCs never met (Table 13). Teachers were also asked if they met with parents outside of the PTA and SMC at both baseline and midline, and if so, how often (Table 13, Figure 7) At baseline, more than 75 percent of teachers in comparison schools and 72 percent of teachers in treatment schools reported that they met with parents. At midline, 73 percent of teachers in comparison and treatment schools met with parents outside of the PTA and SMC. When asked about frequency of meeting with parents, teachers met with parents more often at midline than at baseline in both treatment and comparison schools. Teachers meeting with parents four or more times per school year increased from 16 percent in baseline to 32 percent at midline in treatment schools and from 19 percent to 29 percent in comparison schools. 0% 20% 40% 60% 80% 100% 120% Yes No TREATMENT Standard 2 Standard 4 Overall 34 Table 13: Community Involvement BASELINE MIDLINE Comparison Treatment Comparison Treatment PTA 99 96 96 98 School Committee N/A N/A 100 100 Parents Invited to Class 75 72 73 73 Other Community Involvement 55 61 45 45 Baseline Teacher Questionnaire 2013 and Midline Teacher Questionnaire 2015 Figure 6: Frequency of PTA Meetings Baseline Teacher Questionnaire 2013 and Midline Teacher Questionnaire 2015. 0% 10% 20% 30% 40% 50% 60% 1-4 times a month Every 2-4 months Once or twice a year Never/as needed Baseline Comparison Treatment 0% 10% 20% 30% 40% 50% 60% 1-4 times a month Every 2-4 months Once or twice a year Never/as needed Midline Comparison Treatment 35 Figure 7: Frequency That Teachers Meet with Parents Baseline Teacher Questionnaire 2013 and Midline Teacher Questionnaire 2015. Donor Support Nearly all (99 percent) head teachers in treatment schools reported that their schools have received support from EGRA at midline, which is to be expected, as these are the schools where the EGRA intervention is taking place. But, 36 percent of head teachers in comparison schools said that their schools have received support from EGRA.14 SI does not believe this is due to spillover effects but instead to head teachers misunderstanding what EGRA is. In fact, when SI looked at EGRA implementation fidelity, there did not appear to be any EGRA implementation effects in comparison schools. 14 This high percentage of comparison schools reporting that they received EGRA support is likely due to head teachers misunderstanding where their support came from, given that all donor and MoEST activities focused on improving reading in primary school are often considered early grade reading activities, especially the MTPDS project. Thus, head teachers may have reported receiving support from EGRA when they actually received support from another donor or the MoEST. 0% 5% 10% 15% 20% 25% 30% 35% 40% Never Once per school year Twice per school year Three times per school year Four or more times per school year Baseline Comparison Treatment 0% 5% 10% 15% 20% 25% 30% 35% 40% Never Once per school year Twice per school year Three times per school year Four or more times per school year Midline Comparison Treatment 36 Head teachers and teachers who reported receiving support from EGRA at midline were also asked about what types of support they received. In treatment schools, head teachers reported that the top types of support they received included those that worked to increase parent involvement, extended the length of reading lessons, and provided teacher trainings, as shown in Figure 8. Head teachers perceived that these interventions resulted in learners’ being able to read and learn better, in better quality teaching, and in more resources for learners and teachers, as shown in Figure 9. No treatment schools reported that EGRA support did not provide any benefit. Figure 8: What Type of Support Has EGRA Provided? Midline Head Teacher and Teacher Questionnaire 2015. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Textbooks for class Textbooks for home Help with lesson plans Parent involvement Extended length of school day Extended length of reading lessons Provided teacher training Provided other teachers with training Provided teacher with coaching Signed MOU with EGRA Received grants or inventive money Other Treatment Schools 37 Figure 9: Effect of EGRA Midline Head Teacher Questionnaire 2015. Teachers at midline were also asked if EGRA provided them with coaching. 54 percent of teachers in treatment schools reported receiving coaching from EGRA, as shown in Figure 10. Teachers who received EGRA coaching were then asked to rate the coaching on a scale of 1 to 5, with 1 being least useful and 5 being most useful. A total of 61 percent of teachers reported that the coaching was a 5 (most useful), while only 2 percent of treatment school teachers rated the coaching at 1 (least useful). Figure 10: Has EGRA Provided You with Coaching? Midline Teacher Questionnaire 2015. More than half of head teachers across comparison and treatment schools at midline reported receiving support from other individuals, organizations, or businesses, as shown in Figure 11. Treatment school 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% It didn't benefit the school at all Better facilities More resources for teachers More resources for learners More motivation on the part of staff More motivation on the part of learners Better quality teaching Longer school day Learners are able to read better Learners are able to learn better in other learning areas Learners are getting better scores on their tests Better or more regular attendance Treatment Schools Yes 54% No 46% TREATMENT SCHOOLS 38 head teachers noted receiving about the same amount of support as comparison school head teachers, at 56 percent and 57 percent, respectively. When asked about what specific donors their schools received support from, head teachers in treatment schools reported that the majority of donor support came from World Vision, Forum for African Women Educationists in Malawi (FEWEMA), and the United Nations Children’s Fund (UNICEF), as shown in Figure 12. Head teachers in comparison schools reported that the majority of their donor support came from these three sources, along with Save the Children (through Tiana, a USAID-funded activity and Literacy Boost). Figure 11: Have Other Individuals, Organizations, or Businesses Provided Support? Midline Head Teacher Questionnaire 2015. Figure 12: Other Donor Support Midline Head Teacher Questionnaire 2015. Yes 57% No 43% COMPARISON Yes No Yes 56% No 44% TREATMENT Yes No 0% 5% 10% 15% 20% 25% 30% 35% Concern International DFID FAWEMA Mary's Meals Plan Malawi Save the Children UNICEF World Vision Yoneco Other Comparison Treatment Overall 39 SAMPLE HOUSEHOLD AND LEARNER FEATURES Midline results are shown below and compared with baseline wherever data were available and relevant. The findings below are obtained from the household and learner surveys, and the survey primarily focused on gathering information on the learning environment at home to help explain learner reading outcomes. Access to Reading Materials at Home As shown in Figure 13, at midline less than half of the households reported that learners had access to reading materials at home. But more learners from households in the treatment group (42 percent) had access to reading materials than learners in the comparison group (34 percent). This question was not asked at baseline to compare with midline results. Figure 13: Does Learner Have Access to Reading Materials at Home? Midline Household Questionnaire 2015. As shown in Figure 14, learners in treatment schools were more likely to take books home across Standard 2 and Standard 4 than those in comparison schools, and learners in Standard 2 were even more likely than learners in Standard 4. Of those learners who reported that they do take books home, more than 95 percent of Standard 4 learners said that they used them across treatment and comparison schools. In Standard 2, 87 percent and 83 percent of learners in treatment and comparison schools, respectively, reported that they used the books they took home. This question was not asked at baseline to compare with midline results. 34% 42% 39% 66% 58% 61% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Comparison Treatment Overall Yes No 40 Figure 14: Do Learners Take Books Home? If Yes, Do They Use Them? Midline Learner Questionnaire 2015. Midline Learner Questionnaire 2015. Learner Receives Help at Home As shown in Figure 15, according to learners at midline, slightly more girls in Standard 2 were read to at home of learners from treatment schools versus comparison schools, but no difference was noticed for boys. For Standard 4, boys were read to at home equally (73 percent) in both treatment and comparison schools at midline, but more girls in treatment (73 percent) than in comparison schools (67 percent) were read to at home. At baseline, the differences between learners in Standard 2 and Standard 4 was of much greater magnitude - less than 45 percent of girls and boys in Standard 2 were read to at home, while this figure was over 75 percent for girls and boys in Standard 4. The results were similar across treatment and comparison schools. So even though the percentage of learners being read to at home increased at midline for Standard 2, it decreased slightly for learners in Standard 4 across comparison and treatment schools. Figure 15: Household Member Reads to Learner Midline Learner Questionnaire 2015. 0% 10% 20% 30% 40% 50% 60% 70% 80% Standard 2 Standard 4 Overall DO YOU TAKE BOOKS HOME? Comparison Treatment 75% 80% 85% 90% 95% 100% Standard 2 Standard 4 Overall IF YES, DO YOU USE THEM? Comparison Treatment 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Girls Boys Girls Boys Standard 2 Standard 4 Comparison Treatment 41 As shown in Figure 16, about 60 percent of learners reported receiving help with their homework from a household member, with slightly more learners in treatment schools receiving help than learners in comparison schools. However, these were higher at baseline where 72 percent and 76 percent of learners in comparison and treatment schools, respectively, reported receiving help with their homework from a household member. Figure 16: Does a Household Member Help with Homework? Midline Learner Questionnaire 2015. As shown in Figure 17, at midline when household members were asked to elaborate on how they helped learners at home, the majority responded that they encouraged the child to read (more than 65 percent gave this answer across comparison and treatment groups). This decreased from baseline, though, where more than 68 percent of households said that they encouraged the child to read. More than 35 percent of household members at midline across comparison and treatment groups responded that they worked to communicate high expectations to their child. This figure increased from 28 percent at baseline. Finally, 17 percent and 20 percent at midline bought or borrowed books for their child to read in comparison and treatment schools, respectively. This also increased from baseline, especially in the treatment group (14 percent at baseline). 59% 60% 60% 41% 40% 40% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Comparison Treatment Overall Yes No 42 Figure 17: How Household Member Helps Learner Midline Household Questionnaire 2015. Importance of Learner Going to School As shown in Figure 18, nearly all heads of households (more than 96 percent) reported that it is important for girls and boys to go to school, with almost no difference across comparison and treatment schools or between girls and boys. This was an increase from baseline, where about 90 percent of households reported that it was important for girls and boys to go to school across treatment and comparison groups. Figure 18: How Important is it for Learner to Go to School? Midline Household Questionnaire 2015. 0% 10% 20% 30% 40% 50% 60% 70% 80% Help with homework Buy/borrow books for them to read Take them to library Take them to a reading event Talk with teacher/head teacher about learning progress Participate in PTA Participate in School Committee Regularly read to child Encourage child to read Communicate to child high expectations Don't know Refuse to respond Comparison Treatment Overall 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Girls Boys Comparison Treatment Overall 43 Household involvement in school As shown in Figure 19, the majority (79 percent) of household members at midline reported that they were involved in their children’s school, with no variance across comparison and treatment schools. When compared to baseline, this figure slightly increased in comparison schools, but remained the same in treatment schools. Figure 19: Is Household Member Involved in Learner's School? Midline Household Questionnaire 2015. CHARACTERISTICS OF SAMPLE TEACHERS OBSERVED AT MIDLINE The data were obtained from classroom observations and the teacher and head teacher questionnaires, which aimed to measure teacher practices in the classroom to explain learning outcomes for learners. Midline results are discussed below and compared with baseline wherever relevant and data were available. Enumerators observed a total of 1,826 teachers at midline through the classroom observation tool. Of those teachers, 44 percent were female and 56 percent male; and half of the observed classes were Standard 2 with the other half being Standard 4. Further, 51 percent and 49 percent of the classes observed were of Chichewa reading classes and English classes, respectively. These figures did not vary when disaggregated by teacher gender. Teacher Behavior Enumerators were instructed to observe many aspects of teacher behavior and record their observations. As can be seen in Figure 20, enumerators reported that the majority of teachers at midline exhibited all aspects of observed behavior except for assessing learners’ learning, where just under half of teachers across comparison and treatment schools were observed to have exhibited this behavior. All teachers across comparison and treatment schools were observed to have treated all learners equally, while 100 percent and 99 percent of teachers demonstrated effective classroom management skills in treatment and comparison schools, respectively. Further, all teachers in treatment schools and 97 percent of teachers in comparison schools were observed to have met at least 67 percent of the essential teaching practices, 79% 79% 79% 21% 21% 21% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Comparison Treatment Overall Yes No 44 and 47 percent and 42 percent of teachers in treatment and comparison schools, respectively, were observed to have met all thirteen essential teaching practices at midline. Teacher practices improved at midline from baseline in all of the above categories except for using a lesson plan, where 65 percent and 73 percent of teachers used a lesson plan at baseline in comparison and treatment schools, respectively. Teachers improved notably at midline in providing learners with opportunities to develop critical thinking skills (from 40 percent at baseline to 91 percent in midline in treatment schools), effectively using a variety of instructional resources (from 55 percent in baseline to 95 percent in midline in treatment schools), and asking open-ended questions to encourage learning (from 55 percent in baseline to 99 percent in midline in treatment schools). Figure 20: Teacher Behavior Midline Classroom Observation 2015. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Uses lesson plan Connects lesson plan to what learners have previously learned Introduces lesson with advance organizer Manages instructional time effectively Demonstrates effective classroom management skills Effectively uses different instructional resources and strategies Treats all students equally/fairly Engages learners in cooperative learning strategies Asks open-ended questions to encourage thinking Provides learners with structured opportunities to apply skills Provides opportunities for learners to develop critical thinking skills Uses appropriate learning materials besides textbooks Assesses learner learning Met at least 67% of practices Met all essential practices Comparison Treatment Overall 45 VI. MIDLINE FINDINGS: READING ASSESSMENT This section presents the following: 1. Overall reading assessment (RA) midline results of all learners assessed on the Chichewa RA across the 10 sample districts, and proportion of Standards 2 and 4 learners attaining MoEST and EGRA-Coordinating-Committee-recommended benchmarks along with changes in learning since the baseline to show whether any improvements have occurred since EGRA implementation to address the Task 1 evaluation questions of this IE study. 2. Analysis of which household, school, and community factors are correlated with midline learner reading outcomes to address this study’s Task 2 evaluation questions. Are they different from baseline? 3. A comparison of the change in oral reading fluency scores between baseline and midline in treatment and comparison groups to examine program effects. SUMMARY At midline, the performance was better in treatment schools relative to comparison schools in terms of average scores and proportion of learners meeting benchmarks set for Reading Fluency and Comprehension. Also, percentages of learners scoring zero in both standards were lower in the treatment group relative to the comparison group. However, when compared to baseline, performance across nearly all subtasks and standards at midline dropped considerably in both treatment and comparison groups of schools. The exceptions to this trend were in the syllable segmentation and syllable reading subtasks, in which the treatment group slightly improved from the baseline to midline, while the comparison group showed a decrease in performance. Also, percentage of learners scoring zero in Standard 2 increased in comparison schools for all nine subtasks while it was noticed in treatment schools for only four subtasks, especially for syllable reading and oral reading comprehension. In Standard 4, percentage of learners scoring zero increased from baseline to midline in both treatment and comparison schools for all subtasks, except listening comprehension and syllable segmentation that are related to pre-reading stage. As a result of the low reading performance on lower-level tasks, as discussed later, at midline about 1 percent of learners in Standard 2 were able to read grade-level text by the end of Standard 2, and 0 percent of learners were able to read with comprehension according to the benchmark established to be achieved by 2018. For Standard 4, nearly 8 percent of learners met the Oral Reading Fluency benchmark, and 5 percent of learners were able to comprehend 80 percent of the Reading Comprehension questions in treatment schools. But, it was 7 and 4 percent, respectively, for Oral Reading Fluency and Reading Comprehension in comparison schools for the proportion of learners attaining benchmarks. Baseline results were slightly higher for Oral Reading Fluency in that it was 1.2 percent for Standard 2 and 10 percent for Standard 4 by 2014 benchmark. But, baseline results were similar in both standards for proportion of learners attaining 2014 benchmark for Reading Comprehension (0 percent in Standard 2 and 5 percent in Standard 4). As described in the EGRA Implementation Section of this report, Standard 4 learners had only benefitted from school and community MOUs and schools-level GUCs, and not from new materials or trained teachers at the time of this midline. As such, evaluators do not expect large changes in Standard 4 results. 46 The results also appear to indicate that the declining trend in scores could plausibly be linked to factors such as increases from baseline to midline in learner-to-teacher ratios, and decreases from base to midline in learners receiving help with their homework from a household member and in households encouraging the child to read, although there were improvements in teacher practices and length of school day at lower standards. Also, the declining trend appears to be national because scores have declined from base to midline similarly across districts. However, the decline is typically less so in treatment schools than in comparison schools, meaning that the decline is not necessarily associated with the EGRA intervention. Learners with Zero Scores at Midline The percentage of zero scores at midline for Standards 2 and 4 are presented in Figure 21, and base and midline are shown in Table 14. At midline, the percentage of zero scores for all subtasks and standards were lower in the treatment group relative to the comparison group. However, as indicated in Figure 21 and Table 14 for Standard 2, as the task difficulty increased, so did the percentage of zero scores in both treatment and comparison groups. For instance, the scores progressively increased from 11.9 percent for the comparison group and 8.4 percent for the treatment group in the Listening Comprehension subtask to 98.4 percent for the comparison group and 95.6 percent for the treatment group in the Reading Comprehension subtask. The exception to this upward trend was for Letter Name Recognition, which implies that learners were more able to read letters than they were able to recognize sounds, syllables, or words. In Standard 4, the subtasks with the highest percentage of zero scores were Initial Sound Identification and Reading Comprehension. While a higher percentage of Standard 4 learners compared to Standard 2 learners could recognize at least one letter or word, 70 percent of learners in the comparison group and nearly 60 percent in the treatment group could not recognize one initial sound of a word. More than half of the learners (62.8 percent in the comparison group and 55.8 percent in the treatment group) were unable to read a grade-level passage fluently with comprehension, which is likely correlated with their scores on the Initial Sound Identification. Learners scoring zero on both tasks are likely deficient in using reading strategies to interpret meaning from text (e.g., decoding, inferring, predicting, etc.). Relative to baseline, however, percentage of learners scoring zero in Standard 2 increased in comparison schools for all nine subtasks while it was noticed in treatment schools for only four subtasks, especially for syllable reading and oral reading comprehension. In Standard 4, percentage of learners scoring zero increased from baseline to midline in both treatment and comparison schools for all subtasks, except listening comprehension and syllable segmentation, which are related to pre-reading stage. It is to be noted that a comparison of the characteristics of learners scoring zero with those above zero showed that there were no marked differences in learners’ age, language spoken by the learner with friends, in learner taking and using books at home, and in learner household’s involvement in school. Although, the proportions of learners having access to learning materials in the house, learner getting help from household members to do homework, and learners having some household members reading to them were slightly higher among those that scored above zero than those that scored zero. Overall, though, the learners scoring zero were not considerably different from those that scored above zero at both base and midline. 47 Table 14: Percentage of Students with Zero Scores, Baseline and Midline SUBTASKS BASELINE MIDLINE Standard 2 Standard 4 Standard 2 Treatment Standard 2 Comparison Standard 4 Treatment Standard 4 Comparison Listening comprehension < 5 < 5 8.4 11.9 2.1 2.6 Syllable segmentation 30 19 30.9 38.8 13.7 16.4 Initial sound identification 69 58 64.5 78.5 58.1 71 Letter name recognition 41 8 33.2 48.1 8.1 9.7 Syllable reading 59 12 66 77.9 16.4 21 Familiar word reading 70 15 69.6 79.9 15.6 21.5 Non-word reading 76 16 74 83.5 19.3 25.4 Oral reading fluency 76 19 73.6 84.4 19.3 25.7 Oral reading comprehension 86 28 95.6 98.4 55.4 62.8 Baseline Early Grade Reading Assessment 2013 and Midline Early Grade Reading Assessment 2015. 48 Figure 21: Percent Zero Scores at Midline Midline Early Grade Reading Assessment 2015. READING ASSESSMENT FINDINGS BY SUBTASKS This section reports EGRA results by subtasks and is organized based on the three stages of reading developments shown in Annex 2. SI presents reading assessment results for both the treatment and comparison groups, and against the benchmarks for reading established by the MoEST in December 2014 to be reached by 2018 for some subtasks, and by benchmarks recommended by the EGRA Coordinating Committee in 2011 for others (see earlier discussion on benchmarks in the report).15 15 All data were weighted, and midline data were calibrated using a conversion factor to equate midline with baseline EGRA tool. 11.9% 8.4% 38.8% 30.9% 78.5% 64.5% 48.1% 33.2% 77.9% 66.0% 79.9% 69.6% 83.5% 74.0% 84.4% 73.6% 98.4% 95.6% 0.0% 25.0% 50.0% 75.0% 100.0% Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Listening Comprehens ion Syllable Segmentatio n Initial Sound Identification Letter Name Recognition Syllable Reading Familiar Word Reading Non-Word Reading Oral Reading Fluency Oral Reading Comprehens ion STANDARD 2 2.6% 2.1% 16.4% 13.7% 71.0% 58.1% 9.7% 8.1% 21.0% 16.4% 21.5% 15.6% 25.4% 19.3% 25.7% 19.3% 62.8% 55.4% 0.0% 25.0% 50.0% 75.0% 100.0% Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Control Treatment Listening Comprehens ion Syllable Segmentatio n Initial Sound Identification Letter Name Recognition Syllable Reading Familiar Word Reading Non-Word Reading Oral Reading Fluency Oral Reading Comprehens ion STANDARD 4 49 STAGE 1: PRE-READING SKILLS In this section, learners’ oral language and phonemic awareness skills are assessed through three subtasks: listening comprehension, syllable segmentation, and initial sound identification. The average scores at baseline and midline for the treatment and comparison groups in Standards 2 and 4 on these pre-reading subtasks are illustrated in Table 15, Table 17 and Table 19. In Table 16, Table 18 and Table 20 the benchmark and percent meeting the benchmark at midline are provided for all sampled learners in Standards 2 and 4. Listening Comprehension: Midline Results Compared to Baseline As shown in Table 15, for Standard 2, the average scores in correct responses for the comparison group declined from 2.85 at baseline to 2.67 at midline. But, the scores for the treatment group improved considerably from 2.85 at baseline to 2.98 at midline. For Standard 4 learners, while the average score remained almost the same across time for the comparison group (3.6 at base and 3.7 at midline), it improved considerably in the treatment group, from 3.7 at baseline to 3.84 at midline. Table 15: Listening Comprehension Midline Scores Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 2.85 -0.06 1693 2.67 -0.08 Treatment 2139 2.85 -0.09 3110 2.98 -0.06 Standard 4 Comparison 2268 3.6 -0.05 1678 3.66 -0.07 Treatment 2184 3.7 -0.06 3101 3.84 -0.06 * Avg = weighted mean number of correct responses, N = sampled number of learners; SE = Standard errors. Baseline Early Grade Reading Assessment 2013 and Midline Early Grade Reading Assessment 2015. Listening Comprehension: Percent Meeting Benchmarks The benchmark for Standard 2 is 60 percent, which means learners should be able to answer 3 out of 5 listening comprehension questions correctly. As Table 16 illustrates, for Standard 2, while the treatment group scored an average of 60 percent, meeting the benchmark, the average in the comparison group was 7-percentage points below the benchmark (53 percent answered 3 out of 5 correctly compared to 60 percent benchmark). But, more than half of the learners in both the treatment and comparison groups are meeting the listening comprehension benchmark in Standard 2. On average, 54 percent of learners in the comparison group met the benchmark compared to 58 percent in the treatment. The benchmark for Standard 4 is 80 percent, indicating that learners should correctly respond to 4 out of 5 comprehension questions. Table 16 shows that 52 percent of learners in the comparison group and 56 percent of learners in the treatment group met the benchmark. Thus, more than half of learners met the benchmark for listening comprehension. The average in the treatment and comparison groups was 5- percentage points below the benchmark, averaging 73 percent for the comparison group and 77 percent for the treatment group. 50 Table 16: Percentage Reaching Benchmarks at Midline — Listening Comprehension SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 60% (3 out of 5) 53% 54% Treatment 60% (3 out of 5) 60% 58% Standard 4 Comparison 80% (4 out of 5) 73% 52% Treatment 80% (4 out of 5) 77% 56% Syllable Segmentation: Midline Results Compared to Baseline The midline results for Standard 2 illustrated in Table 17 indicate that average scores in correctly segmenting syllables decreased from baseline to midline for the comparison schools – from 4.57 to 4.38. But, the scores improved for treatment schools over time - from 4.87 at baseline to 5.14 at midline. As for students in Standard 4, the difference between syllable segmentation average scores from baseline to midline was marginal for the treatment group (6.69 to 6.7), while scores slightly improved in comparison schools (from 6.13 to 6.33). Table 17: Syllable Segmentation Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 4.57 -0.15 1684 4.38 -0.22 Treatment 2139 4.87 -0.19 3110 5.14 -0.17 Standard 4 Comparison 2268 6.13 -0.18 1693 6.33 -0.15 Treatment 2184 6.69 -0.18 3101 6.7 -0.11 *Avg = weighted mean number of correct responses, N = sampled observations Syllable Segmentation: Percent Meeting Benchmarks The benchmark for Standard 2 is 70 percent. To achieve this benchmark, learners must be able to segment 7 out of 10 syllables. The midline results presented in Table 18 show that 43 percent of learners in the comparison group and 47 percent in the treatment group were able to achieve the target. The average in the comparison group was 26-percentage points below the benchmark (44 percent compared to 70 percent benchmark), while in the treatment group, the average was 19-percentage points below the benchmark (51 percent compared to 70 percent benchmark). The syllable segmentation benchmark for Standard 4 is 80 percent. According to Table 18, 49 percent of learners in the comparison group and 55 percent of learners in the treatment group met the benchmark. Hence, approximately half of learners in both groups were able to segment 8 out of 10 syllables. On average, learners in the comparison group were 17-percentage points below the benchmark (63 percent compared to 80 percent benchmark), while learners in the treatment group were 13-percentage points below (67 percent compared to 80 percent benchmark). Table 18: Percentage Reaching Benchmarks — Syllable Segmentation SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 70% (7 out of 10) 44% 43% Treatment 70% (7 out of 10) 51% 47% Standard 4 Comparison 80% (8 out of 10) 63% 49% Treatment 80% (8 out of 10) 67% 55% 51 Initial Sound Identification: Midline Results Compared to Baseline As can be seen in Table 19, Standard 2 learners in both the comparison and treatment groups could identify at least one initial sound during the baseline assessment (1.12 and 1.39, respectively). At midline, both groups scored lower than the baseline with 0.55 in comparison and 1.27 in treatment group. Scores at midline in comparison schools were less than half of baseline scores while the drop from base to midline was modest in the treatment schools. Again, for Standard 4, the average scores at baseline were higher than at midline for both treatment and comparison groups. In both groups, the midline scores were less than half of that of baseline, indicating a sharp drop in scores across time (drop from 1.92 to 0.6 in comparison group, drop from 2.33 to 1.28 in treatment schools). Table 19: Initial Sound Identification Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 1.12 -0.11 1684 0.55 -0.09 Treatment 2139 1.39 -0.23 3110 1.27 -0.09 Standard 4 Comparison 2268 1.92 -0.19 1693 0.6 -0.06 Treatment 2184 2.33 -0.23 3101 1.28 -0.09 *Avg = weighted mean number of correct responses, N = sampled observations Initial Sound Identification: Percent Meeting Benchmarks The benchmark for initial sound identification assumes that children should be able to identify 8 out of 10 initial sounds correctly. As shown in Table 20, only 3 percent of learners in the comparison group and 4 percent in the treatment group were able to meet this benchmark. On average, learners in the comparison group could identify 6 percent of initial sounds, which is 74-percentage points below the 80 percent benchmark. In comparison, learners in the treatment group identified 13 percent of initial sounds, which is approximately twice as many as the comparison group, but still 67-percentage points lower than the 80 percent benchmark. While the benchmark for Standard 4 is 10 percent higher (90 percent compared to 80 percent) than Standard 2, the percent meeting the benchmark was similar to Standard 2, and average scores were on par with Standard 2 scores. The midline results presented in Table 20 indicate that only 3 percent of learners in the comparison group and 5 percent in the treatment group were able to achieve the benchmark. The average score in the comparison group was 6 percent, which is 83-percentage points below the 90 percent benchmark, while the treatment group scored an average of 13 percent, representing a 77-percentage point gap between the midline results and the benchmark. Table 20: Percentage Reaching Benchmarks — Initial Sound Identification STANDARD SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 80% (8 out of 10) 6% 3% Treatment 80% (8 out of 10) 13% 4% Standard 4 Comparison 90% (9 out of 10) 6% 3% Treatment 90% (9 out of 10) 13% 5% 52 STAGE 2: INITIAL READING SKILLS The initial reading subtasks measure learners’ ability to recognize letters, syllables, and familiar sight words as well as to decode unfamiliar (invented) non-words. Table 21, Table 23, Table 25, and Table 27 show the results of learner performance at baseline and midline for treatment and comparison groups on the initial reading subtasks. In Table 22, Table 24, Table 26, and Table 28 benchmark and percent meeting the benchmark is provided for all sampled learners in Standards 2 and 4. Letter Name Recognition: Midline Results Compared to Baseline On the Letter Name Recognition subtask for Standard 2 at baseline, there was a 0.02 correct letters per minute (clpm) difference in performance between the treatment group and comparison group. The midline results for Standard 2 illustrated in Table 21 indicate that the comparison group dropped by 2.5 clpm, from correctly reading 8.6 letters at baseline to 6.1 at midline. The treatment group improved by 2.9 points, reading an average of 8.5 letters correct at the baseline compared to 11.4 correct letters per minute at midline. Overall, the difference in scores between midline and baseline was large, as shown in Table 21. Standard 4 learners had the same mean scores at baseline (both groups scored read approximately 38 clpm correctly). Both groups’ mean scores dropped from the baseline to midline. The comparison group dropped 10 clpm to identifying 28 clpm at midline. Meanwhile, the treatment group read 38 clpm at baseline compared to 30 at midline. Similar to Standard 2, the difference in Standard 4 scores from midline to baseline was more than 20 percent. Table 21: Letter Name Recognition Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 8.58 -0.59 1684 6.09 -0.6 Treatment 2139 8.54 -0.9 3110 11.4 -0.59 Standard 4 Comparison 2268 37.86 -0.96 1693 27.79 -0.89 Treatment 2184 38.4 -1.41 3101 30.16 -0.72 * Avg = weighted mean correct letters per minute, N = sampled observations Letter Name Knowledge: Percent Meeting Benchmarks To meet the Standard 2 benchmark, learners must be able read 24 correct letters per minute (clpm). As shown in Table 22, on average, learners in the comparison group could read 6.1 clpm, while learners in the treatment group read 11.4 clpm. Overall, 9 percent of learners in the comparison group and 15 percent in the treatment group met the benchmark. These findings indicate that the majority of learners have not learned the letter names of the Chichewa alphabet. The Standard 4 benchmark is 50 correct letters per minute (clpm). Learners in the comparison group read an average of 28 clpm, while those in the treatment read 30 clpm. Overall, 21 percent of learners in the treatment and 26 percent in the comparison group met the benchmark, indicating that the majority of Standard 4 learners have not developed the initial reading skill of letter name knowledge. 53 Table 22: Percentage Reaching Benchmarks — Letter Name Knowledge STANDARD SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 24 clpm 6.1 clpm 9% Treatment 24 clpm 11.4 clpm 15% Standard 4 Comparison 50 clpm 27.8 clpm 21% Treatment 50 clpm 30.2 clpm 26% Syllable Reading: Midline Results Compared to Baseline Overall, there was a decline in performance on the syllable-reading subtask for both Standard 2 and Standard 4 learners, as shown in Table 23. In Standard 2, the comparison group scored 2.2 correct syllables per minute (cspm) lower than the baseline (mean score of 3.1 at midline compared to 5.3 at baseline), while the treatment group decreased by almost 1 cspm (from a mean score of 6.9 at baseline to a mean score of 6 at the midline). Standard 4 trends were similar to those in Standard 2. Learners in both the comparison and treatment groups scored lower at midline than at the baseline. The comparison group dropped from a mean score of 36 at baseline to 27 at midline, while the treatment group fell from an average of 37 syllables read at baseline to 30 at midline. Table 23: Syllable Reading Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 5.31 -0.53 1684 3.12 -0.48 Treatment 2139 6.88 -1.12 3110 5.97 -0.46 Standard 4 Comparison 2268 35.69 -1.16 1693 27.07 -0.93 Treatment 2184 36.97 -1.48 3101 29.74 -0.83 * Avg = weighted mean correct syllables per minute, N = sampled observations Syllable Reading: Percent Meeting Benchmarks The syllable reading benchmark for Standard 2 is 50 correct syllables per minute (cspm). As illustrated in Table 24, less than 1 percent of learners in the comparison group and 1 percent of those in the treatment group achieved the benchmark. The comparison group was able to read an average of 3.1 syllables per minute, 46.9 syllables below the benchmark, while the treatment group read an average of 6 syllables per minute, scoring 44 syllables below the benchmark. In Standard 4, the syllable reading benchmark is 65 correct letters per minute. The midline results presented in Table 24 shows that 9 percent of learners in the comparison group and 10 percent in the treatment group were able to meet the 65 cspm benchmark. The mean score for the comparison group was 27.1 cspm, which was less than half of the benchmark. The treatment group scored an average of 29.7 cspm, 55 percent below the benchmark. Table 24: Percentage Reaching Benchmarks — Syllable Reading STANDARD SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 50 cspm 3.1 cspm 0% Treatment 50 cspm 6.0 cspm 1% Standard 4 Comparison 65 cspm 27.1 cspm 9% Treatment 65 cspm 29.7 cspm 10% 54 Familiar Word Reading: Midline Results Compared to Baseline Learners in both the treatment and comparison groups displayed a decline in their ability to read familiar sight words (Table 25). During the baseline, Standard 2 learners in the comparison group read an average of 3.3 words per minute compared to 2 words per minute at the midline, indicating that they read 1.3 fewer words. The treatment group read an average of 4.3 words per minute during the baseline, but only 3.7 words at midline, representing a decline by 0.6 words. The difference from the baseline is considerable although the decline was steeper in comparison than in treatment schools, as shown in Table 25 (40 percent lower for the comparison group and 16 percent lower for the treatment group). Although there was a decline in scores from baseline in both groups, with larger decline in comparison than in treatment schools, a positive treatment effect could be noted for the subtask. Standard 4 learners also demonstrated a decline at midline from baseline in their ability to read familiar words. Learners in the comparison group could read 3 fewer words at midline, dropping from an average of 24 words per minute during the baseline to reading 21 words correctly at midline. The treatment group fell by 2 points from a mean score of 25 at baseline to a mean score of 23 at the midline. Table 25: Familiar Word Reading Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 3.27 -0.33 1684 1.97 -0.31 Treatment 2139 4.25 -0.65 3110 3.67 -0.32 Standard 4 Comparison 2268 24.24 -0.78 1693 20.89 -0.79 Treatment 2184 25.27 -1.04 3101 23.17 -0.72 * Avg = weighted mean correct words per minute, N = sampled observations Familiar Word Reading: Percent Meeting Benchmarks The Familiar Word Reading benchmark for Standard 2 is 30 correct words per minute (cwpm). According to the midline results in Table 26, one percent of learners in the comparison group and treatment group met this benchmark. Learners in the comparison group read an average of 2.0 words per minute, which is 28 cwpm below the benchmark. The treatment group read an average of 3.7 words per minute, scoring 26 cwpm below the Standard 2 benchmark. In Standard 4, the familiar word reading benchmark is 45 correct words per minute. The midline results in Table 26 showed that 9 percent of learners in the comparison group and 11 percent in the treatment group were able to meet the benchmark. The mean score for the comparison group was 21 cwpm, reaching 47 percent of the benchmark. The treatment group scored an average of 23 cwpm, achieving 51 percent of the benchmark. Table 26: Percentage Reaching Benchmarks — Familiar Word Reading STANDARD SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 30 cwpm 2.0 1% Treatment 30 cwpm 3.7 1% Standard 4 Comparison 45 cwpm 20.9 9% Treatment 45 cwpm 23.1 11% 55 Non-Word Reading: Midline Results Compared to Baseline For the non-word reading subtask, learners in Standard 4 and Standard 2 demonstrated a slight decrease in their ability to decode invented words. As Table 27 indicates, in Standard 2 there was less than a one￾point difference between the baseline and midline for the treatment group, and approximately one-point difference between comparison groups. The comparison group read an average of 2.3 non-words per minute during the baseline and 1.4 non-words at midline (1.1 decrease). The treatment group read an average of 2.9 non-words per minute at baseline and 2.6 words at midline (0.3 decrease). The difference between the midline and baseline was 11 percent lower than baseline, while it was more substantial for the comparison group (39 percent lower than baseline). The difference in scores between midline and baseline for Standard 4 averaged between 2 and 3 points for the comparison and treatment groups. The comparison group read 3 fewer non-words at midline, dropping from an average of 16 non-words per minute during the baseline to 13 non-words per minute at midline. The treatment group read 2 fewer non-words, reading an average of 14 non-words at midline compared to 16 non-words at baseline. The midline scores were 18 percent lower than the baseline for the comparison group and 13 percent lower than baseline for the treatment group. Table 27: Non-Word Reading Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 2.31 -0.26 1684 1.42 -0.22 Treatment 2139 2.93 -0.43 3110 2.62 -0.22 Standard 4 Comparison 2268 15.7 -0.56 1693 12.9 -0.47 Treatment 2184 16.19 -0.74 3101 14.06 -0.39 * Avg = weighted mean correct letters per minute, N = sampled observations Non-Word Reading: Percent Meeting Benchmarks The Standard 2 benchmark for the Non-Word Reading subtask is 15 correct non-words per minute (cnwpm). As indicated in Table 28, 3 percent of learners in the comparison group and 7 percent in the treatment group reached the benchmark. Learners in the comparison group read an average of 1.4 non￾words per minute, which is 13.6 points below the benchmark. The treatment group read twice as many non-words, averaging 2.6 non-words per minutes and scoring 12.4 cnwpm below the benchmark. The Standard 4 Non-Word Reading benchmark is 40 cnwpm. Two percent of learners in both groups met the benchmark. The midline results were very similar across both groups, with the comparison group reading an average of 13 non-words per minute and the treatment group reading an average of 14 cnwpm. Both groups scored an average of 27 cnwpm below the benchmark. Table 28: Percentage Reaching Benchmarks — Non-Word Reading SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 15 cnwpm 1.4 3% Treatment 15 cnwpm 2.6 7% Standard 4 Comparison 40 cnwpm 12.9 2% Treatment 40 cnwpm 14.1 2% 56 STAGE 3: READING FLUENCY AND COMPREHENSION The Reading Fluency and Comprehension subtasks measure learners’ abilities to read with fluency, accuracy, and comprehension. The results of learner performance at the baseline and midline for the treatment and comparison groups on the reading fluency and comprehension subtasks are illustrated in Table 29 and Table 31. As the reading fluency subtask is a timed subtask, the results are presented according to number of words read correctly per minute in a simple story passage. The reading comprehension subtask is based on the number of questions answered correctly from a total of five possible questions. The results for the reading comprehension subtask are reported by the percentage of total correct responses out of the total possible. In Table 30 and Table 32, the benchmark and percent meeting the benchmark are provided for all sampled learners in Standards 2 and 4. Oral Reading Fluency: Midline Results Compared to the Baseline The midline results for Standard 2 and Standard 4 illustrated in Table 29 indicate a drop in performance from baseline to midline for all learners sampled, though with less magnitude in treatment schools. During the baseline, Standard 2 learners in the comparison group could read an average of 3.2 words per minute, compared to 1.8 words per minute at the midline (1.4 difference). The treatment group read an average of 3.9 words per minute during the baseline, but only 3.5 words at midline (0.4 difference). Given that these results mirror the results for the Familiar Word Reading subtask (comparison group baseline: 3.3; comparison group midline: 2.0; treatment group baseline: 4.3; treatment group midline: 3.7), this indicates that learners were likely able to read some familiar sight words, but were not yet able to decode words in isolation or infer meaning from the connected text in a simple story. In Standard 4, the treatment and comparison group read 4-6 fewer words per minute in the midline compared to the baseline results. As Table 29 illustrates, the comparison group read an average of 26 words per minute at baseline, but only 20 at midline (-6 difference). The treatment group had a mean score of 27 at baseline and dropped 5 points to a mean score of 22 at midline. Similar to Standard 2 scores, Standard 4 results largely reflect performance on the familiar word task, also indicating that learners in Standard 4 have trouble decoding unfamiliar words or extracting meaning from a reading passage. Table 29: Oral Reading Fluency Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 3.22 -0.34 1684 1.84 -0.3 Treatment 2139 3.89 -0.71 3110 3.52 -0.3 Standard 4 Comparison 2268 26.21 -0.94 1693 20.28 -0.75 Treatment 2184 26.68 -1.15 3101 22.27 -0.68 * Avg = weighted mean correct letters per minute, N = sampled observations Oral Reading Fluency: Percent Meeting Benchmarks In order to meet the Standard 2 oral reading fluency benchmark, learners must read 30 correct words per minute (cwpm). As shown in Table 30, learners in the comparison group read an average of 1.8 cwpm, while those in the treatment read 3.5 cwpm. One percent of learners in the both the treatment and comparison groups met the benchmark, indicating that the majority of Standard 2 learners were unable to read a simple grade-level passage with fluency and accuracy. 57 The Standard 4 benchmark is 50 cwpm. Learners in the comparison group read an average of 20 cwpm, achieving 40 percent of the benchmark. Meanwhile, the treatment group fluently read 22 cwpm, reaching nearly 50 percent of the benchmark. Overall, 7 percent of the comparison group and 8 percent of the treatment group met the 50 cwpm benchmark. The findings indicate that the majority of Standard 4 learners were unable to read at a sufficient rate (with fluency and accuracy) to comprehend what was read. Table 30: Percentage Reaching Benchmarks — Oral Reading Fluency SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 30 cwpm 1.8 1% Treatment 30 cwpm 3.5 1% Standard 4 Comparison 50 cwpm 20.3 7% Treatment 50 cwpm 22.3 8% Reading Comprehension: Midline Results Compared to Baseline Table 31 and Table 32 show the results for the Reading Comprehension subtasks. Reading Comprehension results are typically correlated with the oral reading fluency results. In other words, if learners are reading fewer words per minute, they will likely be able to comprehend less of what they read. Such is the case with this subtask because across both standards and groups, performance dropped considerably at midline from baseline. Table 31: Reading Comprehension Midline Results Compared to Baseline Baseline Midline N Avg. SE N Avg. SE Standard 2 Comparison 2283 0.15 -0.02 1684 0.03 -0.01 Treatment 2139 0.21 -0.05 3110 0.06 -0.01 Standard 4 Comparison 2268 1.39 -0.06 1693 0.58 -0.04 Treatment 2184 1.5 -0.08 3101 0.68 -0.04 * Avg = weighted mean number of correct answers, N = sampled observations Reading Comprehension: Percent Meeting Benchmarks The Reading Comprehension benchmark for Standard 2 is 60 percent, which means fluent learners should be able to correctly answer 3 out of 5 questions. The midline results illustrated in Table 32 indicate that none of the learners in the comparison or treatment group were able to meet the benchmark. While the results are startling, they tie in with the oral reading fluency results and the initial reading results, which all point to a lack of foundational reading skills, such as phonemic awareness and decoding skills. In Standard 4, the Reading Comprehension benchmark is 80 percent. Therefore, fluent learners should be able to accurately respond to 4 out of 5 questions asked based on a simple reading passage. The midline results in Table 32 indicate that 4 percent of learners in the comparison group and 5 percent of learners in the treatment group were able to meet the benchmark. These results are aligned with the oral reading fluency results and the other subtasks in which Standard 4 learners scored considerably low compared to the benchmark. 58 Table 32: Percentage Meeting Benchmarks — Oral Reading Comprehension STANDARD SCHOOL TYPE BENCHMARK MIDLINE RESULTS % MEETING BENCHMARK Standard 2 Comparison 60% 1% 0% Treatment 60% 1% 0% Standard 4 Comparison 80% 12% 4% Treatment 80% 14% 5% FACTORS PREDICTING LEARNING SCORES The Task 2 evaluation question focus on determining what factors outside of EGRA are correlated with reading outcomes at midline. To answer the question, the evaluation team used measures of statistical correlation to examine the relationship between the outcomes of interest (oral reading fluency) at midline and potential prediction variables derived from midline data collected using the head teacher, teacher, learner, and household questionnaires as well as the school climate and classroom observation protocols. The team selected the factors to evaluate through the literature reviews it completed in 2013 and 2015 as well as the findings of 2013 IE Baseline Study and 2014 NRA. In its analysis, the team sought to capture effects of the following types of factors on learner reading scores: • Household Resources • Household Support/Involvement • Household Education Levels • Learner Health and Food Security • Learner Attitude toward School • School Resources • Classroom Resources • Teacher Experience, Training, and Use of Best Practices in Teaching • Community Involvement in the School • School Support from Outside Organizations The evaluation team examined multiple variables from each of the categories listed above to select those that remained stable across various regression specifications. Below, the team presents only those factors found to be most consistently and robustly correlated. The team conducted regressions separately for Standard 2 boys and Standard 2 girls and likewise for Standard 4 boys and Standard 4 girls since earlier summary statistics on learner characteristics showed significant variations by standard and gender. This heterogeneity might be explained by the possibility that some factors differentially affect learners of different ages or different levels of reading fluency. For instance, whether someone in the household reads to the learner may matter less for girls than boys since household members are more likely to read to girls than boys (finding from previous reading assessment study). Similarly, EGRA may affect Standard 2 learners more because it specifically targeted Standard 2 learners in its second project year (2014-2015).16 See Tables 33-36, below, for regression results by standard and gender, followed by a discussion of these results. 16 Some parts of the EGRA intervention benefit the school as a whole. So, all learners in treatment schools could have benefitted from the intervention in the 2013-2014 or 2014-2015 academic year—the first two EGRA project years. However, in its first year—the 2013-2014 academic year— in addition to supporting entire schools through the MOUs and Standard 1-3 teachers through one five-day training, EGRA provided more intense support to Standard 1 classrooms via training Standard 1 teachers and providing materials specifically for Standard 1 learners. And, in the 2014-2015 academic year, EGRA provided more intense support to Standard 2 teachers and provided materials specifically for Standard 2 learners. 59 Table 33: Factors Predicting Standard 2 Boys’ Oral Reading Fluency Scores (Measured by correct words read per minute) VARIABLES BASIC MODEL (1) WITH ACTIVITY INTERACTIONS (2) WITH DONOR ACTIVITIES (3) Learner reports taking books home from school (Dummy) 4.433*** 4.492*** 4.576*** (1.630) (1.628) (1.590) Learner reports being read to at home (Dummy) 4.606*** 4.589*** 4.565*** (1.607) (1.610) (1.611) Learner reports getting tired at school (Dummy) -1.945 -1.887 -1.842 (1.743) (1.745) (1.747) Learner-to-teacher ratio -0.0108 -0.0117 -0.0100 (0.0135) (0.0139) (0.0135) Teacher reports having sufficient resources (Dummy) 3.703** 3.593* 3.561* (1.787) (1.836) (1.825) Learner is above the average age for Learner’s standard (Dummy) 1.838 1.794 1.854 (1.547) (1.556) (1.551) Length of the school day in hours 1.965* 1.891* 1.797* (1.082) (1.094) (1.063) Learner reports speaking Chichewa either at home or with friends 4.518** 4.353** 4.458** (2.111) (2.143) (2.151) EGRA (treatment level 4) (Dummy) 5.494*** 4.993** 4.980** (1.736) (1.989) (1.992) EGRA + INVC (treatment level 3) (Dummy) 1.841 1.750 (2.325) (2.326) EGRA + SSDI (treatment level 2) (Dummy) 1.272 1.301 (2.245) (2.246) EGRA + SSDI + INVC (treatment level 1) (Dummy) -2.640 -2.540 (3.392) (3.389) Tiana and Literacy Boost Activities 0.213 (2.904) World Vision Reading Activity -1.632 (2.549) Constant -38.33*** -37.72*** -37.46*** (5.507) (5.513) (5.491) Observations 2,301 2,301 2,301 Coefficients represent marginal effects. *** p < 0.01; ** p < 0.05; * p <0.1. Standard errors in parentheses. Midline IE Data 2015. 60 Table 34: Factors Predicting Standard 2 Girls’ Oral Reading Fluency Scores (Measured by correct words read per minute) VARIABLES BASIC MODEL (1) WITH ACTIVITY INTERACTIONS (2) WITH DONOR ACTIVITIES (3) Learner reports taking books home from school (Dummy) 9.016*** 9.196*** 9.432*** (1.867) (1.865) (1.833) Learner reports being read to at home (Dummy) -3.297* -3.243* -3.155* (1.872) (1.865) (1.841) Learner reports getting help with her homework (Dummy) 4.475** 4.459** 4.305** (1.867) (1.858) (1.842) Learner reports getting tired at school (Dummy) -4.581** -4.573** -4.780** (1.871) (1.867) (1.860) Learner-to-teacher ratio -0.0197 -0.0191 -0.0211 (0.0179) (0.0179) (0.0168) Teacher reports having sufficient resources (Dummy) 5.346*** 4.776** 4.816** (1.896) (1.955) (1.950) Length of the school day in hours 3.021** 3.051** 2.978** (1.317) (1.333) (1.284) Learner reports speaking Chichewa either at home or with friends 1.144 0.870 0.843 (2.038) (2.064) (2.072) EGRA (treatment level 4) (Dummy) 5.299*** 4.197* 4.315** (1.942) (2.149) (2.157) EGRA + INVC (treatment level 3) (Dummy) -1.338 -0.996 (2.618) (2.623) EGRA + SSDI (treatment level 2) (Dummy) 4.430* 4.601* (2.391) 2.390 EGRA + SSDI + INVC (treatment level 1) (Dummy) -0.983 -2.064 (3.779) (3.761) Tiana and Literacy Boost Activities 2.892 (3.289) World Vision Reading Activity 3.317 (2.937) Constant -37.28*** -37.23*** (6.467) (6.560) Observations 2,303 2,303 2,303 Coefficients represent marginal effects. *** p < 0.01; ** p < 0.05; * p <0.1. Standard errors in parentheses. Midline IE Data 2015. 61 Table 35: Factors Predicting Standard 4 Boys’ Oral Reading Fluency Scores (Measured by correct words read per minute) VARIABLES BASIC MODEL (1) WITH ACTIVITY INTERACTIONS (2) WITH DONOR ACTIVITIES (3) Learner reports taking books home from school (Dummy) 0.327 0.281 0.444 (1.072) (1.065) (1.068) School resources index 0.767** 0.780** 0.606* (0.363) (0.366) (0.360) Essential teaching skills index (includes 13 key skills) 0.243 0.186 0.138 (0.278) (0.289) (0.288) Learner is above the average age for his standard -0.928 -0.866 -0.817 (1.062) (1.062) (1.058) School participates in a school-feeding program (Dummy) -4.383*** -4.579*** -4.489*** (1.175) (1.199) (1.190) Length of the school day in hours 0.776 0.748 0.862 (0.926) (0.923) (0.919) Learner reports speaking Chichewa either at home or with friends 6.358*** 6.582*** 6.282*** (1.883) (1.913) (1.918) EGRA (treatment level 4) (Dummy) 2.865** 3.384** 3.513*** (1.122) (1.319) (1.308) EGRA + INVC (treatment level 3) (Dummy) -0.821 -0.595 (1.634) (1.649) EGRA + SSDI (treatment level 2) (Dummy) 0.152 0.195 (1.511) (1.518) EGRA + SSDI + INVC (treatment level 1) (Dummy) -0.478 -1.143 (2.425) (2.465) Tiana and Literacy Boost Activities 1.986 (2.125) World Vision Reading Activity 2.835* (1.708) Constant 6.973 6.965 6.005 (5.343) (5.330) (5.317) Observations 2,257 2,257 2.892 Coefficients represent marginal effects. *** p < 0.01; ** p < 0.05; * p <0.1. Standard errors in parentheses. Midline IE Data 2015. 62 Table 36: Factors Predicting Standard 4 Girls’ Oral Reading Fluency Scores (Measured by correct words read per minute) VARIABLES BASIC MODEL (1) WITH PROGRAM INTERACTIONS (2) WITH DONOR ACTIVITIES (3) Household reports that leaner has access to reading materials at home (Dummy) 3.020*** 1.917** 2.945** (1.157) (0.892) (1.152) Learner reports being read to at home (Dummy) 2.809** 1.770* 2.762** (1.282) (0.976) (1.264) Someone in the household volunteer at least one hour/month 1.455 1.664* 1.465 (1.172) (0.888) (1.162) School resources index 0.259 0.582** 0.218 (0.373) (0.296) (0.372) Learner-to-teacher ratio -0.0415*** -0.0298*** -0.0399*** (0.0112) (0.00855) (0.0114) Number of years teacher has been teaching -0.108 -0.0760 -0.105 (0.0683) (0.0563) (0.0684) School participates in a school-feeding program (Dummy) -3.016** -1.851* -2.760** (1.257) (1.007) (1.270) Learner reports speaking Chichewa either at home or with friends -0.283 1.201 -0.560 (1.504) (1.438) (1.547) EGRA (treatment level 4) (Dummy) 1.357 1.613 1.108 (1.163) (1.137) (1.365) EGRA + INVC (treatment level 3) (Dummy) 5.902*** 5.669*** (1.485) (1.875) EGRA + SSDI (treatment level 2) (Dummy) 4.016** 5.087*** (1.587) (1.646) EGRA + SSDI + INVC (treatment level 1) (Dummy) -10.50*** -12.49*** (2.325) (2.680) Tiana and Literacy Boost Activities 2.799 (1.847) World Vision Reading Activity -1.632 (2.089) Constant 21.03*** 19.18*** 20.94*** (2.372) (1.948) (2.405) Observations 2,162 2,162 2,162 Coefficients represent marginal effects. *** p < 0.01; ** p < 0.05; * p <0.1. Standard errors in parentheses. Midline IE Data 2015. ACCESS TO READING MATERIALS AT HOME As shown in Tables 33-36 and also found to be true at both baseline and in the 2014 NRA, the predicted oral reading fluency scores for learners who had access to reading materials at home (either because their household typically had books and magazines available for him/her to read or because that learner reported taking books home from school with him or her) were generally higher than for learners and households that reported not having access to reading materials at home. Tobit regression results show that Standard 4 boys who reported taking books home from school with them did not have predicted reading fluency scores that were statistically significantly higher than Standard 4 boys who did not. But, Standard 2 boys and Standards 2 and 4 girls who reported taking books home from school with them or whose households reported having access to reading materials are predicted to be able to read more words per minute than learners who did not, holding other factors in the regressions constant. Standard 2 boys who reported taking books home with them had predicted scores more than double that of their peers, reading an average of 4.5 cwpm more than their counterparts. Standards 2 and 4 girls also read an average of about 9.2 and 2.5 cwpm more, respectively, than their counterparts. 63 Both access to books and the desire to take them home likely affect whether learners take books home with them. According to teachers, about 49 percent of all learners took books home from school with them. The reason some learners may not take books home was because their teachers did not allow them to. Teachers said they do not hand out all of the textbooks they have been provided for the following three reasons: 1) there are not enough for each learner to have one, 2) they are worried that learners will not take good care of them, and 3) they are worried that learners will lose them. A full 29 percent of teachers at midline reported not having enough textbooks for each learner (36 percent of comparison￾school teachers and 25 percent of treatment-school teachers). As shown in Figure 22, sampled Standard 4 learners were more likely to report taking books home from school with them than sampled Standard 2 learners, but boys and girls were equally likely to take books home. Overall, 39 percent of sampled Standard 2 learners and 73 percent of sampled Standard 4 learners reported taking books home from school with them. Sampled Standard 4 learners were probably more likely to take books home from school with them because they were better able to read them. Evidence from the study supported this, as whether or not learners reported reading the books they take home with them was almost perfectly correlated with learner scores on their reading tests. Furthermore, 40 percent of all learners reported also having access to other reading materials at home. These numbers were up from those at baseline, suggesting improvement in learner access to reading materials. Figure 22: Percentage of Learners who Take Books Home Midline IE Data 2015. HOUSEHOLD MEMBER READS TO LEARNER Both the baseline and the 2014 NRA reports showed that whether learners reported being read to at home was one of the factors that was most consistently correlated with the predicted value of learner reading outcomes, and its predictive capability on learner reading scores was quite high. However, midline results did not corroborate this fact. While reading to Standard 2 boys and Standard 4 girls appeared to be both highly correlated with predicted oral reading fluency scores and statistically significant, the results were the opposite for Standard 2 girls, as shown in Table 34. The reason for this inverse relationship may be related to household beliefs that girls need more support than boys, and that the lower performing girls need even more support (suggesting that the low scores or weaker reading skills likely lead to learners 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% Standard 2 Standard 4 Boys Girls 64 being read to rather than vice-versa). A qualitative study consisting of ten focus group discussions (FGDs), conducted by the evaluation team in May 2014 suggested that parents may have been more likely to read to girls than boys because they suspected that girls needed more assistance than boys. There was a common belief that girls are more likely to drop out of school than boys because they may become pregnant or get married young. Furthermore, fathers reported believing that girls are weaker than boys and, thus, require more coddling and support in school. While this study may help to explain the negative relationship between Standard 2 girls’ predicted reading scores and being read to at home, it does not explain why there was a shift in the relationship between the baseline and midline results. That might be explained by the possibility that due to recommendations from this study and EGRA programming in general, households are now focusing more reading to those learners who are performing more poorly. The team will explore this shift further in the coming year. According to heads of households at midline, approximately 61 percent of sampled Standard 2 learners and 72 percent of Standard 4 learners sampled were read to at home. These numbers are significantly up from 43 and 63 percent, respectively, at baseline. Also, treatment learners were slightly more (and statistically significantly more) likely to be read to at home than comparison learners, at 68 and 66 percent, respectively. But, girls and boys were equally likely to be read to, which was not the case at baseline or for the NRA, at which time girls were more likely to be read to. HOUSEHOLD MEMBER HELPS LEARNER WITH HOMEWORK Next, at baseline, whether or not a household member helps learners with their homework appeared to be positively correlated with reading scores for Standard 2 learners at a statistically significant level and negatively correlated with higher oral reading scores for Standard 4 learners. At midline, results were similar, as shown in Table 34, with Standard 2 girls’ predicted oral reading fluency scores showing a high and statistically significant correlation with learners receiving help on their homework. Standard 2 girls who reported receiving help on their homework had predicted oral reading fluency scores slightly more than double the average score, an increase of about 4.4 cwpm. The reason this variable appears to only be a strong predictor of Standard 2 girls reading scores is unknown and should be explored further in future studies. More than 54 percent of sampled Standard 2 learners and 67 percent of Standard 4 learners reported receiving help on their homework from someone in their household (up from 44 percent of Standard 2 learners at baseline). Also, while at baseline girls were statistically slightly more likely than boys to receive help on their homework, both sexes received equal support at midline. LEARNER GETS TIRED AT SCHOOL AND OTHER HEALTH AND NUTRITION VARIABLES As was also true at both the baseline and in the 2014 NRA, predictors of learner health and nutrition do not appear to explain much of the variation in oral reading fluency scores at midline for either Standard 2 learners or Standard 4 learners (the one exception in the 2014 NRA was number of learner-reported sick days, which was negatively correlated with reading outcomes).17 However, one factor that does appear to be consistently correlated with Standard 2 oral reading fluency scores among the sample population is whether the learner reports feeling tired at school. As shown in Table 33 and Table 34, those Standard 2 boys who reported feeling tired at school at midline scored an average of 1.9 cwpm lower on their reading fluency test than do those boys who did not, and Standard 2 girls who reported feeling tired scored an average of 4.6 cwpm lower than girls who did not. Further, 29 percent of Standard 2 learners reported 17 Nutrition and health information was largely gathered through the household survey using food security and nutrition modules from the Feed the Future (FtF) Population Survey instrument as well as questions related to learner health. The one exception to this rule is the variable about the number of days a learner did not eat in the week prior to the survey, which is derived from self-reported data from learners gathered during the RA in schools. 65 sometimes feeling tired at school (that number was 22 percent for Standard 4 learners, and sleepiness did not vary by sex). Sleepiness also appears to have decreased since baseline, at which time 46 percent of Standard 2 learners and 35 percent of Standard 4 learners reported feeling tired at school sometimes). While a few of the other health and nutrition variables appear to consistently show a relationship with oral reading fluency scores for learners, none of those variables was consistently statistically significant with the exception of whether or not a learner went to a clinic to receive treatment the last time he/she was sick. Those Standard 4 learners who were seen at a clinic the last time they were sick had predicted reading scores more than 2 cwpm higher than their peers who did not attend a clinic while sick. Those numbers were not included in the above tables because they reduced the sample size of the model down quite drastically due to the fact that many learners were not sick in the past month (the recall period used). In other specifications of the regression model, the team found that the number of days in the past week a learner has not eaten (the more days the lower the reading score) and the number of weeks the child was sick in the past month (the more sick weeks, the lower the scores) appeared to be correlated with learner reading scores. However, when other variables were added into the regressions or taken out, these factors became less significant, suggesting that they were not robust. This lack of a strong relationship between health and nutrition does not mean that health and nutrition are not important predictors of learner performance, but only that the evaluation team cannot be certain of their level of significance. This could be due to poor recall by learners and households when it comes to the health and nutrition variables (it should be noted that the team found very little correlation between answers for questions asked of both learners and households, providing evidence of this poor recall). Or, it could be due to the strong relationship between household education and health, as described in the baseline literature review and the relationship between the education level of households and learners being read to or helped with their homework. Follow-up studies should look more closely at this relationship. LEARNER-TO-TEACHER RATIO Learner-to-teacher ratio was a significant predictor of oral reading scores at baseline and when included in regressions in a binary form in the 2014 NRA (i.e. class sizes of greater than 150 or not). In both cases, evaluators found that the learner-to-teacher ratio was negatively correlated with predicted learner oral reading fluency scores. Midline results corroborated this evidence, as shown in Table 33, Table 34 and Table 35, above. At midline, the team found that class size or learner-to-teacher ratio was a predictor of oral reading fluency scores for all groups except Standard 4 boys; however, it was only significant for Standard 4 girls. For Standard 4 girls, every additional 33 learners in a class predicted a decrease in learner cwpm of about one point. Head teachers reported that the learner-to-teacher ratio was about 114 learners per teacher for Standard 2 and 90 learners per teacher for Standard 4 at midline, which is up from the learner-to-teacher ratio at baseline of 94 in Standard 2 and 65 in Standard 4. Further, the learner￾to-teacher ratio is not the same as class size, as oftentimes two teachers teach the same class. Standard 2 teachers reported that, on average, they had about 212 learners enrolled in their classes at midline, and Standard 4 teachers reported an average of 162 learners enrolled in their classes at midline. SCHOOL RESOURCES The Baseline IE Assessment showed a strong correlation between the level of school resources (generated through an index of resources, as described in the PCA Section of the Methodology Section of this report) and learner reading outcomes. And, midline data corroborated this evidence, as shown in Table 33 and Table 34, which suggest about a 3.6 point increase in predicted oral reading fluency scores for Standard 2 boys and a more than 5 point increase for Standard 2 girls from classes where teachers reported having sufficient resources to teach. Further, data showed that the PCA scores generated for school resources were correlated with Standard 4 boys and girls reading outcomes. While the coefficient values in the case of the school resource index scores do not seem high, this could simply be a function of the fact that the 66 independent variable is being measured here as an index. PCA scores for school resources range from -4 to +4, and a one-point increase in the index resulted in an increase of between a 0.5 and 1 cwpm on the oral reading fluency subtask. Approximately 20 percent of both Standard 2 and Standard 4 teachers reported feeling as though they had sufficient teaching and learning resources at midline (This broke down to 12 percent of comparison￾school teachers and 24 percent of treatment-school teachers). And, the level of observed school resources varied significantly by district, as shown in Figure 23 below, with Ntcheu having the highest level of school resources and Machinga having the lowest levels of school resources. Figure 23: School Resources PCA Scores by District Midline IE Data 2015. SCHOOL FEEDING School feeding is an interesting variable. While at baseline, the evaluation team found no notable correlations between school feeding and learner outcomes, and in the 2014 NRA, the team found a statistically significant positive correlation between whether a school has a school-feeding program and Standard 1 learner oral reading fluency scores, at midline, the team found that whether a school had a school-feeding program predicted a statistically significant drop in learner reading scores of between 2 and 4.5 cwpm for Standard 4 boys and girls, as shown in Table 35 and Table 36. The reason for this negative correlation is unknown, but it could be due to the fact that school feeding programs tend to be offered in poorer, likely lower-performing areas. That said, regression results where school feeding was set as the dependent variable showed that average household assets do not appear to predict whether or not a school has a school-feeding program. Furthermore, the regression showed that the higher the level of school resources, the more likely a school had a school-feeding program, suggesting the hypothesis that the school is receiving school feeding because it is worse off is not a strong one. As such, the evaluation team will monitor the relationship between school feeding and oral reading fluency scores, and if data continue to show a negative relationship between the two, the team will explore the reasons for this unexpected outcome. LENGTH OF SCHOOL DAY Many studies have shown the benefit of longer school days for increasing learner educational levels. This study explored whether the length of the school day was correlated with predicted learner reading scores. As shown in Table 33 and Table 34, the study team found a statistically significant correlation at midline 4.4 4.5 4.6 4.7 4.8 4.9 5 5.1 5.2 5.3 Balaka Blantyre Rural Lilongwe Rural Machinga Mzimba North Ntcheu Ntchisi Salima Thyolo Zomba Rural 67 between the length of the school day and predicted learner reading scores for Standard 2 boys and girls. Specifically, the team found that a one-hour increase in the length of the school day resulted in an increase of oral reading fluency scores of between 1.9 cwpm to 3 cwpm, depending on the sex. RTI staff report that the MoEST has been quicker to lengthen the school day for Standard 1 and 2 than for the higher standards because typically Standards 1 and 2 have the shortest school day in Malawi. So, extending their school day by an hour doesn’t cause major disruptions in the planning/scheduling of people’s breaks, lunches, etc. Extending the school day for higher standards, on the other hand, is significantly more complex. The average length of the school day for Standard 2 at midline was about 4.5 hours, and for Standard 4, it was about 5.5 hours. See Figure 24 for more details on the differences in the average length of the school day by district. Machinga appears to have the shortest average school day and Ntcheu the longest. Figure 24: Average Length of the School Day by District Midline IE Data 2015. LEARNER SPEAKS CHICHEWA AT HOME OR WITH FRIENDS Many studies have shown that learners tend to learn how to read quicker in their native language or the language they speak at home or with friends. Given that learners in Malawi come from a variety of language backgrounds, the evaluation team examined the correlation between whether learners reported speaking Chichewa at home or with friends and their oral reading fluency outcomes. As shown in Table 33 and Table 35, SI found that boys in Standards 2 and 4 scored 4.5 and 6.5 cwpm higher, respectively, on their oral reading fluency test if they reported speaking Chichewa outside of school than if they did not. Results were also positive, just not significant for Standard 2 girls, and they were mixed, but also not significant for Standard 4 girls. A total of 89 percent of learners reported speaking Chichewa outside of school, and this percentage is higher for Standard 4 learners than Standard 2 learners, at 92 and 86 percent, respectively. This may be because as learners become more familiar with Chichewa through school, they begin to speak it outside of school more often. 4.4 4.5 4.6 4.7 4.8 4.9 5 5.1 5.2 5.3 Balaka Blantyre Rural Lilongwe Rural Machinga Mzimba North Ntcheu Ntchisi Salima Thyolo Zomba Rural 68 OTHER ACTIVITY EFFECTS (SSDI AND INVC) At baseline, there were no significant differences in oral reading fluency scores by treatment assignment across the treatment levels/types. As such, the evaluation team examined the various types of treatments provided in this study: 1) Treatment level/Type 1 – EGRA + SSDI + INVC, 2) Treatment level/Type 2 – EGRA+SSDI, 3) Treatment level/Type 3 – EGRA +INVC, and 4) Treatment level/Type 4 – EGRA-only at midline to determine if there were any major differences in outcomes for learners in each of these treatment levels. As shown in model two of Tables 33-36, above, results were a bit mixed. • Treatment Level 4—or the area in which learners benefitted from EGRA but not the SSDI or INVC activities—showed the strongest link with oral reading fluency outcomes, with high levels of correlation and statistical significance across standards and sexes (for Standard 4 girls, Level 4, while the correlation was still positive, it was not statistically significant). • Treatment Level 3—or EGRA + INVC—was correlated with higher predicted learner reading scores for Standard 2 boys and Standard 4 girls, and results were statistically significant for Standard 4 girls. Results were negative but insignificant for Standard 2 girls and Standard 4 boys. • Treatment Level 2—or EGRA + SSDI—was consistently correlated with higher predicted oral reading fluency scores, and those differences were statistically significant for girls in both Standards 2 and 4. • Finally, Treatment Level 1—EGRA + SSDI + INVC—or what is called as USAID/Malawi’s Country Development Cooperation Strategy (CDCS) focus area showed negative correlations across standards and sexes, and the correlation was statistically significant for Standard 4 girls. The results may be surprising to donors and the implementing partner and may not match with expectations. As such, the evaluation team explored a few possible causes for this difference, including determining if CDCS districts might just be lower-performing districts overall. However, the team found that this was not the case. But, the integration of activities across sectors and projects within a sector as envisioned under CDCS are yet to be fully operational and may likely influence results as CDCS goes into full effect with its integration of activities. SI will explore this further in future studies. OTHER DONOR ACTIVITIES As described above, there are several other donor education activities that overlap with EGRA. Thus, to ensure the effects of EGRA are isolated from these other donor interventions, SI examined all of the other donor interventions and found that those most commonly associated with positive outcomes were the two activities implemented by Save the Children—Tiana, which is funded by USAID, and Literacy Boost, which is funded by Save the Children—and the activity implemented by World Vision. When these interventions were added to the regression model, SI found that World Vision had a statistically significant positive correlation with oral reading fluency outcomes for Standard 4 boys. While Tiana and Literacy Boost did not prove to be statistically significant in any of the above models, when they are included in models that only examine Level 4 outcomes, they do prove positive and statistically significant. This is because their activities were focused in Zomba. EFFECTS OF INTERMEDIATE EGRA OUTCOMES ON LEARNER SCORES Task 4 evaluation questions include questions about intermediate EGRA results, including what effect each of the following have on learner reading outcomes: • Level of effort of reading instruction’s impact on children’s reading abilities • Effect of extra-curricular reading activities 69 • Effect of time on task in improving reading outcomes To assess these intermediate outcomes, the SI team created a regression model that included each of them but none of the other direct project variables—such as EGRA, SSDI, or INVC. That model is presented in Table 37, below. Table 37: Factors Predicting Intermediate EGRA Result (Measured by correct words read per minute) VARIABLES MODEL Standard 4 16.84*** (0.323) Sex (Girls) 2.561*** (0.648) Household reports that leaner has access to reading materials at home 2.134*** (0.664) PCA Score for level of school resources 0.716*** (0.218) Number of years teacher has been teaching -0.125*** (0.0408) Number of times teacher reports meeting with parents annually (outside of PTA meetings) 0.780*** (0.214) School participates in a school-feeding program -2.854*** (0.727) EGRA provided the school with books -0.281 (0.864) Number of days of EGRA training provided to the teacher in the last three year 0.0683 (0.0506) Number of EGRA coaching visits the teacher reported receiving in last term -2.019* (1.059) Head teachers who reported signing an MOU with EGRA to extend the length of the school day 2.927*** (0.990) Head teachers who reported signing an MOU with EGRA to add an hour of reading instruction to the day 2.112** (1.003) Head teachers who reported signing an MOU with EGRA to encourage parents to read to their learners -2.299*** (0.871) Head teachers who reported signing an MOU with EGRA to reduce class size or split classes 4.395*** (1.273) Head teachers who reported their school received an incentive (a GUC) from EGRA 1.214 (3.457) Number of reading fairs the head teacher reports the school hosted in the past 2 years 0.919*** (0.168) Constant -52.98*** (1.378) Observations 8,436 Coefficients represent marginal effects. *** p < 0.01; ** p < 0.05; * p <0.1. Standard errors in parentheses. Midline IE Data 2015. As shown in Table 37, the number of coaching visits per term appears to be negatively correlated with learner reading outcomes. This is strange and may simply be because after a certain point, the coaching efforts have a diminishing effect. SI will explore this further if this result continues at endline. Outside of this, the EGRA MOUs appear to have been very successful, with the exception of the MOU that encouraged parents to read to learners. The latter may have been less successful because the MOU is fairly new (having just been signed in the 2013-2014 academic year for Cohort A schools and the 2014- 2015 academic year for Cohort B schools). Further, it likely targeted parents who were not already reading 70 to their learners; thus, based on earlier findings, these learners were probably scoring lower on reading tests prior to the 2013-2014 academic year. By 2017, this MOU may likely prove to have been beneficial to learner reading scores. On the other hand, the MOU that worked to reduce class size appears to have been very successful, improving reading scores by an average of 4.4 cwpm. The MOUs to extend the school day and the length of the reading lesson also appear to have had a large effect, increasing learner reading scores by an average of 2.9 cwpm and 2.1 cwpm, respectively. Finally, the number of reading fairs the school hosted in the past two years also appears to be a good predictor of learning reading scores, with each additional fair increasing scores by almost 1 cwpm. On average, EGRA treatment school head teachers reported hosting a much larger number of fairs than did comparison-school head teachers. Treatment-school head teachers reported hosting about 2.3 reading fairs in the past two years whereas comparison-school head teachers reported only hosting about half of a fair (the fraction is due to the fact that many comparison schools did not host any reading fairs in the last two years). These results suggest implementation fidelity and success of at least the MOUs and reading fairs. EGRA PROGRAM IMPACTS ON LEARNER SCORES: DIFFERENCE IN DIFFERENCES RESULTS In this section, SI presents estimates of program impacts obtained by comparing the oral reading fluency scores measured in correct words per minute across time and treatment status in order to address the evaluation question on EGRA’s impact on children’s reading abilities (disaggregated by sex). SI examined changes from base to midline data to understand EGRA program effect on learning scores for Standards 2 and 4 in comparison and treatment schools. The analysis helped to measure EGRA program impacts over a period of two years of EGRA implementation in order to test the hypothesis that learner reading outcomes will improve with EGRA intervention. The evaluation team first examined treatment effect across all levels. Then, SI also examined by various types treatment levels used in this study: i) Treatment level Type 1 – EGRA + SSDI + INVC, ii) Treatment level Type 2 – EGRA+SSDI, iii) Treatment level Type 3 – EGRA +INVC, and iv) Treatment level Type 4 – EGRA or what is called as USAID/Malawi’s Country Development Cooperation Strategy (CDCS) zone to determine if there were any major differences in outcomes for learners in each of these treatment levels. Using the data from 264 panel schools and treatment status, SI team examined the EGRA treatment effects using the difference in differences (DiD) model18. Covariates such as enrollment, dropout rates and PTA monthly meeting were added to control for differences noted at baseline. The analysis helped in examining whether there were differences between base and midline scores of learners in EGRA treatment schools (treatment effects), and whether those differences were distinct from the differences for comparison school learners (effect size); and to attribute the results to EGRA and not to chance or measurement error. Changes in average test score for each school were regressed using ordinary least squares model against treatment assignment at midline. Therefore, the panel of schools were used and matched across baseline and midline using unique IDs, and average scores for each school and with school as a unit of analysis were used. We also disaggregated effects by standard and gender. Accordingly, six regressions for this model were estimated to test by Standard 2 overall, Standard 4 overall, Standard 2 girls, Standard 4 18 The standard DiD method (compared to propensity score matching) assumes that unobserved heterogeneity in participation is present, but it is time invariant. The assumption of time invariance at times poses a limitation. If projects are targeted in selected areas or schools using specific criteria, there could be dynamic response in both observed and unobserved ways in comparison and treated areas. In practice, while designing the evaluation, the ex-ante, time-varying unobserved heterogeneity could be accounted for by ensuring that treatment and comparison schools are drawn at baseline from similar districts or within districts. SI, as noted before, randomly drew comparison and treatment schools from the same district. For remaining selection bias, SI controlled for initial conditions to resolve nonrandom fixed events that might bias the program effect. Therefore, SI has applied DiD with covariates to control for initial conditions (found at baseline) to infer program effects. 71 girls, Standard 2 boys, and Standard 4 boys. Here, treatment referred to treatment assignment (at both baseline and midline), expressed as a dummy variable taking a value of one for treatment status and zero for comparison status, and coefficient estimates of the dummy variable represented the treatment effect. Using the standardized coefficients, effect sizes were calculated and were reported in standard deviations. Table 38: EGRA Program Effects on Oral Reading Fluency for Standard 2 Learners: Results of Difference in Difference Estimates, by Gender GIRLS BOYS OVERALL Constant -0.238 (0.402) -1.843*** (0.485) -1.136*** (0.386) Treatment (Dummy) 1.184 (0.765) 0.947 (1.045) 0.992 (0.842) Observatio ns 237 254 256 R-squared 0.0111 0.0067 0.0098 Mean (cwpm) 0.426 (5.586) -1.3 (5.731) -0.569 (4.980) Effect Size (SD) 0.076 -0.227 -0.114 Standard errors in parentheses; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable was change in school averages. All treatment schools in Levels 1 to 4 compared to all comparison schools in Levels 1, 3 and 4. Covariates were added on enrollment, PTA meets monthly and on dropout rates to control for differences at baseline. Table 39: EGRA Program Effects on Oral Reading Fluency for Standard 4 Learners: Results of Difference in Difference Estimates, by Gender GIRLS BOYS OVERALL Constant -4.928*** (1.305) -6.452*** (1.221) -5.464*** (0.943) Treatment (Dummy) 2.400 (1.852) -0.087 (1.663) 0.840 (1.334) Observations 245 260 262 R-squared 0.0033 0.0001 0.001 Mean (Std. Deviation) -3.549 (11.415) -6.503 (10.742) -4.973 (8.689) Effect size (SD) -0.311 -0.605 -0.572 Standard errors in parentheses; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable was change in school averages. All treatment schools in Levels 1 to 4 compared to all control schools in Levels 1, 3 and 4. Covariates were added on enrollment, PTA meets monthly, and on dropout rates to control for differences at baseline. Results presented in Table 38 for Standard 2 show that EGRA has had a positive effect in treatment schools, although it was not statistically significant. EGRA has resulted in increasing oral reading fluency by 1.2 cwpm for girls, by 0.95cwpm for boys, and by about 1 cwpm overall when boys and girls were combined at midline from baseline in treatment schools. When the difference in effects from base to 72 midline were compared between comparison and treatment schools, however, the effect size of EGRA was a 0.11 standard deviation reduction, although it was very small to be considered as notable.19,20 In Standard 4, as shown in Table 39 , EGRA has resulted in an additional 2.4 cwpm for girls and by 0.84 overall when boys and girls were combined in treatment schools at midline relative to comparison schools. While a negative effect was noted for boys, it was negligible at 0.08. When the differences in effects from base to midline were compared between comparison and treatment schools, however, the effect size of EGRA was found to be 0.57 standard deviation reduction, with a larger reduction noted for boys than girls (0.605 vs. 0.311). 21 19 Recall the results presented earlier for oral reading fluency in this report under the section on reading assessment scores found at base and midline. They showed that midline scores in treatment scores were higher than in comparison schools. The results were also confirmed by findings from regressions on predictors of midline scores, presented under results from midline data analysis. However, the earlier findings also indicated that when baseline and midline scores were compared, they declined at midline in both groups, although with a lesser reduction in treatment schools relative to comparison schools. The results in the earlier sections were, however, not controlled for baseline differences. The results presented under DiD were derived from standardized coefficients and also were controlled for initial conditions, and therefore are indicative of effect sizes. 20 A systematic review of education interventions showed that effect sizes were generally at: 0.12 for teacher training, 0.09 for learner and teacher performance incentives, and 0.08 for instructional materials. In Mali, for oral reading fluency in correct words per minute, effect size at midline for EGRA type intervention was 0.92 and 0.12 for grades 1 and 2, respectively. In Liberia, the effect size for oral reading fluency in an EGRA intervention was 0.8 at endline (see literature review section above). 21 Recall the results presented earlier for oral reading fluency in this report under the section on reading assessment scores found at base and midline. They showed that midline scores in treatment scores were higher than in comparison schools. The results were also confirmed by findings from regressions on predictors of midline scores, presented under results from midline data analysis. However, the earlier findings also indicated that when baseline and midline scores were compared, they declined at midline in both groups, although with a lesser reduction in treatment schools relative to comparison schools. The results in the earlier sections were, however, not controlled for baseline differences. The results presented under DiD were derived from standardized coefficients and also were controlled for initial conditions, and therefore are indicative of effect sizes. 73 Table 40. EGRA Program Effects on Oral Reading Fluency for Standard 2 Learners: Results of Difference in Differences Estimates, by Gender and Treatment Groups (Ordinary Least Squares Estimates) Treatment schools at each of Level 1 to 4 compared with pure control schools in Levels 3 and 4. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. Covariates were added on enrollment, PTA meets monthly and on dropout rates to control for differences at baseline. We also examined effects of EGRA on oral reading fluency when EGRA operated in areas where other USAID programs such as INVC and SSDI were also implemented, and when only EGRA was operational. Results are presented in Table 40 and Table 41 for Standards 2 and 4, respectively. For Standard 2, as shown in Table 40, EGRA intervention has had a positive and significant effect overall in Levels 1 and 4 as indicated by learners improving their reading skills by an additional 2.3 and 2.5 cwpm, respectively, in treatment schools at midline relative to comparison schools. There were variations, however, by gender of the learners. For the girls, results were significant in Levels 1 and 2 with a positive effect in Level 1 at 3.1 cwpm and a negative effect in Level 2 at 2.9 cwpm. For the boys, significant effect was found only in Level 4 with a positive 2.9 cwpm. Overall, the results produced an effect size of 0.04 and 0.06 standard deviations, respectively, for Level 1 and Level 4. The results indicate that EGRA was effective, albeit very small, relative to comparison schools in EGRA-only locations and in areas where EGRA operates in addition to SSDI and INVC. For girls, the effect size was a positive 0.56 standard deviation in Level 1 and a negative effect size of 0.56 standard deviations in Level 2. For the boys, an effect size of 0.05 standard deviation was found for Level 4. GIRLS BOYS OVERALL Constant -.508 (1.338) -1.317 (2.025) -0.892 (1.653) Treatment level 1 3.084*** (1.112) 2.307 (1.602) 2.323* (1.287) Mean 2.375 (4.265) -0.938 (7.084) 0.234 (5.724) Effect Size (SD) 0.557 -0.132 0.041 Treatment level 2 -2.992* (1.567) 0.848 (1.589) -0.465 (1.171) Mean -3.618 (6.440) -2.384 (6.589) -2.479 (4.841) Effect Size -0.562 -0.362 -0.512 Treatment level 3 -.0349 (2.294) 1.937 (1.507) 1.190 (1.697) Mean -1.050 (11.358) -0.662 (6.535) -0.630 (8.138) Effect Size -0.092 -0.101 -0.077 Treatment level 4 1.534 (1.644) 2.936** (1.386) 2.467** (1.206) Mean 0.456 (5.922) -0.250 (4.738) 0.277 (4.099) Effect Size 0.077 -0.053 0.068 Control level 1 1.01535 (0.946) 1.957* (1.104) 1.267 (0.903) Observations 214 230 232 R-squared 0.088 0.043 0.046 74 In Standard 4, as shown in Table 41, EGRA intervention has had a positive effect overall in treatment schools as indicated by learners improving their reading skills by an additional 0.5, 2.3 and 0.7 cwpm, respectively, in Levels 1, 3 and 4 in treatment schools at midline relative to comparison schools, although none of the results were significant. There were differences by gender, although none of the results were significant, in that positive effects were noticed in Levels 2 and 4 for girls while they were negative for the boys. The results further showed that EGRA schools were able to perform slightly better than control schools at midline, but effect sizes were all negative indicating a reduction from baseline. Also, treatment schools in EGRA only locations (Level 4) and in areas where EGRA operates in addition to SSDI and INVC (Level 1) appeared to do better relative to comparison schools, especially for girls. The differences in effects noticed by gender and by levels of treatment should be further examined at endline when EGRA intervention would have been completed and as CDCS integration activities among various USAID implementing partners such EGRA, SSDI and INVC take effect. Table 41. EGRA Program Effects on Oral Reading Fluency for Standard 4 Learners: Results of Difference in Differences Estimates, by Gender and Treatment Groups (Ordinary Least Squares Estimates) Treatment schools at each of Level 1 to 4 compared with pure control schools in Levels 3 and 4. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. Covariates were added on enrollment, PTA meets monthly, and on dropout rates to control for differences at baseline. GIRLS BOYS OVERALL Constant -3.837 (2.491) -5.487* (3.027) -4.497 (2.166) Treatment level 1 0.594 (2.823) 1.014 (2.605) 0.526 (1.925) Mean -3.073 (11.887) -5.914 (10.776) -4.719 (8.657) Effect Size -0.259 -0.549 -0.545 Treatment level 2 1.301 (3.306) -2.268 (2.441) -0.870 (2.087) Mean -2.395 (13.677) -9.102 (8.142) -5.877 (8.867) Effect Size -0.175 -1.118 -0.663 Treatment level 3 2.713 (2.379) 1.749 (2.323) 2.263 (1.888) Mean 0.005 (10.205) -4.201 (8.471) -1.950 (7.767) Effect Size 0.000 -0.496 -0.251 Treatment level 4 2.618 (2.961) -1.361 (3.033) 0.735 (2.467) Mean -2.768 (11.085) -8.197 (11.631) -5.297 (9.128) Effect Size -0.250 -0.705 -0.580 Control level 1 -3.494 (2.738) 0.054 (2.642) -1.434 (1.967) Observations 222 236 238 R-squared 0.053 0.032 0.050 75 VII. COST EFFECTIVENESS OF EGRA ON LEARNER SCORES Several activities are undertaken under EGRA in Malawi in order to improve early grade reading skills. In addition to examining whether and how much EGRA has improved reading skills, it is also essential to understand the cost effectiveness of the activity in order to scale up. Therefore, one of the evaluation questions to be addressed by SI under Task 4 focuses on cost effectiveness. Specifically, USAID wants to know which of the EGRA components have the largest effect and the relative cost effectiveness of the various components. At midline, the evaluation team has calculated cost effectiveness as cost in US dollars per learner to achieve unit increase in reading skills measured by oral reading fluency in correct words read per minute during the 2014-15 school year. This year was selected to capture a full year during which EGRA has been implemented such that the costs incurred and effects measured at midline can be related. It is to be noted that EGRA costs in 2014-15 were more tilted toward the focus Standards 1 and 2. Therefore, caution is needed in applying the costs across both standards. However, RTI could have incurred some costs through its parental and community involvement in the schools, at least two reading fairs, and some Standard 4 teachers might have also been more motivated due to the distribution of the grants under contract in the 2014-15 academic year. Therefore, we proportionately allocated the total costs in 80:20 ratio towards Standard 2 and 4, respectively. The team used the total direct costs (excluding labor), which include all costs for the four components. While the evaluation question asks for cost effectiveness by component, at early stages of the project, in order to raise learner reading abilities and EGRA scores, the project needed to provide teacher training, improvement of facilities and provision of instructional materials, promote community and parental engagement, and engage in improving the policy environment simultaneously. Each component, while it may incur separate costs, may also reinforce the others for a total effect. Therefore, the team did not disaggregate the costs by components at midline. The direct cost data were obtained from RTI, the implementing partner of EGRA, for the year 2014-15. While reading skills were assessed by SI under various subtasks in this evaluation, the evaluation team used oral reading fluency, since it represents the major indicator of progress for USAID/Malawi and also the main focus of EGRA. SI calculated cost effectiveness as follows: Step 1. Using midline and baseline data gathered through reading assessments on oral reading fluency from treatment and comparison schools, SI calculated program impacts by measuring the difference in changes in oral reading fluency in correct words read per minute from baseline to midline in treatment and comparison schools. SI also calculated effect sizes. These results were discussed in the sections above but are summarized in Table 42 below. Step 2. Using the data provided by RTI on total direct costs and number of learners enrolled in intervention schools, evaluators calculated unit cost per learner (see Table 42 and Table 43 below). Step 3. Using the unit cost per learner and impacts, the evaluation team calculated cost effectiveness in US dollars required per learner for achieving one unit improvement in words read correctly per minute. UNIT COSTS OF EGRA INTERVENTION FOR 2014-15 Data in Table 42 and Table 43 show the data obtained from RTI on direct costs and counts of intervention items, and unit cost calculated by SI per school, teacher, and learner. 76 RTI reported that a total of US $4,190,168 was spent on direct costs, excluding labor, to carry out activities during the school year 2014-15 in 11 districts that included 134 zones. A total of 1,603 schools were supported through the intervention, where RTI trained 10,806 teachers in Standards 1–3 and head teachers. The number of learners in the 1,603 intervention schools was reported by RTI to be 554,796. Table 42: Direct Costs Incurred by RTI for EGRA Implementation during School Year July 2014–June 2015 COMPONENTS FUNDS SPENT (US DOLLARS) Component 1 Quality reading instruction for early grade students 2,220,421 Component 2 Teaching and learning materials on reading 752,356 Component 3 Parental and community engagement to support student reading 919,219 Component 4 Policy environment to support early grade reading 298,172 Total Direct Costs (excluding labor and indirect costs) 4,190,168 Table 43: Coverage of RTI Intervention in School Year 2014–15 INTERVENTION ITEM COUNT UNIT COST (US DOLLARS) District 11 380,924.36 Zone 134 31,269.91 School 1603 2,613.95 Total Teachers 10806 387.76 Std 1 3181 - Std 2 2910 - Std 3 2955 - HTs 1760 - Total learners in intervention schools 554,796 7.55 Source: RTI. Note: Unit costs calculated by dividing total direct costs by counts of intervention item. COST EFFECTIVENESS OF EGRA Based on the above unit costs and effects calculated by SI as shown in earlier sections, cost effectiveness is calculated and presented in Table 44 below. Table 44: Cost Effectiveness of EGRA per Learner for Unit Change in Oral Reading Fluency ITEMS EFFECTS COSTS/LEARNER ($) COST EFFECTIVENESS ($ PER CWPM) Std 2 Std 4 Std 2 Std 4 Std 2 Std 4 Program effect in cwpm 0.992 0.840 6.0421 1.5105 6.0908 1.7982 As shown in Table 44, it would cost around $6.10 per learner in Standard 2 to improve by one unit in oral reading fluency in correct words per minute. For Standard 4, with the share of costs approximately allocated by SI to have been incurred by RTI in 2014 academic year, it would cost about $1.80 per learner to improve by one unit in oral reading fluency in correct words per minute. The above cost effectiveness estimates, however, should be interpreted with caution, especially for scaling up, because it is early to measure impacts since intervention focus standards may change in coming years and costs are aggregated under all components and approximately allocated across the two standards. Nonetheless, they provide some insights into cost effectiveness at early stages and also of phased implementation of EGRA by standard. The interventions to date have focused on Standards 1 and 2, and 77 are planned to be expanded to other standards in coming years. As the project matures and is expanded to more standards, economies of scale and scope may occur leading to reduction in some costs, and also effects may improve, thus altering the cost effectiveness estimates of EGRA. 78 VIII. MIDLINE FINDINGS: FACTORS PREDICTING DROPOUTS AND REPETITION Task 2 evaluation questions also ask about repetition and dropouts. As such, the evaluation team created a model to test factors that help predict dropouts or repetition. DROPOUTS The dropout model, which is presented in Table 45, used total head-teacher-reported dropout rate (head teacher-reported dropouts divided by head teacher-reported enrollment) across Standards 1 through 4 as the dependent variable and school averages across household, classroom, and school-level variables as the predictor (or independent) variables. Table 45: Factors that Predict Total Annual Standard 1-4 Dropouts (at the school level) SCHOOL-LEVEL VARIABLES DROPOUT MODEL School average oral reading fluency scores 0.000823 (0.000956) Percent of learner households reporting that learners have access to reading materials at home -0.0138 (0.0256) Average household wealth, as indicated by level of household assets -0.0140*** (0.00510) Percent of sampled school learner households reporting that at least one household member has graduated from Standard 4 -0.0868 (0.0797) Percent of sampled school learners that attended preschool -0.0271 (0.0200) Average teacher use of essential teaching practices for sampled teachers at the school -0.00220 (0.00251) Average sampled classroom-reported learner-to-teacher ratio 0.000162* (8.37e-05) Number of reading fairs the school hosted in the past two years -0.00358 (0.00258) EGRA (treatment level 4) (Dummy) -0.0176 (0.0111) EGRA + INVC (treatment level 3) (Dummy) 0.0118 (0.0150) EGRA + SSDI (treatment level 2) (Dummy) 0.0241 (0.0152) EGRA + SSDI + INVC (treatment level 1) (Dummy) -0.0217 (0.0228) Constant 0.0862*** (0.0187) Observations 315 Coefficients represent probabilities. *** p < 0.01; ** p < 0.05; * p < 0.1. Standard errors in parentheses. Midline IE Data 2015. As shown in Table 45, many of the same variables that predicted learner oral reading fluency scores also predicted dropouts. However, the correlation appears to run in the opposite direction. For instance, average household wealth, higher household levels of education, learner attendance at preschool, teacher 79 use of best practices, and number of school reading fairs were all negatively correlated with learner dropouts, though household wealth was the only variable that had a statistically significant correlation. Also, the average learner-to-teacher ratio for sampled classrooms predicted an increase in dropouts, meaning the more learners in a class, the more drop out. Treatment levels were not statistically significantly correlated with dropouts either. Figure 25: Total Average Annual Standard 1-4 Dropouts by District at Midline Midline IE Data 2015. REPETITION RATES The repetition model, which is presented in Table 46, below, used logistic regression to examine binary head-of-household-reported information on whether learners repeated their standard in the 2014-2015 academic year against individual, household, classroom, and school-level predictor (or independent) variables. 0 5 10 15 20 25 30 35 40 45 Balaka Blantyre Rural Lilongwe Rural East Lilongwe Rural West Machinga Mzimba North Ntcheu Ntchisi Salima Thyolo Zomba Rural 80 Table 46: Factors Predicting Whether Learners Repeat a Standard (Logistic Regression) VARIABLES REPETITION MODEL Standard 4 0.0102 (0.0311) Sex (Girls) -0.0178 (0.0615) Whether the learner reports having moved schools in the last year -0.146 (0.168) Household reports that learner has access to reading materials at home -0.233*** (0.0641) Learner reports being read to at home -0.107 (0.0705) Learner reports getting help with her homework 0.0596 (0.0691) Household wealth, as indicated by level of household assets -0.0427*** (0.0154) Learner comes from a household where at least one member has graduated from Standard 8 -0.350*** (0.0657) Learner attended preschool -0.297*** (0.0622) Learner reports speaking Chichewa either at home or with friends 0.235** (0.0949) EGRA (treatment level 4) (Dummy) -0.0585 (0.0736) EGRA + INVC (treatment level 3) (Dummy) -0.216** (0.100) EGRA + SSDI (treatment level 2) (Dummy) -0.101 (0.0926) EGRA + SSDI + INVC (treatment level 1) (Dummy) 0.483*** (0.144) Constant -0.308** (0.146) Observations 7,968 Coefficients represent marginal effects. *** p < 0.01; ** p < 0.05; * p<0.1. Standard errors in parentheses. Midline IE Data 2015. Results presented in Table 46 show that the factors that appear to be the best predictors of learner repetition in a standard are: • Learner access to reading materials at home (Learners with access were 23 percent less likely to have been repeating their standard at midline). • Household wealth (Learners with higher levels of household wealth were less likely to be repeating a standard at midline). • Highest level of household education (Learners from households where at least one member had graduated from Standard 8 were 35 percent less likely to be repeating a standard at midline). • Whether the learner attended preschool (Learners who attended preschool were 30 percent less likely to be repeating a standard at midline). • Whether the learner was in Treatment Level 2 (Learners from Treatment Levels 2 were 22 percent less likely to be repeating a standard at midline). • Whether the learner was in Treatment Level 1 (Learners from Treatment Level 1 were 48 percent more likely to be repeating a standard this year). Most of these variables were also found to be predictors of learner reading and/or dropout outcomes, and were, thus, discussed in detail above. However, learner preschool attendance is a new variable not 81 explored above. Approximately 50 percent of Standard 2 boy and girl learners attended preschool, according to the head of their household, while only 49 percent of Standard 4 boys and girls attended preschool. 82 IX. SUMMARY AND CONCLUSIONS AT MIDLINE, WHAT PROPORTION OF STANDARDS 2 AND 4 LEARNERS HAVE ATTAINED MOEST ESTABLISHED BENCHMARKS FOR READING SKILLS? HAVE THEY CHANGED FROM BASELINE? At midline, in Standard 2 about one percent of learners were able to read grade-level text by the end of Standard 2, and zero percent of learners were able to read with comprehension according to the MoEST benchmark established in 2014. The results were similar in both comparison and treatment groups in proportion of learners attaining benchmarks. In Standard 4 at midline, nearly 8 percent of learners met the Oral Reading Fluency benchmark, and 5 percent of learners were able to comprehend 80 percent of the Reading Comprehension questions in Treatment schools. But, it was 7 and 4 percent, respectively, for Oral Reading Fluency and Reading Comprehension in comparison schools for the proportion of learners attaining benchmarks. Baseline results were slightly higher for Oral Reading Fluency in that it was 1.2 percent for Standard 2 and 10 percent for Standard 4 by 2014 benchmark. But, baseline results were similar in both standards for proportion of learners attaining 2014 benchmark for Reading Comprehension (0 percent in Standard 2 and 5 percent in Standard 4). At midline, both the comparison and treatment groups performed similarly across all subtasks for the pre￾reading subtasks, initial reading subtasks, and reading fluency and comprehension subtasks. Among all subtasks, the sampled learners scored highest in listening comprehension, but more than half missed the benchmark. The learners performed well below the benchmarks on the initial sound identification subtask and across all initial reading subtasks, suggesting a lack of pre-reading and initial reading skills such as phonemic awareness and decoding. Since learners were able to read a similar number of words on the Familiar Word Reading subtask and the Oral Reading Fluency subtask, the results implied that they can read sight words from memory; however, they are not yet able to decode words in isolation or infer meaning from connected text in a simple reading passage. WHAT ARE THE HOUSEHOLD, SCHOOL, AND OTHER PREDICTORS OF LEARNING SCORES? DO THEY DIFFER BY GENDER AND TREATMENT STATUS, AND FROM BASELINE? The evaluation team found numerous predictors of oral reading fluency scores at midline. These factors do differ by sex and standard, as described in great detail in the body of the report. For Standard 2 at midline, factors such as learners reporting that they take books home from school, are being read to at home, and getting tired at school were found to be significantly correlated with midline oral reading fluency scores. Also, learner-to-teacher ratio, teachers reporting adequate teaching resources, and overage learners were correlated with reading scores. These factors were similar to those found as predictors of learning scores at baseline. For Standard 4, learners reporting that they take books home from school appear to be significantly correlated with midline reading scores. Additionally, overage learners, school resources, essential teaching skills were correlated with reading scores. These factors were similar to those found as predictors of learning scores at baseline. There were two factors that were found to be additionally correlated only at midline for both standards: length of school day and speaking Chichewa 83 either at home or with friends. While at baseline no notable correlations between school feeding program at school and learner outcomes were found, and in the 2014 NRA it was found to be significantly and positively correlated with standard 1 learner oral reading fluency scores, at midline, the team found that whether a school had a school-feeding program predicted statistically significant results for Standard 4 only. SI also examined correlations between midline reading fluency scores and other donor interventions and found that those most commonly associated with positive outcomes in reading scores at midline were the two activities implemented by Save the Children—Tiana, which is funded by USAID, and Literacy Boost, which is funded by Save the Children—and the activity implemented by World Vision. World Vision had a statistically significant positive correlation with oral reading fluency outcomes for Standard 4 boys. While Tiana and Literacy Boost did not prove to be statistically significant, they were positive and significant in Level 4 likely because their activities were focused in Zomba district located within the Level 4 study area. HAVE LEARNING SCORES FOR STANDARDS 2 AND 4 CHANGED SINCE THE EGRA INTERVENTION, AND WHAT IS THE EGRA PROGRAM EFFECT? When midline average scores were compared with baseline, performance across nearly all subtasks and standards dropped considerably in both treatment and comparison groups of schools. The exception to this trend was in the syllable segmentation and syllable reading subtasks, in which the treatment group slightly improved from the baseline to midline, while the comparison group showed a decrease in performance. Regression results using DiD indicated that, overall, learners in EGRA-treatment schools in both Standards 2 and 4 have shown improvements relative to comparison schools at midline, albeit they were less than one cwpm and were not significant. The program effects were better in Standard 2 than in Standard 4 (0.99 cwpm in Standard 2; 0.84 cwpm in Standard 4) likely due to EGRA’s explicit focus in its first year (2013-2014 academic year) on Standard 1 learners and teachers, some support for Standard 1-3 teachers, and again more intense support to Standard 2 teachers and provision of materials specifically for Standard 2 learners in the 2014-2015 academic year. Most Standard 4 learners were not exposed to EGRA with the exception that their Standard 3 teachers received one, five-day training in 2013 and learners could have likely received the benefits of two years of the extended school day and reading lessons, reduced class sizes, more parental and community involvement in the schools, and at least two reading fairs. Also some Standard 4 teachers might have also been more motivated due to the distribution of the grants under contract in the 2014-2015 academic year. But, notable impacts have not occurred yet to demonstrate positive and larger effect sizes due to decline noted in scores at midline from baseline. Indeed, the reduction from baseline scores in treatment schools has been lower than in comparison schools. But, for EGRA to show notable impacts, it is important for EGRA and USAID/Malawi to examine the reasons behind the declining trend and find ways to stop the decline in treatment schools in both standards and also further increase learner skills from midline. Our results appear to indicate that the declining trend in scores could plausibly be linked to factors such as increase from baseline to midline in learner-to-teacher ratio, and decreases from base to midline in learners receiving help with their homework from a household member and in households encouraging the child to read, although there were improvements in teacher practices and length of school day at lower standards. Also, there were differences noticed by gender in that girls appear to perform better than boys in both standards. The differences in scores by gender need further inquiry to confirm the results and understand how and why EGRA activities contribute to such differences. 84 DO OTHER USAID ACTIVITIES IN AGRICULTURE AND HEALTH BESIDES EGRA TREATMENT AFFECT LEARNER SCORES? Treatment schools in areas that received just the EGRA treatment (Level 4) had the greatest reading outcomes by analysis of both midline scores only and through DiD that compared changes between base and midline across treatment and comparison schools. This indicates that EGRA has had a clear program effect in EGRA-only areas. EGRA effects in Treatment Level 1—EGRA + SSDI + INVC—or what is called as USAID/Malawi’s Country Development Cooperation Strategy (CDCS) focus area was mixed. Analysis of midline scores of learners in treatment and comparison schools showed negative correlations across standards and sexes, and the correlation was statistically significant for Standard 4 girls. The evaluation team explored a few possible causes for this difference, including determining if CDCS districts might just be lower-performing districts overall. However, the team found that this was not the case. However, analysis using DiD that compared changes between base and midline in treatment and comparison panel schools in average scores showed that Level 1 treatment schools that benefitted from all three - EGRA, SSDI and INVC - were more likely to have higher reading scores than schools in comparison areas, especially for Standard 2 learners where results were significant. The difference in magnitude of changes in average scores from base to midline in comparison and treatment schools likely explain the difference in results seen between the two methods of analysis. Interestingly, DiD results also indicated differences in effect sizes between boys and girls in Level 1, in that better effect sizes were noted for girls relative to boys. This requires further inquiry to understand reasons behind the trend since the study areas are located within the USAID/Malawi’s CDCS focus districts. It is likely that INVC and SSDI programs in addition to EGRA were more favorable to girls than boys in treatment schools located in Level 1. Analysis of midline learner scores showed that Treatment Level 2 - EGRA + SDSI - was consistently correlated with higher predicted oral reading fluency scores, and those differences were statistically significant for girls in both Standards 2 and 4. But, DiD analysis for Standard 2 showed that Level 2 treatment was associated with higher but insignificant average change in scores for boys, but lower and significant change for girls. But, the results were reversed, although insignificant, in Standard 4 where Level 2 treatment was associated with lower average change in scores for boys, but higher change in scores for girls. Analysis of midline learner scores showed that Treatment Level 3 - EGRA + INVC - was correlated with higher predicted learner reading scores for Standard 2 boys and Standard 4 girls, and results were statistically significant for Standard 4 girls. Results were negative but insignificant for Standard 2 girls and Standard 4 boys. While DiD results were similar for Standard 2 girls, they were slightly different for others. While none of the results were significant, effects were positive for boys in Standard 2, while positive for both genders in Standard 4. WHAT ARE THE INTERMEDIATE EFFECTS OF EGRA? The number of coaching visits per term appears to be negatively correlated with learner reading outcomes. This is strange and may simply be because after a certain point, the coaching efforts have a diminishing effect. SI will explore this further if this result continues at endline. Outside of this, the EGRA MOUs appear to have been very successful, with the exception of the MOU that encouraged parents to read to learners. The latter may have been less successful because the MOU is fairly new (having just been signed in the 2013-2014 academic year for Cohort A schools and the 2014-2015 academic year for Cohort B schools). Further, it likely targeted parents who were not already reading to their learners; thus, based on earlier findings, these learners were probably scoring lower on reading tests prior to the 2013-2014 85 academic year. By 2017, this MOU may likely prove to have been beneficial to learner reading scores. On the other hand, the MOU that worked to reduce class size appears to have been very successful, improving reading scores by an average of 4.4 cwpm. The MOUs to extend the school day and the length of the reading lesson also appear to have had a large effect, increasing learner reading scores by an average of 2.9 cwpm and 2.1 cwpm, respectively. Finally, the number of reading fairs the school hosted in the past two years also appears to be a good predictor of learning reading scores, with each additional fair increasing scores by almost 1 cwpm. On average, EGRA treatment school head teachers reported hosting a much larger number of fairs than did comparison-school head teachers. Treatment school head teachers reported hosting about 2.3 reading fairs in the past two year whereas comparison-school head teachers reported only hosting about half of a fair (the fraction is due to the fact that many comparison schools did not host any reading fairs in the last two years). These results suggest implementation fidelity and success of at least the MOUs and reading fairs. WHAT IS THE COST EFFECTIVENESS OF THE EGRA INTERVENTION ON LEARNER SCORES? Using the program effects calculated by SI using DiD and total direct costs obtained from RTI for year 2014-15, the evaluation team calculated cost effectiveness of EGRA intervention. The results showed that it would cost around $6.10 per learner in Standard 2 to improve by one unit in oral reading fluency in correct words per minute. For Standard 4, with the share of costs approximately allocated by SI to have been incurred by RTI in 2014 academic year, it would cost about $1.80 per learner to improve by one unit in oral reading fluency in correct words per minute. The cost effectiveness estimates, however, should be interpreted with caution, especially for scaling up, because it is early to measure impacts since intervention focus standards may change in coming years, and because direct costs were aggregated under all components (excluding labor) implemented by RTI and were approximately allocated across the two standards. At this stage, the estimates only provide some insights into cost effectiveness at early stages and also for phased implementation of EGRA by standard. The interventions to date have focused on Standards 1 and 2, and are planned to be expanded to other standards in coming years. As the project matures and is expanded to more standards, economies of scale and scope may occur leading to reduction in some costs, and also effects may improve, thus altering the cost effectiveness estimates of EGRA. WHAT ARE THE FACTORS PREDICTING DROPOUTS AND REPETTITION AT MIDLINE? Factors predicting dropouts at midline included average household wealth, household head’s level of education, learner attendance at preschool, teacher use of best practices, and number of school reading fairs. They were all negatively correlated with learner dropouts, though household wealth was the only variable that had a statistically significant correlation. Also, the average learner-to-teacher ratio for sampled classrooms predicted an increase in dropouts, meaning the more learners in a class (larger classes), the more drop out. None of the four Treatment Levels were statistically significantly correlated with dropouts. Student repetition was found to be correlated with learner access to reading materials at home, household wealth, highest level of household head’s education, whether the learner attended preschool, whether the learner was in Treatment Level 2 (learners from Treatment Levels 2 were 22 percent less likely to be repeating a standard at midline), and whether the learner was in Treatment Level 1 (learners from Treatment Level 1 were 48 percent more likely to be repeating a standard this year). 86 X. RECOMMENDATIONS Based on the above findings and conclusions, the study recommends USAID and MoEST do the following: • Build up community programs that work to get parents and household members involved in learner reading and ensure these programs encourage households to read to learners, and explain the benefits of doing so. • Consider other ways of ensuring learners are read to more often, possibly by creating after￾school peer mentoring programs. This method has been tried in many other education interventions and proved beneficial both for the mentors and mentees. • Work with schools to ensure they have enough textbooks or a system of protecting textbooks to allow learners to take books home from school with them, and encourage learners to do so—possibly through reading incentive programs such as those often used in the U.S. that provide small rewards for learners who read multiple books over school break periods (or even throughout the academic year). • Continue to work with teachers through targeted capacity-building and coaching interventions to improve teacher use of “essential” reading practices. • Train additional teachers and identify additional resources to allow schools to reduce the average learner-to-teacher ratio. This might also be accomplished by simply not pairing teachers together, but instead, having them teach their own classes or expanding EGRA MOUs to other schools. • Work with RTI to ensure all EGRA schools actually adopt the provisions of the MOUs. This means ensuring all standards are extended by an hour—not just the lower standards. This may require USAID and the MoEST working together to discuss the larger policy implications of this extended day in terms of financial costs for keeping teachers at schools longer. This also means ensuring more schools sign the MOU to reduce class sizes or split up classes between more teachers. • Identify ways to better integrate EGRA, SSDI, and INVC Projects. 87 ANNEXES ANNEX 1: STATEMENT OF WORK The following is the full statement of work (SOW) section, section C.3, from Contract number AID-612- C-13-00001. This report addresses Section 3 of this SOW only. C.3 SCOPE OF WORK The Contractor shall provide evaluation services that will include data collection, data analyses, and report writing. The contractor shall conduct evaluations, assessments and surveys in accordance to the Statement of Work (SOW) and Contract Performance Standards reflective of the Contractor’s proposed approach. The evaluation services shall include baseline data collection, tracking of key indicators on an annual /bi￾annual basis and report findings through the life of the five year EGRA and CDCS period as necessary. The data collected and analyzed will measure the impact of the USAID/Malawi Early Grade Reading Activity (EGRA), with a corresponding baseline (2013), mid-line (2015), and end-line (2017). Additional assessments and surveys conducted by the contractor of reading abilities will examine additional factors that are assumed to effect reading outcomes in Malawi. The Contractor shall provide the results of these evaluations, assessments and surveys to USAID/Malawi to inform EGRA implementation, contribute to USAID Malawi’s collaborative learning approach under the CDCS, and improve the ability of USAID to adapt to changing program needs based on data. C.3.1 Objectives The Early Grade Reading Activity (EGRA) Impact Evaluation has two main objectives: 1. To measure the impact of USAID/Malawi’s EGRA efforts in target districts on student reading outcomes, and 2. To assess the hypotheses of integration and community strengthening related to student learning in the USAID/Malawi CDCS: A. to measure how integration of USAID programming across sectors (education, health, agriculture) working in the same geographic areas impacts student reading outcomes; and B. to measure how community strengthening through capacity-building of local institutions, and promotion of citizen participation impacts sustainable of reading interventions. C.3.2 Tasks The Contractor shall provide evaluation services of four major tasks: Baseline Mid-point Endline Required Tasks May 2013 May 2014 May 2015 May 2016 May 2017 1. Evaluation of the USAID/Malawi Early Grade Reading Activity (EGRA) on Standard 2 and 4 Students Reading Outcomes X X X 88 2. Household Survey of Sub￾Sampled Standard 2 and 4 Students.X X X 3. National Reading Assessment for Standards 1 and 3 students X X 4.Final Impact Evaluation of EGRA and CDCS Hypotheses X TASK 1: EVALUATION OF THE USAID/MALAWI EARLY GRADE READING ACTIVITY (EGRA) ON STANDARD 2 AND 4 STUDENTS READING OUTCOMES. 1.1 Overview The Contractor shall collect data, prepare analyses, and reports of Standard 2 and 4 reading outcomes. The Contractor shall conduct all the necessary data collection, data analysis and report writing. The Contractor shall measure the impact of the USAID/Malawi Early Grade Reading Activity22 in target districts compared to control districts. The Contractor shall have all data collected for Task 1 Evaluation by May 2013, May 2015, and May 2017, respectively. 1.2 General Approach The contractor will implement activities under this Task in accordance with USAID principles and requirements, including those outlined in the USAID’s Evaluation Policy and ADS 203. Prior to carrying out the evaluation, the contractor shall submit to the USAID Contracting Officer’s Representative (COR) an annual Work Plan that details the work to be conducted. The Contractor will use an evaluation design that best meets USAID evaluation policy standards and principles. The design will ensure reliability and validity of the data collected and allow disaggregation by sex. The design will enables analysis of USAID/Malawi’s CDCS hypotheses of integration and community strengthening focus on geographic regions as outlined in Section C2.1. The design shall enables analysis to determine variation in outcomes based on level of integration of USAID/Malawi sectoral and geographic integration (Level I: those in Mission integrated intervention districts (Lilongwe, Balaka, and Machinga); Level II: districts/zones where education intervention overlaps with either FtF or GHI; Level III: education only intervention districts; and Level IV: control schools in non–intervention districts). The contractor will assess Standard 2 and 4 students to determine their reading ability. The samples will include sufficient numbers of students to disaggregate by Standard and by sex. The Contractor will conduct classroom observations in at least one Standard 2, and 4 classroom and interview the head teacher of each school. The classroom based assessment shall be developed in close collaboration with the Malawi National Examination Board and the Department of Inspection and Advisory Services (DIAS) to ensure that it is grade and curriculum appropriate and will at a minimum measure 22 See RFP No. SOL-612-13-000003 for detailed description of intervention 89 early grade reading skills. Data on number of students in the class (classroom size) and its relationship to reading outcomes must be included in the assessment. The Contractor will disseminate the annual results generated from the data collection to key stakeholders, including USAID and its implementing partners, the MoEST, Development Partners, and the larger early grade reading community of practice. These reports will be due to USAID/Malawi by July 31 of each year. This information will provide the basis for learning and adaptive programming decisions to ensure that the program remains flexible to changing needs learned throughout the course of the evaluation. The contractor shall further match schools in the intervention districts with schools in the non – intervention or control districts to allow for comparability. In matching the schools, the contractor shall use scientific matching methods such as propensity score matching or other scientifically rigorous methods. Baseline data collection may require oversampling to determine appropriate control districts. The following are illustrative examples that could be used: Student Level Data such as: a. Participation in early childhood development (ECD) program b. Participation in a school feeding program c. Time spent in the classroom on reading instruction School Level Data such as: a. Student to qualified teacher ratio, b. Dropout rate c. Repetition rate d. Average number of students per class e. Timing of school feeding in the school timetable f. Absenteeism rates, and g. Average number of teacher supervision/coaching visits to the teacher h. Other interventions including: classroom block and teacher housing construction, disability education interventions, complementary basic education, child-friendly schools. i. Text availability: textbook to student ratio j. Level of print rich environment found in the classroom. k. Language of instruction in the classroom Community Level Data such as: a. Beneficiary of GHI programming (note: will need to be triangulated with USAID health team data as households may not be aware of GHI investments they are benefitting from) b. Beneficiary of FtF programming (note: will need to be triangulated with USAID FtF team data as households may not be aware of FtF investments they are benefitting from) c. If secondary data source is available: d. Prevalence of stunting, wasting, or underweight 1.3 Evaluation Questions At a minimum, the classroom based assessment will report on how the USAID/Malawi EGRA impacts, at a minimum, the following indicators: (i) Proportion of primary school students who are able to read with comprehension, according to their countries’ curricular goals, by the end of lower primary school (Standard 4). 90 a. The proportion of students who by the end of the fourth school year (Standard 4) are able to read grade level text, as measured by the number of correct words per minute b. The proportion of students who by the end of the fourth school year (Standard 4) are able to answer comprehension questions after reading grade level text, as measured by the number of correct comprehension questions answered correctly. (ii) Proportion of students, who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text (Standard 2). a. The proportion of students who by the end of the second school year (Standard 2) are able to read grade level text, as measured by the number of correct words per minute b. The proportion of students who by the end of the second school year (Standard 2) are able to answer comprehension questions after reading grade level text, as measured by the number of correct comprehension questions answered correctly. 1.4 Sampling Frame The Evaluation sampling will be of sufficient size to disaggregate by district, by sex, and by Standard. The sampling framework will enable analysis to examine how levels of integration of Mission programming across sectors in various districts impacts learning outcomes differently. The sample will include a minimum of 30 schools randomly selected per district. The schools shall be selected between control and treatment schools that ensures comparability and disaggregation by the various levels of geographic integration: Level 1: those in Mission integrated intervention districts; Level 2: districts/zones where education intervention overlaps with either FtF or GHI; 3: education only intervention districts; and Level 4: control schools in non–intervention districts. This sample shall be scientifically representative. From the sampled schools (control and treatment), the contractor shall randomly draw a representative sample of children per Standard 2 and Standard 4. From each selected school, at a minimum, a random selection of 10 students, equal numbers of boys and girls, will be selected from Standard 2 and 4 for inclusion in the assessment. The Contractor will at a minimum include a classroom observation of a Standard 2 and 4 classroom and conduct interviews with a Standard 2 and 4 teacher and the head teacher for each school visited. If a different sample size is needed to achieve the requirements of this SOW, the Contractor shall provide justification based on power and confidence of estimation to the COR for approval. The Contractor will at a minimum draw from all four levels of USAID/Malawi geographic integration for analysis of the USAID/Malawi EGRA and to test the CDCS hypothesis. The levels include: Level 1: Mission integrated districts and zones in Lilongwe, Balaka, and Machinga. The sample will also draw from Level 2: zones within Salima and Ntcheu to examine where the Early Grade Reading intervention has overlap with either FtF or Health interventions. The sample will be required to draw on two districts from Level 3: education intervention only districts (these include Mzimba North, Ntchisi, Zomba Rural, Blantyre Rural, and Thyolo). To determine the control districts, the sample will draw at a minimum three additional Level 4: districts receiving no early grade reading interventions from the remaining 24 education districts using a matched pair approach that enables the comparison of effects across intervention and non-intervention districts. Where possible for data on community and household-level variables, the Contractor shall utilize secondary data sources such as national or population-level demographic and economic surveys, data from Education Management Information System collected by the Ministry of Education, Science and Technology (MoEST) annually or other USAID or donor-supported household surveys. Specifically, the USAID SSDI activity is a potential source for health-related data in target areas, and the USAID FtF impact evaluation is a potential source of data on agriculture and socio-economic variables in target communities. 91 To enhance comparability of study data with other USAID data analyses, all questionnaires shall include appropriate geo- referenced data. The Contractor will disseminate the annual results generated from the data collection to key stakeholders, including USAID and its implementing partners, the MoEST, Development Partners, and the larger early grade reading community of practice. These reports will be due to USAID/Malawi by July 31 of each year. This information will provide the basis for learning and adaptive programming decisions to ensure that the program remains flexible to changing needs learned throughout the course of the evaluation. Additionally, the final impact evaluation report will be presented to key stakeholders and disseminated widely to encourage sharing of results, lessons learned and best practices and identify USAID achievements under the CDCS targets. TASK 2: HOUSEHOLD SURVEY OF SUB-SAMPLED STANDARD 2 AND 4 STUDENTS. 2.1. Overview The Contractor shall collect data, prepare analyses, and reports of a randomly selected sub-sample of students assessed in Task 1. The student’s sub-sample will equally represent male and female students. The contractor will conduct a household survey of a sub-sample of children assessed in Task 1 to understand the dynamics and effects of other factors that contribute to children reading outcomes. The Contractor will use data collected from the household survey to isolate household and socio-economic related factors. The Contractor will collect data at the household level to reduce external bias and measure potential multiplier effects of complementary Mission interventions at the community and household level of USAID’s programming under the Global Health Initiative (GHI) and Feed the Future (FtF). The Contractor shall incorporate data on relevant multiplier, socioeconomic and household factors and select appropriate control/comparison districts and communities to detect differential effects. The Contractor shall utilize secondary data sources from the GoM, USAID, or other sources to the greatest extent possible. 2.2. General Approach The contractor will implement activities under this Task taking into account USAID principles and requirements, including those with USAID’s Evaluation Policy and ADS 203. The Contractor will measure the Early Grade Reading Activity’s efforts to increase parental and community engagement in supporting student reading. The Contractor will assess how social mobilization of parents, guardians, communities and other relevant stakeholders for supporting children reading has changed household behaviors and student learning outcomes. The Contractor will account for activities within the community that bridge schools and communities around reading or provide alternatives sources of reading support to students. The Contractor will examine the dynamics and effects of other factors that contribute to children learning outcomes within the household and community. The household survey of sampled Standard 2 and 4 children will isolate households and socio- economic related factors, enabling the analysis to linking children’s reading performance to household factors and community factors. The Contractor will include appropriate geo-referenced data to enhance comparability of study data with other USAID data analyses. “The Contractor will disseminate the annual results generated from the data collection to key stakeholders, including USAID and its implementing partners, the MoEST, Development Partners, and the larger early grade reading community of practice. These reports will be due to USAID/Malawi by August 31 of each year. This information will provide the basis for learning and adaptive programming decisions 92 to ensure that the program remains flexible to changing needs learned throughout the course of the evaluation.” Prior to carrying out the household survey, the contractor shall submit to the USAID Contracting Officer’s Representative (COR) a detailed annual Work Plan describing the work to be conducted. The Contractor will use an evaluation design that best meets USAID evaluation requirements and is robust enough to measure the complexity of integration. 2.3. Evaluation Questions 1. What household and community factors relate to student reading outcomes? 2. What level of household and community resources/factors are dedicated to schooling and reading? 3. How have Health and Agricultural interventions at the household and community level affected schooling and reading outcomes? 4. What factors at the household and community level have been identified that relate to repetition and drop out and are there sex differences at the household level? Illustrative indicators of interest include: • Participation in early childhood development (ECD) program • Participation in a school feeding program • Timing of school feeding in the school timetable • Family/household level variables for sub-group • Parental literacy o Household size o Food security • Number of times child ate breakfast before school or the number of missed meals in the past week • Incidence of diarrhea in past 2 weeks • Number of days of school missed due to illness • Number of days of school missed due to family/farm responsibilities • Health factors • Practice of key nutrition, water, sanitation and hygiene (WASH) behaviors related to school access(particularly hand washing, latrine use, micronutrient supplementation, and malnutrition) • Water access and quality, including access to a protected water source, and time required to access water • Access to child health services targeted by USAID programs • Access to de-worming • Other relevant health factors which may be related to early grade reading • Socio-economic variables • School infrastructure, including water, sanitation, and hygiene facilities, which are particularly factors relevant to access and retention of girls and people with disabilities • Average household time spent supporting child reading, and School level related data such as: • Student to qualified teacher ratio, • Dropout rate • Repetition rate • Classroom size, • Absenteeism rates, and • Average number of teacher supervision/coaching visits to the teacher • Other interventions including: classroom block and teacher housing construction, disability education interventions, complementary basic education, child-friendly schools. 93 Community-level variables • Beneficiary of GHI programming (note: will need to be triangulated with USAID health team data as households may not be aware of GHI investments they are benefitting from) • Beneficiary of FtF programming (note: will need to be triangulated with USAID FtF team data as households may not be aware of FtF investments they are benefitting from) • If secondary data source is available: • Prevalence of stunting, wasting, or underweight 2.4. Sampling Frame The Contractor will sample a sub-group of the students assessed in Task 1 to understand the dynamics and effects of other factors that contribute to children’s learning outcomes. The contractor will select Standard 2 and 4 students and their households to participate in the household survey. The sample will link children and households within communities to isolate household and community socio-economic related factors. The sample will link children’s reading performance to household and community factors. The Contractors sample size must adhere to criteria determined to have sufficient power and confidence of estimation. The sub-sample should come directly from the sampled schools and students being assessed under Task 1 of this Contract. In determining the sampling framework, the Contractor will take into account the Mission’s CDCS development hypothesis on education interventions and outcomes – including integrating USAID FtF, GHI, and education programs in the same geographic regions. The Contractors sampling framework will enable USAID to examine its investments in community participation and institutional capacity development within education programs to test the validity of the CDCS hypothesis related to the education sector in Malawi. Where possible for data on community and household-level variables, the Contractor shall utilize secondary data sources such as national or population-level demographic and economic surveys, data from Education Management Information System collected by the MoEST annually or other USAID or donor￾supported household surveys. Specifically, the USAID SSDI activity is a potential source for health-related data in target areas, and the USAID FtF impact evaluation is a potential source of data on agriculture and socio-economic variables in target communities. To enhance comparability of study data with other USAID data analyses, all questionnaires shall include appropriate geo-referenced data. TASK 3: NATIONAL READING ASSESSMENT FOR STANDARDS 1 AND 3 STUDENTS 3.1 Overview The Contractor shall collect data, prepare analyses, and reports that provide a snapshot of Standard 1 and 3 student reading skills. For Task 3, the contractor will conduct a nationally representative high quality reading assessment of Standard 1 and 3 students at the end of the academic year in 2014 and 2016. This assessment will be conducted near the end of the school year on a nationally representative sample of Standard 1 and 3 students. The assessment will allow for a comparison of results over time. 23 23 USAID has conducted annual assessments in 2010, 2011, and 2012 on students at the beginning of Standard 2 and 4 through the Malawi Teacher Professional Development Support Activity. The assessments tested the same nine reading skills in Chichewa and included letter naming, syllable segmentation, initial sound identification, syllable reading, familiar word reading, nonsense word reading, oral reading fluency, reading comprehension, and listening comprehension. 94 3.2 General Approach The contractor will implement activities under this Task taking into account USAID principles and requirements, including those with USAID’s Evaluation Policy and ADS 203. Prior to carrying out the assessment, the contractor shall receive approval of an Annual Work plan by the USAID Contracting Officer’s Representative (COR) that provides a detailed description of the work to be conducted. The Contractor will use its expertise conduct the evaluation design that best meets USAID evaluation policy standards and principles. The Contractor’s approach will allow comparison of results over time. The contractor will conduct a nationally representative reading assessment during the third term, in April or May of 2014 and 2016. The Contractor shall conduct the reading assessments in at least three national languages, Chichewa, Tumbuka, and Yao. The Contractor will assess students in the dominant language of instruction at the school where the assessment will be administered. For assessments in additional languages the contractor will be required to ensure comparability of assessments between languages to enable national aggregation. The reading skills assessed, must at a minimum include letter naming, syllable reading, familiar word reading, nonsense word reading, oral reading fluency, and reading comprehension. The Contractor will work closely with the Host Country partners, including: the MoEST, Directorate of Basic Education, Directorate of Inspection and Advisory Services, Teacher Training Colleges, Malawian University faculties of education, Directorate of Planning, particularly the Education Management Information Systems Unit, Malawi National Examination Board, and Primary Education Advisors during development, assessment, and analysis of the national assessment. The Contractor will work with Host Country partners and use appropriate Host Country institutions to build the capacity of the MoEST to sustainably implement early grade reading assessments. The Contractor will administer the reading assessment of students with an accompanying questionnaire that investigates various aspects of the student’s backgrounds that could potentially be associated with performance. The Contractor will conduct teacher and head teacher interviews based on a standardized classroom observation and interview protocol in each sampled school. The contractor will be responsible for training all evaluators and assuring the strictest adherence to ensure validity and reliability of the assessment, and the protection of human subjects. During administration of these assessments the Contractor shall ensure that all personnel are adequately trained in assessing children’s reading abilities. The Contractor will include data on classroom size and its relationship to reading outcomes in the assessment. Primary data collection processes and exact data collection will be managed by the Contractor with concurrence from USAID/Malawi and will correspond directly to the methodological approach, sample size, and evaluation team size required to adhere to reporting deadlines. To enhance comparability of study data with other USAID data analyses, all questionnaires shall include appropriate geo-referenced data. Secondary data which are available from other national or USAID-supported household surveys will be incorporated wherever possible in lieu of original data collection. The Contractor will disseminate the annual results generated from the data collection to key stakeholders, including USAID and its implementing partners, the MoEST, Development Partners, and the larger early grade reading community of practice. These reports will be due to USAID/Malawi by July 31 of each year. This information will provide the basis for learning and adaptive programming decisions to ensure that the program remains flexible to changing needs learned throughout the course of the evaluation. Additionally, the final impact evaluation report will be presented to key stakeholders and disseminated widely to encourage sharing of results, lessons learned and best practices and identify USAID achievements under the CDCS targets. 3.3 Evaluation Questions 95 Malawi has set benchmarks and targets for performance in Standards 1 through 3 for reading by the MoEST-convened National Early Grade Reading Coordination Committee. In 2011, more than 10% of students reached the benchmarks for letter naming in Standard 4, syllable segmentation in Standards 2 and 4, and listening comprehension in Standards 2 and 4. For the other subtests, the percentages reaching the benchmark were considerably lower. Less than 10% of students reached the level expected in subtests that required decoding, which included syllable reading, familiar word reading, nonsense word reading, and oral reading fluency (with comprehension). The National Reading Assessment will examine how Malawian Primary Students in Standard 1 and 3 are progressing towards reaching MoEST benchmarks in reading through the following questions: (i) Proportion of primary school students who are at the Standard 1 benchmarks for reading skills. a. The breakdown of students grouped by sub-divisions and progress towards attaining benchmarks in Standard 1. b. The relationship of Standard 1 reading skill acquisition to additional factors that relate to or predict achievement, including classroom size for Standard 1. (ii) Proportion of primary school students who are at the Standard 3 benchmarks for reading skills. a. The breakdown of students grouped by sub-divisions and progress towards attaining benchmarks in Standard 3. b.The relationship of Standard 3 reading skill acquisition to additional factors that relate to or predict achievement, including classroom size for Standard 1. 3.4 Sampling Frame The Contractor shall conduct a nationally representative snapshot of early grade reading skills for Standard 1 and 3 students. The Contractor will select at a minimum two districts randomly within each of the six educational divisions. At a minimum, 30 schools within each district will be randomly selected and ten students per Standard 1 and 3 will be assessed. The Contractors sampling framework will allow adequate disaggregation by sex, urban and rural, and educational division. The Contractor’s sampling framework will clearly identify power calculations for sample sizes used and demonstrate acceptable levels of statistical power for interpretation of results. The Contractor will Contractor shall provide a justification based on power and confidence of estimation for all sample sizes to be used during the annual Work Plan. The contractor shall conduct the nationally representative assessment of student reading abilities in Standards 1 and 3 in 2014 and 2016. The Contractor shall provide a national snapshot of early grade reading outcomes for Standard 1 and 3 students. The Contractor will use the data collected and corresponding results, findings, conclusions, and recommendations to inform and monitor Malawi’s progress in improving early grade reading skills for Malawian primary students in line with Malawi Global Partnership for Education targets and objectives. The Contractor will provide evidence on best practices, lessons learned and cost-effectiveness approaches identified through the nationally representative assessment that correlate to improved reading skills in a low-resource setting such as Malawi directly drawn from the data. TASK 4: FINAL IMPACT EVALUATION OF EGRA AND CDCS HYPOTHESES 4.1 Overview The Contractor shall collect data, prepare analyses, and reports that provide an overall analysis of the USAID Early Grade Reading Activity and the USAID/Malawi CDCS hypotheses related to education. The Contractor shall measure the impact of the USAID/Malawi Early Grade Reading Activity in target districts and the Hypotheses of the USAID/Malawi CDCS as outlined in Section C2.1. The Final Impact Evaluation will draw from the data collected over the life of the contract to answer the evaluation questions below. 96 4.2 General Approach The contractor will implement activities under this Task taking into account USAID principles and requirements, including those outlined in USAID’s Evaluation Policy and ADS 203. Prior to carrying out the assessment, the Contractor shall receive approval of the Annual Work Plan from the USAID Contracting Officer’s Representative (COR) that provides detailed description of the work to be conducted. The Contractor’s evaluation design will be in compliance with USAID evaluation policy standards and principles. The Contractor’s approach will evaluate the impact of the USAID Early Grade Reading Activity and the hypotheses of the USAID/Malawi CDCS related to integration and community engagement as outlined in Section C2.1. The design shall enables analysis to determine variation in outcomes based on level of integration of USAID/Malawi sectoral and geographic integration (Level I: those in Mission integrated intervention districts (Lilongwe, Balaka, and Machinga); Level II: districts/zones where education intervention overlaps with either FtF or GHI; Level III: education only intervention districts; and Level IV: control schools in non–intervention districts). The contractor will assess Standard 2 and 4 students to determine their reading ability. The samples will include sufficient numbers of students to disaggregate by Standard and by sex. The Contractor will conduct classroom observations in at least one Standard 2, and 4 classroom and interview the head teacher of each school. The classroom based assessment shall be developed in close collaboration with the Malawi National Examination Board and the Department of Inspection and Advisory Services (DIAS) to ensure that it is grade and curriculum appropriate and will at a minimum measure early grade reading skills. Data on number of students in the class (classroom size) and its relationship to reading outcomes must be included in the assessment. The Contractor will disseminate the annual results generated from the data collection to key stakeholders, including USAID and its implementing partners, the MoEST, Development Partners, and the larger early grade reading community of practice. These reports will be due to USAID/Malawi by August 31 of 2017. This information will provide the basis for learning and adaptive programming decisions to ensure that the program remains flexible to changing needs learned throughout the course of the evaluation. The contractor shall further match schools in the intervention districts with schools in the non – intervention or control districts to allow for comparability. In matching the schools, the contractor shall use scientific matching methods such as propensity score matching or other scientifically rigorous methods. Baseline data collection may require oversampling to determine appropriate control districts. The following are illustrative examples that could be used: Student Level Data such as: a. Participation in early childhood development (ECD) program b. Participation in a school feeding program c. Time spent in the classroom on reading instruction School Level Data such as: a. Student to qualified teacher ratio, b. Dropout rate c. Repetition rate d. Average number of students per class 97 e. Timing of school feeding in the school timetable f. Absenteeism rates, and g. Average number of teacher supervision/coaching visits to the teacher h. Other interventions including: classroom block and teacher housing construction, disability education interventions, complementary basic education, child-friendly schools. i. Text availability: textbook to student ratio j. Level of print rich environment found in the classroom. k. Language of instruction in the classroom Community Level Data such as: a. Beneficiary of GHI programming (note: will need to be triangulated with USAID health team data as households may not be aware of GHI investments they are benefitting from) b. Beneficiary of FtF programming (note: will need to be triangulated with USAID FtF team data as households may not be aware of FtF investments they are benefitting from) c. If secondary data source is available: d. Prevalence of stunting, wasting, or underweight 4.3 Evaluation Questions The Contractor must at a minimum, address the following questions over the life of the award: i. What is the USAID/Malawi Early Grade Reading Activity’s impact on children’s (disaggregated by sex) reading abilities in terms of the following: a. Level of effort of reading instruction’s impact on children reading abilities b. Effect of the use of mother tongue or local language text on reading outcomes c. Effect of extra-curricular reading activities d. Effect of time on task in improving reading outcomes ii. Which components have the largest effects and what is the relative cost effectiveness of these various components? iii. How does teachers’ classroom behavior and practices impact on the ability of children to read? a. How did the level of coaching impact teacher behavior and student reading outcomes? iv. How does the level of integration with other USAID/Malawi FtF and GHI programs, and other related DP interventions in the target districts, impact the reading outcomes of students? a. What interactions can be identified with other major USAID/Malawi Mission interventions in agriculture and health? b. What other multiplier effects have been identified over the life of the Early Grade Reading Activity? c. What are the key external factors that were found to have a multiplier effect, i.e. early childhood development (ECD) attendance, participation in school feeding, change in WASH behaviors, access to a secondary school, etc.? d. How does the provision of non – cash incentives to performing teachers and schools translate into changes in children’s reading abilities? v. What secondary effects can be attributed to the Early Grade Reading Activity? a. Impact on repetition rate b. Impact on dropout rate c. Impact on school completion, particularly for girls and students with disabilities vi. What is the effect of USAID/Malawi investments in institutional capacity-building and community engagement to improve community participation on the effectiveness and sustainability of USAID Education programs and learning outcomes? 98 The Contractor’s approach will adequately answer these evaluation questions at baseline (2013), two years after baseline (2015), and four years after baseline (2017), with a detailed methodological approach that uses impact evaluation methodologies be it quantitative, qualitative, or mixed methods. The Contractor will use existing data to the greatest extent possible using impact evaluation methodology where appropriate. The Contractor will use primary and secondary data to answer evaluation questions. Where existing data is insufficient, the Contractor will purposefully sample districts and schools (and their surrounding communities) via based on sampling methods that draw conclusions to inform the evaluation questions The Contractor shall use a quasi-experimental design to clearly demonstrate the impact of program interventions on reading outcomes, and to test the CDCS hypotheses and enable identification of differential impacts that result from geographic integration with GHI and FtF programming. The Contractor shall address evaluation questions related to integration, capacity- building and community participation, as well as identifying best practices and lessons learned. The Contractor’s research design will be conducted over a five year period. The Contractor will provide a baseline, mid-line, and end-line data points. USAID/Malawi reserves the right to have the ultimate authority to approve the evaluation design prior to the roll out of the evaluation. 4.4 Sampling Frame The Contractor will use a sampling framework that is of sufficient size to disaggregate by district, by sex, and by Standard. The Contractor’s sampling framework will enable analysis to examine how levels of integration of Mission programming across sectors in various districts impacts learning outcomes differently. The sample will include a minimum of 30 schools randomly selected per district. The schools shall be selected between control and treatment schools that ensures comparability and disaggregation by the various levels of geographic integration: Level 1: those in Mission integrated intervention districts; Level 2: districts/zones where education intervention overlaps with either FtF or GHI; 3: education only intervention districts; and Level 4: control schools in non–intervention districts. This sample shall be scientifically representative. From the sampled schools (control and treatment), the contractor shall randomly draw a representative sample of children per Standard 2 and Standard 4. From each selected school, at a minimum, a random selection of 10 students, equal numbers of boys and girls, will be selected from Standard 2 and 4 for inclusion in the assessment. The Contractor will at a minimum include a classroom observation of a Standard 2 and 4 classroom and conduct interviews with a Standard 2 and 4 teacher and the head teacher for each school visited. The Contractor will sample a sub-group of the students assessed to understand the dynamics and effects of other factors that contribute to children’s learning outcomes. The contractor will select Standard 2 and 4 students and their households to participate in the household survey. The sample will link children and households within communities to isolate household and community socio-economic related factors. The sample will link children’s reading performance to household and community factors. The Contractors sample size must adhere to criteria determined to have sufficient power and confidence of estimation. The sub-sample should come directly from the sampled schools and students being assessed under Task 1 of this Contract. If a different sample size is needed to achieve the requirements of this SOW, the Contractor shall provide justification based on power and confidence of estimation to the COR for approval. In determining the sampling framework, the Contractor will take into account the Mission’s CDCS development hypothesis on education interventions and outcomes – including integrating USAID FtF, GHI, and education programs in the same geographic regions. The Contractors sampling framework will enable USAID to examine its investments in community participation and institutional capacity development within education programs to test the validity of the CDCS hypothesis related to the education sector in Malawi. The Contractor will at a minimum draw from all four levels of USAID/Malawi geographic integration for analysis of the USAID/Malawi EGRA and to test the CDCS 99 hypothesis. The levels include: Level 1: Mission integrated districts and zones in Lilongwe, Balaka, and Machinga. The sample will also draw from Level 2: zones within Salima and Ntcheu to examine where the Early Grade Reading intervention has overlap with either FtF or Health interventions. The sample will be required to draw on two districts from Level 3: education intervention only districts (these include Mzimba North, Ntchisi, Zomba Rural, Blantyre Rural, and Thyolo). To determine the control districts, the sample will draw at a minimum three additional Level 4: districts receiving no early grade reading interventions from the remaining 24 education districts using a matched pair approach that enables the comparison of effects across intervention and non- intervention districts. Where possible for data on community and household-level variables, the Contractor shall utilize secondary data sources such as national or population-level demographic and economic surveys, data from Education Management Information System collected by the MoEST annually or other USAID or donor-supported household surveys. Specifically, the USAID SSDI activity is a potential source for health-related data in target areas, and the USAID FtF impact evaluation is a potential source of data on agriculture and socio-economic variables in target communities. To enhance comparability of study data with other USAID data analyses, all questionnaires shall include appropriate geo-referenced data. The Contractor will disseminate the annual results generated from the data collection to key stakeholders, including USAID and its implementing partners, the MoEST, Development Partners, and the larger early grade reading community of practice. These reports will be due to USAID/Malawi by August 31 of 2017. This information will provide the basis for learning and adaptive programming decisions to ensure that the program remains flexible to changing needs learned throughout the course of the evaluation. Additionally, the final impact evaluation report will be presented to key stakeholders and disseminated widely to encourage sharing of results, lessons learned and best practices and identify USAID achievements under the CDCS targets. 100 ANNEX 2: A BRIEF EXPLANATION OF THE STAGES OF LEARNING TO READ AND THE CORRESPONDING READING ASSESSMENT TASKS Regardless of the language, all children who learn to read advance from being pre-readers to initial readers to fluent readers. At each stage, they develop a different set of competencies, from oral language speaking and listening skills in the first stage, to initial decoding skills in the second stage, to achieving reading fluency and comprehension in Stage 3. The EGRA RA measures children’s abilities according to these three stages of reading development:24 Stage 1: Pre-Reading Skills. In Stage 1, which typically lasts from birth to kindergarten, children learn oral language skills. Oral language skills are comprised of both listening comprehension and phonemic awareness skills (the ability to hear and manipulate sounds in spoken words). Testing of listening comprehension and phonemic awareness skills is critical because they are pre-requisites to reading skills acquisition. One of the most compelling findings in beginning reading research is that phonemic awareness is a strong predictor of early reading success.25 These pre-reading skills are measured through three orally administered subtasks: (1) Listening Comprehension, (2) Syllable Segmentation, and (3) Initial Sound Identification. Stage 2: Initial Reading Skills. This stage consists of phonics, or alphabetic understanding, and decoding skills. In Stage 2, children should learn how to associate letters with their corresponding sounds. Understanding that there is a clear link between a letter and a sound is known as alphabetic understanding, or phonics.26 This is the second phase of initial reading, because it builds on the concept of identifying sound patterns in speech by connecting the sound patterns to the printed letters. Identification of letters and sounds is measured through the Letter Name Knowledge and Syllable Reading subtasks. Stage 2 also measures whether children can read sight words and whether they can read unfamiliar words by blending and segmenting sounds into recognizable words (decoding skills). The Familiar Word Reading and Non-Word Reading subtasks measure these skills. Stage 3: Reading Fluency and Comprehension Skills. Once beginning readers have recognized speech sounds (pre-reading) and developed decoding skills (initial reading), the third stage is reading with enough fluency and automaticity to retain words long enough in memory to comprehend what is read. Reading fluency is defined as the ability to read orally aloud or silently with speed, accuracy, and proper expression; reading comprehension is the ability to connect sentences, infer new words from the context, and derive meaning. Thus, reading fluency with comprehension is defined as the ability to decode and comprehend text at the same time.27 Because oral reading fluency and reading comprehension capture this complex process, these two subtasks can be used to characterize overall reading competency. Fluency and comprehension skills are measured through the Oral Reading Fluency and Reading Comprehension subtasks. As many factors affect children’s reading progress, when a child progresses from one stage to the next or achieves reading fluency varies. As observed in a 2015 USAID/Dominican Republic final performance evaluation,28 children can achieve reading comprehension with grade-level text as early as first grade if they are taught in their mother tongue and supplied with highly qualified teachers, classroom libraries, and an effective reading methodology. However, in countries with less than ideal conditions, learners should transition from “learning to read” to “reading to learn” by the end of Grade 3 at the latest in order to be prepared to grasp subject-specific knowledge such as science and history in Grade 4. While the Reading Comprehension subtask may assess whether children can read fluently and comprehend texts, the EGRA tool does not measure content knowledge linked to the curriculum. 24 J. Chall (1983), Stages of Reading Development (New York: McGraw-Hill). 25 L. L. Edwards, D.C. Simmons, and M. D. Coyne (2005), “Beginning Reading.” In Encyclopedia of Behavior Modification and Cognitive Behavior Therapy, vol. 3, ed. M. Hersen, J. Rosqvist, A. M. Gross, R. S. Drabman, G. Sugal, and R. Horner (Thousand Oaks, CA: Sage). 26 Ibid. 27 National Institute of Child Health and Human Development (2000), Teaching Children to Read: An Evidence-based Assessment of the Scientific Research Literature on Reading and Its Implications for Reading Instruction (NIH Publication No. 00-4769; Washington, DC: U.S. Government Printing Office); S. Samuels (2006), “Toward a Model of Reading Fluency,” in What Research Has to Say about Fluency Instruction, ed. S. J. Samuels and A. E. Farstrup (Newark, DE: International Reading Association), 24–46. 28 B. Sinclair, J. Campos, and M. Kimsey (2015), Final Performance Evaluation of USAID/Dominican Republic’s Effective Schools Program (ESP) (Washington, DC: USAID/Dominican Republic). 101 ANNEX 3: PILOT OF TWO EGRA TOOLS: RESULTS When learning scores are compared across time to infer impacts and the scores are obtained through different forms of EGRA tools at various rounds of assessments, an equating procedure is needed to convert scores from multiple forms of a test to the same common measurement scale. Ideally, multiple test forms could be created to be similar such that equating is not necessary. But, it is not always practical and therefore equating will be needed. The conversion process obtained through equating adjusts for any difficulty differences existing between forms so that a score on one form can be equated to its equivalent value on another form. As a result, equating makes it possible to estimate the score that a person taking one test form would have received had they taken a different test form. In other words, equating ensures that any differences in scores from the baseline to mid/endline are due to achievement gains (or student ability) as opposed to differences in test difficulty. The most commonly equated EGRA measure has consistently been ORF (USAID, EdData). 29 In order to equate across the two tools used by SI at base and midline, we collected data during a pilot in 2015, prior to midline assessment, with 304 learners from 21 schools. The sample was evenly divided between boys and girls as well as Standard 2 and Standard 4 learners. Each learner was given both EGRA forms of tests, one after the other. Enumerators followed the same protocols for the pilot as for the main data collection, so learners were chosen at random for assessment within gender and standard. The piloted schools were located in Lilongwe Urban and Lilongwe Rural districts. The pilot was conducted on April 16–17, 2015 and all IKI and MoEST enumerators participated in the pilot. The results from the pilot are discussed below. Oral Reading Fluency Score Distribution The ORF subtask was scored using time. There was a total of 603 observations. After removing zero scores, it reduced to 294. Distribution of scores across base and midline tools were comparable to each other, with and without zero scores (shown below). Other subtask score distributions also closely resembled those of the oral reading fluency, and there were no notable differences between versions. Figure 26: ORF Score Distributions Zero Scores Included Zero Scores Excluded Baseline Version Midline Version 29 USAID, EdData II: Education Data for Decision-Making Equating Across Applications, Led by RTI International. www.eddataglobal.org. 102 Reliability To measure reliability within an EGRA version, SI ran Cronbach's alpha test using non-zero scores. Cronbach’s alpha is an estimate of the inter-subtask correlation (of one subtask to another) within the same test. Theoretically, this is a measure of how consistently each individual subtask measures the common, underlying psychometric construct — literacy, in this case. Here, SI found both base and midline versions had alphas at or very close to 0.8, indicating a high degree of internal consistency both within the baseline and midline EGRA. Table 47: Baseline Reliability ITEM OBSERVATIONS ITEM-TEST CORRELATION ITEM-REST CORRELATION AVERAGE INTER-ITEM COVARIANCE ALPHA Listening Comprehension 293 0.2964 0.2825 125.8196 0.8722 Oral Reading Fluency 293 0.936 0.904 79.3217 0.8079 Oral Reading Comprehension 293 0.8233 0.8193 123.7721 0.8688 Non-Word Reading 283 0.8799 0.8336 87.81965 0.8200 Familiar Word Reading 293 0.9115 0.875 85.29577 0.8149 Syllable Reading 284 0.9313 0.872 66.29874 0.8200 Initial Sound Recognition 293 0.3651 0.3454 124.5025 0.8703 Syllable Segmentation 293 0.2851 0.2492 124.1854 0.8706 Letter Name Knowledge 286 0.9166 0.856 71.02858 0.8159 Test scale 98.62917 0.8602 Table 48: Midline Reliability ITEM OBSERVATI ONS ITEM-TEST CORRELATION ITEM-REST CORRELATI ON AVERAGE INTER￾ITEM COVARIAN CE ALPHA Listening Comprehension 293 0.4175 0.4026 103.9281 0.8521 Oral Reading Fluency 293 0.8792 0.8185 66.68926 0.7913 Oral Reading Comprehension 293 0.7041 0.6986 103.4503 0.8509 Non-Word Reading 269 0.9106 0.8842 76.37411 0.7986 Familiar Word Reading 293 0.9353 0.9068 67.49791 0.7827 Syllable Reading 270 0.9419 0.8891 51.98254 0.7841 Initial Sound Recognition 293 0.3029 0.2801 103.7106 0.8519 Syllable Segmentation 293 0.299 0.2593 103.1392 0.8518 Letter Name Knowledge 274 0.8489 0.7299 59.90009 0.8130 Test scale 81.76275 0.8416 Correlation To further check for similarity in versions, SI conducted rank ordered correlation tests for each of the nine subtasks using non-zero scores. The correlations below show that midline scores were highly correlated with baseline scores, and vice versa. Table 49: Rank-ordered Correlation Coefficients EGRA SUBTASK CORRELATION, BASELINE TO MIDLINE SCORE Syllable Segmentation 0.9783 Initial Sound Recognition 0.9801 Oral Reading Comprehension 0.8905 Listening Comprehension 0.8975 Non-Word Reading 0.8372 Oral Reading Fluency 0.9819 Familiar Word Reading 0.9856 Syllable Reading 0.9894 Letter Name Knowledge 0.9869 Mean Scores SI conducted statistical tests around the means from baseline and midline tools for each subtask and results are shown below. 103 Table 50: Testing for Differences in Base and Midline Mean Scores - with Zero Scores SUBTASK COEF. STD. ERR. T P>T [95% CONF. INTERVAL] Letter Name Knowledge -1.519899 2.127478 -0.71 0.475 -5.69891 2.659111 Syllable Reading -0.7774406 2.2874 -0.34 0.734 -5.270696 3.715814 Familiar Word Reading -1.068064 1.511593 -0.71 0.48 -4.037375 1.901246 Oral Reading Fluency -0.2970392 1.548465 -0.19 0.848 -3.338572 2.744494 Non-Word Reading -0.9720352 1.190897 -0.82 0.415 -3.311384 1.367314 Table 51: Testing for Differences in Base and Midline Mean Scores - Without Zero Scores SUBTASK COEF. STD. ERR. T P>T [95% CONF.INTERVAL] Letter Name Knowledge -0.72497 2.407576 -0.3 0.763 -5.45712 4.007174 Syllable Reading 4.643386 2.821976 1.65 0.101 -0.90678 10.19355 Familiar Word Reading -0.59048 1.941471 -0.3 0.761 -4.41061 3.229651 Oral Reading Fluency -0.39542 2.024785 -0.2 0.845 -4.37948 3.588641 Non-Word Reading -0.66651 1.758585 -0.38 0.705 -4.12782 2.794801 The results showed that the null hypotheses that means for each sub-task are not different between the midline and baseline versions could not be rejected. However, in such small sized pilots and also with many untimed tasks, these results do not necessarily indicate that that the null hypotheses could be accepted nor the alternative hypotheses of difference in tools can be accepted. Therefore, SI used a conservative approach that calibrated the tools for equivalence with means equating method and obtained the following conversion factors. These factors were then applied on midline learner scores for analysis. Results are shown below. Table 52. Mean Scores from Pilot – Both Standards without Zero Scores Sub Tasks Baseline Tool Midline Tool Mean Std Mean Std Letter Name Knowledge 28.557 22.608 28.141 25.511 Syllable Reading 31.307 25.968 36.219 24.714 Familiar Word Reading 24.215 16.977 23.714 15.763 Oral Reading Fluency 25.871 17.966 25.073 17.463 Non-Word Reading 17.561 12.423 16.791 11.041 Initial Sound Identification 3.363 2.476 3.448 2.422 Syllable Segmentation 7.798 2.455 7.575 2.371 Oral Reading Comprehension 1.929 1.105 1.793 1.015 Listening Comprehension 3.874 1.024 3.361 1.172 104 Table 53. Equating Conversion Factors Sub Tasks Conversion Factor Used on Midline Scores Letter Name Knowledge 1.015 Syllable Reading 0.864 Familiar Word Reading 1.021 Oral Reading Fluency 1.032 Non-Word Reading 1.046 Initial Sound Identification 0.975 Syllable Segmentation 1.029 Oral Reading Comprehension 1.076 Listening Comprehension 1.153 105 ANNEX 4: MIDLINE BENCHMARKS SUBTASK UNITS STANDARD 2 BENCHMARK STD 2 BENCHMARK SOURCE STANDARD 4 BENCHMARK STD 4 BENCHMARK SOURCE Letter name knowledge (correct letters/min) 24 Std 1 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) 50 Std 3 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) Syllable reading (correct syllables/min) 60 Std 2 Benchmark, from EGR workshop 65 Std 3 Benchmark, from EGR workshop Familiar word reading (correct words per min) 40 Std 2 Benchmark, from EGR workshop 45 Std 3 Benchmark, from EGR workshop Non-word reading (correct non￾words/min) 15 Std 1 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) 40 Std 3 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) Oral reading fluency (correct words/min of text) 40 Std 2 Benchmark, from EGR workshop 50 Std 3 Benchmark, from EGR workshop Reading comprehension (out of 5 questions) 80% Std 2 Benchmark, from EGR workshop 80% Std 3 Benchmark, from EGR workshop Listening comprehension (out of 5 questions) 60% Std 1 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) 80% Std 3 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) Syllable segmentation (out of 10 items) 70% Std 1 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) 80% Std 3 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) Initial sound reading (out of 10 items) 80% Std 1 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) 90% Std 3 Benchmark, EGRA Coordinating Committee (as used in Baseline IE Report) Source: EGRA Coordinating Committee; MoEST 106 ANNEX 5. EGRA PROGRAM EFFECTS ON ORAL READING FLUENCY: RESULTS OF DIFFERENCE IN DIFFERENCE ESTIMATES (ORDINARY LEAST SQUARES ESTIMATES) Standard 2 Learners: by Gender and Treatment Level 1 GIRLS BOYS OVERALL Constant -0.225 (1.113) 3.009 (3.184) 1.630 (2.557) Treatment level 1 2.172** (1.032) 0.101 (1.543) 0.880 (1.188) Mean 2.375 (4.265) -0.938 (7.084) 0.234 (5.724) Effect Size 0.557 -0.132 0.041 Dropout rate -10.824* (6.339) -7.935 (8.627) -10.083 (6.847) Enrollment average .002 (.006) -.018 (.014) -.009 (.012) Essential teaching practices -.023 (.208) -.474 (.396) -.266 (.305) SES -.743* (.388) .011 (1.003) -.111 (.770) Observations 67 70 71 R-squared 0.119 0.092 0.050 Treatment schools at Level 1 compared with control schools in Levels 1. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. Standard 4 Learners: by Gender and Treatment Level 1 GIRLS BOYS OVERALL Constant -12.182** (5.036) -10.579* (6.250) -10.778** (4.548) Treatment level 1 3.836 (2.934) -0.724 (3.060) 0.825 (2.129) Mean -3.073 (11.887) -5.914 (10.776) -4.719 (8.657) Effect Size -0.259 -0.549 -0.545 Dropout rate 61.562** (29.082) 21.468 (34.108) 37.267** (17.052) Enrollment average .009 (.018) .011 (.022) .0103 (.0167) Essential teaching practices -.067 (.973) -.045 (.806) -.0826 (.714) SES -.278 (2.088) -3.267* (1.678) -1.760 (1.371) Observations 66 70 71 R-squared 0.077 0.073 0.084 Treatment schools at Level 1 to control schools in Levels 1. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. 107 Standard 2 Learners: by Gender and Treatment Level 3 GIRLS BOYS OVERALL Constant 0.575 (3.790) 0.710 (2.506) 0.973 (2.631) Treatment level 3 0.819 (3.324) 0.259 (2.420) 0.557 (2.519) Mean -1.050 (11.358) -0.662 (6.535) -0.630 (8.138) Effect Size -0.092 -0.101 -0.077 Learner :Teacher Ratio -.025 (.026) -.017 (.015) -.023 (.019) Observations 58 64 64 R-squared 0.017 0.015 0.022 Treatment schools at Level 3 compared with control schools in Levels 3. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. Standard 4 Learners: by Gender and Treatment Level 3 GIRLS BOYS OVERALL Constant -1.586 (3.757) -0.512 (3.155) 0.181 (2.465) Treatment level 3 1.097 (2.891) -2.291 (2.492) -1.254 (2.087) Mean 0.005 (10.205) -4.201 (8.471) -1.950 (7.767) Effect Size 0.000 -0.496 -0.251 Learner :Teacher Ratio .005 (.023) -.015 (.022) -.010 (.015) Observations 62 64 64 R-squared 0.003 0.020 0.009 Treatment schools in Level 3 compared with control schools in Levels 3. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. Standard 2 Learners: by Gender and Treatment Level 4 GIRLS BOYS OVERALL Constant -0.308 (0.492) -3.006*** (0.600) -1.670*** (0.342) Treatment level 4 0.764 (1.333) 2.756** (1.161) 1.947* (0.961) Mean 0.456 (5.922) -0.250 (4.738) 0.277 (4.099) Effect Size 0.077 -0.053 0.068 Observations 81 81 82 R-squared 0.007 0.097 0.077 Treatment schools in Level 4 compared with control schools in Level 4. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. Standard 4 Learners: by Gender and Treatment Level 4 108 GIRLS BOYS OVERALL Constant -4.644** (1.921) -8.345*** (2.066) -6.687*** (1.474) Treatment level 4 1.876 (2.974) 0.148 (3.077) 1.390 (2.397) Mean -2.768 (11.085) -8.197 (11.631) -5.297 (9.128) Effect Size -0.250 -0.705 -0.580 Observations 82 83 83 R-squared 0.008 0.000 0.006 Treatment schools in Level 4 compared to control schools in Level 4. Standard errors in parentheses under coefficients; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable is change in school averages. Standard deviations presented under the means in parentheses. 109 ANNEX 6: DROPOUT AT BASE AND MIDLINE, BY TREATMENT STATUS, STANDARD AND GENDER AVERAGE DROPOUT – COMPARISON GROUP SE AVERAGE DROPOUT - TREATMENT GROUP SE AVERAGE DROPOUTS￾ALL SE Baseline Standard 2 9.1 0.2 6.9 0.1 7.9 0.1 Girls 9.2 0.2 6.3 0.2 7.6 0.1 Boys 9.0 0.2 7.4 0.2 8.1 0.1 Standard 4 7.1 0.1 4.3 0.1 5.5 0.1 Girls 7.1 0.2 4.2 0.1 5.6 0.1 Boys 7.0 0.2 4.3 0.1 5.5 0.1 All (Std. 2+4) 9.8 0.2 6.8 0.1 8.2 0.1 Girls 9.9 0.2 6.3 0.1 8.0 0.1 Boys 9.7 0.2 7.2 0.2 8.3 0.1 Midline Standard 2 8.5 0.2 7.4 0.1 7.9 0.1 Girls 8.5 0.2 7.4 0.2 7.9 0.1 Boys 8.4 0.2 7.4 0.2 7.8 0.1 Standard 4 5.4 0.1 6.0 0.1 5.7 0.1 Girls 5.4 0.1 6.0 0.1 5.7 0.1 Boys 5.3 0.1 6.0 0.1 5.7 0.1 All (Std. 2+4) 7.8 0.1 7.6 0.1 7.7 0.1 Girls 7.8 0.2 7.6 0.2 7.7 0.1 Boys 7.7 0.2 7.6 0.2 7.6 0.1 Baseline School Questionnaire 2013 and Midline School Questionnaire 2015. 110 ANNEX 7: REGRESSION ANALYSIS TABLES MODEL 2 - Compares all of the treatment groups to the pure control from Levels 3 and 4 Standard errors in parentheses; *** ** and * represent statistical significance at 1%, 5% and 10% levels, respectively. Dependent variable in Approach 1 was change in school averages while it was individual learner scores in Approach 2. STANDARD 2 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant -.508 (1.338) -1.317 (2.025) -0.892 (1.653) Treatment level 1 3.084*** (1.112) 2.307 (1.602) 2.323* (1.287) Mean 2.375 (4.265) -0.938 (7.084) 0.234 (5.724) Effect Size 0.557 -0.132 0.041 Treatment level 2 -2.992* (1.567) 0.848 (1.589) -0.465 (1.171) Mean -3.618 (6.440) -2.384 (6.589) -2.479 (4.841) Effect Size -0.562 -0.362 -0.512 Treatment level 3 -.0349 (2.294) 1.937 (1.507) 1.190 (1.697) Mean -1.050 (11.358) -0.662 (6.535) -0.630 (8.138) Effect Size -0.092 -0.101 -0.077 Treatment level 4 1.534 (1.644) 2.936** (1.386) 2.467** (1.206) Mean 0.456 (5.922) -0.250 (4.738) 0.277 (4.099) Effect Size 0.077 -0.053 0.068 Control level 1 1.01535 (.9463478) 1.957* (1.104) 1.267 (0.903) Mean 0.321 (3.121) -1.107 (3.874) -0.609 (3.180) Effect Size 0.103 -0.286 -0.192 Dropout rate -8.812 (6.160) .116 (4.413) -3.990 (4.504) Enrollment average .001 (.006) -.009 (.008) -.005 (.007) PTA meets monthly -.004 (.855) .486 (1.119) .418 (.939) Observations 214 230 232 R-squared 0.088 0.043 0.046 111 STANDARD 4 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant -3.837 (2.491 -5.487* (3.027) -4.497 (2.166) Treatment level 1 0.594 (2.823) 1.014 (2.605) 0.526 (1.925) Mean -3.073 (11.887) -5.914 (10.776) -4.719 (8.657) Effect Size -0.259 -0.549 -0.545 Treatment level 2 1.301 (3.306) -2.268 (2.441) -0.870 (2.087) Mean -2.395 (13.677) -9.102 (8.142) -5.877 (8.867) Effect Size -0.175 -1.118 -0.663 Treatment level 3 2.713 (2.379) 1.749 (2.323) 2.263 (1.888) Mean 0.005 (10.205) -4.201 (8.471) -1.950 (7.767) Effect Size 0.000 -0.496 -0.251 Treatment level 4 2.618 (2.961) -1.361 (3.033) 0.735 (2.467) Mean -2.768 (11.085) -8.197 (11.631) -5.297 (9.128) Effect Size -0.250 -0.705 -0.580 Control level 1 -3.494 (2.738) 0.054 (2.642) -1.434 (1.967) Mean -6.340 (11.465) -6.217 (9.860) -5.848 (8.004) Effect Size -0.553 -0.631 -0.731 Dropout rate 27.445** (11.638) 21.412 (9.223) Enrollment average -.003 (.012) -.004 (.010) PTA meets monthly -1.097 (1.991) -1.682 (1.466) Observations 222 238 R-squared 0.053 0.050 112 MODEL 3 - Compares Treatment for Level 1 with Control for Level 1 STANDARD 2 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant -0.225 (1.113) 3.009 (3.184) 1.630 (2.557) Treatment level 1 2.172** (1.032) 0.101 (1.543) 0.880 (1.188) Mean 2.375 (4.265) -0.938 (7.084) 0.234 (5.724) Effect Size 0.557 -0.132 0.041 Dropout rate -10.824* (6.339) -7.935 (8.627) -10.083 (6.847) Enrollment average .002 (.006) -.018 (.014) -.009 (.012) Essential teaching practices -.023 (.208) -.474 (.396) -.266 (.305) SES -.743* (.388) .011 (1.003) -.111 (.770) Observations 67 70 71 R-squared 0.119 0.092 0.050 STANDARD 4 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant -12.182** (5.036) -10.579* (6.250) -10.778** (4.548) Treatment level 1 3.836 (2.934) -0.724 (3.060) 0.825 (2.129) Mean -3.073 (11.887) -5.914 (10.776) -4.719 (8.657) Effect Size -0.259 -0.549 -0.545 Dropout rate 61.562** (29.082) 21.468 (34.108) 37.267** (17.052) Enrollment average .009 (.018) .011 (.022) .0103 (.0167) Essential teaching practices -.067 (.973) -.045 (.806) -.0826 (.714) SES -.278 (2.088) -3.267* (1.678) -1.760 (1.371) Observations 66 70 71 R-squared 0.077 0.073 0.084 113 MODEL 4 - Compares Treatment for Level 3 with Control for Level 3 STANDARD 2 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant 0.575 (3.790) 0.710 (2.506) 0.973 (2.631) Treatment level 3 0.819 (3.324) 0.259 (2.420) 0.557 (2.519) Mean -1.050 (11.358) -0.662 (6.535) -0.630 (8.138) Effect Size -0.092 -0.101 -0.077 Learner: Teacher Ratio -.025 (.026) -.017 (.015) -.023 (.019) Observations 58 64 64 R-squared 0.017 0.015 0.022 STANDARD 4 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant -1.586 (3.757) -0.512 (3.155) 0.181 (2.465) Treatment level 3 1.097 (2.891) -2.291 (2.492) -1.254 (2.087) Mean 0.005 (10.205) -4.201 (8.471) -1.950 (7.767) Effect Size 0.000 -0.496 -0.251 Learner: Teacher Ratio .005 (.023) -.015 (.022) -.010 (.015) Observations 62 64 64 R-squared 0.003 0.020 0.009 114 MODEL 5 - Compares Treatment for Level 4 with Control for Level 4 STANDARD 2 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant -0.308 (0.492) -3.006*** (0.600) -1.670*** (0.342) Treatment level 4 0.764 (1.333) 2.756** (1.161) 1.947* (0.961) Mean 0.456 (5.922) -0.250 (4.738) 0.277 (4.099) Effect Size 0.077 -0.053 0.068 Observations 81 81 82 R-squared 0.007 0.097 0.077 STANDARD 4 Girls Boys Overall Approach 1 Approach 1 Approach 1 Constant -4.644** (1.921) -8.345*** (2.066) -6.687*** (1.474) Treatment level 4 1.876 (2.974) 0.148 (3.077) 1.390 (2.397) Mean -2.768 (11.085) -8.197 (11.631) -5.297 (9.128) Effect Size -0.250 -0.705 -0.580 Observations 82 83 83 R-squared 0.008 0.000 0.006 115 ANNEX 8: BALANCE TABLES FOR BASELINE Treatment Level 1 and Control Level 1: EGRA + INVC + SSDI T-Tests School-Level Variable Treatment Level 1: Mean Control Level 1: Mean P-Value Number of Classrooms 6.42 5.84 0.42 Dropout Rate 0.04 0.08 0.01 Enrollment (average per standard) 217.64 181.79 0.10 Learner-Teacher Ratio 105.18 102.29 0.84 Essential Teaching -0.80 0.30 0.05 SES -0.62 -0.24 0.03 School Resources -0.08 -0.27 0.54 School Day Length 18.28 18.07 0.67 MTPDS Support 0.68 0.72 0.63 Chi-squared School-Level Variable Treatment Level 1: % Yes Control Level 1: % Yes P-Value Latrines For Girls 81.6% 89.2% 0.352 PTA Meets Monthly 42.9% 26.3% 0.195 Feeding Programs 21.6% 11.1% 0.226 Head Teacher: Female 8.1% 18.9% 0.174 Treatment Level 3 and Control Level 3: EGRA + INVC T-Tests School-Level Variable Treatment Level 3: Mean Control Level 3: Mean P-Value Number of Classrooms 5.57 5.92 0.55 Dropout Rate 0.04 0.06 0.37 Enrollment (average per standard) 145.64 125.52 0.33 LT Ratio 87.74 84.25 0.07 Essential Teaching 1.22 -0.12 0.94 SES 0.17 0.09 0.56 School Resources 0.19 0.34 0.46 School Day Length 20.89 19.96 0.15 MTPDS 0.93 0.92 0.76 116 Chi-squared School-Level Variable Treatment Level 3: % Yes Control Level 3: % Yes P-Value Latrines For Girls 89.2% 91.7% 0.751 PTA Meets Monthly 45.0% 34.6% 0.402 Feeding Programs 10.8% 0.0% 0.103 Head Teacher: Female 8.0% 8.1% 0.988 Treatment Level 4 and Control Level 4: EGRA only T-Tests School-Level Variable Treatment Level 4: Mean Control Level 4: Mean P-Value Number of Classrooms 6.03 5.30 0.26 Dropout Rate 0.04 0.06 0.29 Enrollment (average per standard) 145.64 125.52 0.25 LT Ratio 87.74 84.25 0.64 Essential Teaching 1.22 -0.12 0.25 SES 0.17 0.09 0.75 School Resources 0.19 0.34 0.66 School Day Length 20.89 19.96 0.23 MTPDS 0.93 0.92 0.89 Chi-squared School-Level Variable Treatment Level 4: % Yes Control Level 4: % Yes P-Value Latrines For Girls 91.4% 91.1% 0.96 PTA Meets Monthly 41.7% 27.7% 0.181 Feeding Programs 22.9% 17.8% 0.573 Head Teacher: Female 10.6% 11.4% 0.91 117 All Treatment Levels and Control Levels 3 & 4: EGRA + INVC + SSDI; EGRA + SSDI; EGRA + INVC; EGRA only T-Tests School-Level Variable All Treatment Levels: Mean Control Levels 3 & 4: Mean P-Value Number of Classrooms 5.94 5.51 0.29 Dropout Rate 0.05 0.08 0.08 Enrollment (average per standard) 169.14 135.04 0.01 LT Ratio 92.61 87.73 0.37 Essential Teaching 0.10 0.42 0.67 SES -0.12 0.06 0.23 School Resources 0.00 0.16 0.41 School Day Length 20.16 20.79 0.15 MTPDS 0.85 0.94 0.05 Chi-squared School-Level Variable All Treatment Levels: & Yes Control Levels 3 & 4: % Yes P-Value Latrines For Girls 87.1% 91.3% 0.375 PTA Meets Monthly 42.5% 30.1% 0.074 Feeding Programs 18.0% 11.8% 0.251 Head Teacher: Female 9.7% 12.2% 0.586 118 ANNEX 9: LITERATURE REVIEW With the explicit Millennium Development Goal (MDG) focus on improving the quality of primary education, interventions focused on improving early grade education outcomes are now increasing across the globe. Evaluations of many of these interventions are being carried out to learn the most effective, and cost-effective, interventions. This section summarizes the latest literature on effectiveness and costs of early grade education interventions, especially in Africa. EFFECTIVENESS OF INTERVENTIONS FOCUSED ON IMPROVING EARLY GRADE EDUCATION OUTCOMES There is a wide body of individual program-based evaluations and research on the effectiveness of early grade education programs (see the EdData website, for example). Also, systematic reviews that distill results from rigorous and reliable studies are now available, to obtain information that compares various types of interventions based on results. The report discuss findings from some of the latest systematic reviews and selected evaluations of individual EGRA interventions below. Petrosino et al. (2012),30 in a systematic review, showed that programs focused on educational practices produced the least effects, similar to provision of information and training to community and parents, while interventions focused on new schools / infrastructure development showed the highest effect, followed by health and nutrition and economic strengthening programs such as conditional cash transfers, incentives, etc., and school nutrition programs (Table 52)The improving educational practices interventions were primarily carried out at school or district levels by providing services or materials to learners such as remedial education, computers, flip charts, textbooks, and English language training for teachers, technology equipment and software, extra teachers (and reducing class size), providing incentives to teachers, monitoring teacher attendance, teacher training, empowering and funding parent-school associations, and more comprehensive school reform and improvement efforts targeted at primary school-aged children. The review also showed that average effect size was positive for outcomes such as enrollment, attendance, progression in grades, and learning scores in math and languages Table 52: Average Effect Size across Types of Interventions (by order of effect size magnitude) OUTCOME STANDARDIZED MEAN EFFECT (D) Broad Intervention Type New Schools/Infrastructure 0.44 Health Care/Nutrition 0.23 Economic 0.16 Educational Practice/Programs 0.06 Providing Information/Training 0.06 Source: Petrosino et al. (2012) Petrosino et al. (2012) also examined the various outcomes gathered from the 59 studies of the 73 that focused on improving educational practices, showing the following: Effect size was the largest for enrollment and language scores with 0.18 standard deviations for both (95 percent CI of [.13–.24] for enrollment and [.12–.25] for language), followed by math at 0.16 (95 percent CI [.10–.23]), attendance at 0.15 (95 percent CI [.10–.20]), and progression at 0.13 (95 percent CI [.08–.18]) outcomes. The effects represented three to nine percent increases in positive outcomes compared to the comparison group in the studies (see Table 53). 30 They conducted a systematic review of 73 evaluations that focused on improving primary and secondary school enrollment using randomized control trial and quasi-experimental designs. Studies were conducted in 27 different countries, with the largest numbers in Kenya (N=12), India (N=9), Bangladesh (N=6), Colombia (N=5), and Jamaica (N=5). The studies focused on various types of interventions that are broadly classified under education practices improvement, infrastructure development, health care, economic aspects, and information dissemination. In general, effects were the lowest for education practices related interventions in comparison with other types of programs. 119 Table 53: Summary of Average Effect Sizes for Overall Intervention Effects (by order of effect size magnitude) OUTCOME AVERAGE STANDARDIZED MEAN EFFECT (IN STANDARD DEVIATION) (NUMBER OF STUDIES IN PARENTHESES) PERCENTAGE IMPROVEMENT IN TREATMENT OVER COMPARISON Main Outcomes Enrollment 0.18 (34) 9% Attendance 0.15 (33) 8% Progression 0.13 (15) 7% Dropout 0.05 (18) 3% Supplemental Outcomes Language 0.18 (25) 9% Math 0.16 (25) 8% Standardized Assessment Scores 0.06 (10) 3% Other Achievement 0.05 (5) 3% Source: Petrosino et al. (2012) In contrast, another systematic review conducted by McEwan (2014) using meta-analysis of 77 randomized control experiments conducted in primary schools in developing countries in Asia and Africa found that, on average, the largest mean effect sizes included treatments with computers or instructional technology (0.15); teacher training (0.12); smaller classes, smaller learning groups within classes, or ability grouping (0.12); contract or volunteer teachers (0.10); learner and teacher performance incentives (0.09); and instructional materials (0.08). The monetary grants and deworming treatments had mean effect sizes that were close to zero and not statistically significant. Nutritional treatments, treatments that disseminated information, and treatments that improved school management or supervision, had small mean effect sizes (0.04–0.06) that were not always robust to control for study moderators. Data were insufficient to judge the relative cost-effectiveness of categories of interventions. Similar to findings by McEwan (2014), another systematic review using meta-analysis conducted by Conn (2014) to identify effective educational interventions in Sub-Saharan African with an impact on learner learning by comparing twelve different types of education interventions or programs (56 articles containing 66 separate experiments, 83 treatment arms, and 420 effect size estimates were used) found that interventions targeting pedagogical methods (changes in instructional techniques) had higher pooled effect size on learning achievement outcomes than all other eleven intervention types (e.g., school management programs, school supplies interventions, or interventions that change the class size or composition). The pooled effect size associated with the pedagogical interventions was 0.918 standard deviations (SE = 0.314, p = 0.01), 0.566 in the sample excluding outliers and including only randomized controlled trials (SE = 0.194, p = 0.01), and 0.228 in a sample that includes only the highest quality studies (SE = 0.078, p = 0.03). The findings were robust to a number of moderating factors. Using meta-regression, she found that interventions in pedagogical methods had an effect size over 0.30 standard deviations (significant at the 5 percent level), greater than all other intervention areas combined, even after controlling for multiple study-level and intervention￾level variables. Also, studies that employed adaptive instruction and teacher coaching techniques were particularly effective. Interventions that provided health treatments or school meals had on average the lowest pooled effect size. In an IE of final year EGRA intervention in Liberia, Piper et al (2011) showed that EGRA full treatment had moderate impacts on listening comprehension, large impacts on phonemic awareness, letter fluency, familiar word fluency, oral reading fluency, and reading comprehension, and found very large impacts for unfamiliar word fluency, indicating that the EGRA Plus program had particularly large impacts on improving children’s ability to manipulate sounds to make words. Specifically, the effect sizes are shown in Table 54 below. Table 54: Effect Size for EGRA Intervention by Subtasks (full treatment) in Liberia (in Standard Deviations) SUBTASKS EGRA FULL TREATMENT Letter Naming Fluency 0.52*** Phonemic Awareness 0.55*** Familiar Word Fluency 0.78** Unfamiliar Word Fluency 1.23*** Oral Reading Fluency 0.80** Reading Comprehension 0.82** Listening Comprehension 0.39 ***,**, and * represent significance at 1%, 5% and 10% levels, respectively. Source: Piper et al. (2011) 120 Similarly, a midterm EGRA evaluation in Mali conducted in year three of a five-year IE by Gove and Wetterberg (2011; pp. 97-98) showed that scores on every subtask outcome in both grade 1 and 2 were higher in treatment schools relative to comparison schools, and the differences were robust, consistent across genders, and statistically significant. In grade 1, the estimate of the treatment effect overall was 0.81 standard deviations while it was 0.27 for grade 2. For oral reading fluency in correct words per minute, effect size was 0.92 and 0.12 for grades 1 and 2, respectively. A recent report (by IBTCI, 2015) based on midline evaluation of an EGRA project in Mozambique has been released. The evaluation used a randomized control design to collect data on reading skills of approximately 3,600 second and third graders at baseline at beginning of school year in 2013 and compared it with two midlines, one at near the end of the 2013 school year and another at end of the 2014 school year. Midline results showed that after a full school year of project implementation with full EGRA treatment, learners performed at significantly higher levels than their counterparts in comparison schools on all EGRA subtests. The project was found to have strengthened reading instruction in the intervention schools on all EGRA measures, as evidenced by learner reading outcomes and the observation of teacher instructional behavior. Looking across EGRA subtasks, the authors found that the intervention groups showed the greatest improvements in letter knowledge (identifying and sounding out letters), familiar word reading, and reading connected text (fluency) relative to comparison schools. COST EFFECTIVENESS OF EGRA INTERVENTIONS There were few studies that examined the cost effectiveness of early grade education interventions in developing countries due in part to limitations in data availability on costs with which to conduct the analysis. Evaluators found three studies that provided some insights on EGRA-type interventions. Healey (2014) examined the costs of four EGRA efforts — pilots that took place in Egypt, Kenya, Liberia, and Malawi. Seven phases of an EGRA program were identified: EGRA development, implementation of EGRA to generate data needed to conduct a policy dialogue around the need for an EGRA intervention, policy dialogue around the results, implementation of an EGRA to generate a baseline in some pilot region, development of an EGRA, implementation of the EGRA intervention, and implementation of an EGRA to generate an endline. Four cost domains were examined: technical labor, technical materials, technical other, and administration. The resulting data were used to cost all four programs and were combined with net gains in learner scores from each EGRA program by subtask (impact of EGRA programs on the treatment learners compared to comparison learners) to calculate the cost effectiveness of each program. The review of cost effectiveness in terms of technical costs per pupil to implement EGRA programs in the four countries showed that it costs the lowest per pupil in Malawi for both syllable reading and oral reading fluency subtasks than in the other three countries (see Table 55). Table 55: Egypt, Kenya, Liberia, and Malawi: Cost Effectiveness of EGRA (technical Cost per Pupil in US dollars) for Implementing EGRA EGYPT – ENGLISH KENYA – ENGLISH KENYA – SWAHILI LIBERIA – ENGLISH MALAWI – ENGLISH Syllable Reading Cost per Pupil ($) 2.63 0.45 0.45 74.57 0.13 Net Gain in Mean Score NA 17 16.8 NA NA Oral Reading Fluency Cost per Pupil ($) 5.62 0.99 1.70 40.39 0.25 Net Gain in Mean Score 8 7.7 4.5 9.6 8 Cost effectiveness (C𝐸𝐸) = Technical Costs / Number of learners in treatment group ∗ 𝑁𝑁𝑒𝑒𝑡𝑡 𝐺𝐺𝑎𝑎 𝑛𝑛 in scores. Source: Healey (2014). Spratt et al. (2013) estimated the cost effectiveness of an EGRA program in Mali where they combined cost estimations with effect sizes calculated on the basis of IE results at Year 1 and at Year 3 of program implementation (see Table 56). For grade 1, the more mature program, at Year 3, it was only marginally more cost effective than it was in its first year of implementation, to produce a 0.2 standard deviation of gain in reading skills at a cost per pupil of $12.78 relative to $13.35. For the grade 2 learners, achieving 0.2 standard deviation of gain for more mature program was estimated to cost $15.73, more than what it costs for the same level of gain among grade 1 learners. Results at Year 3 for grade 3 learners (one year after completing the two-year program) suggested that the cost of provision to produce more lasting gains could be considerably higher than the cost of producing immediate gains. The results also suggested that stopping an intervention prematurely, before a minimum level of performance was achieved, could prove costly. 121 Table 56: Mali: Estimation of Program Cost Effectiveness at Grade 1 by Maturity of Treatment, on Selected EGRA Measures EGRA COMPONENT AVERAGE EXPERIMENT AL GROUP SCORE EXPERIMENTA L EFFECT SIZE PER￾STUDENT COST COST OF 0.2 SD IMPROVEMENT AVERAGE COST OF 0.2 SD IMPROVEME NT Grade 1 results at end Year 1 at a “less mature” state of implementation Letter Knowledge 10.6 lpm (a) 0.763 $21.00 $5.51 $13.35 Familiar Word Reading 1.6 wpm (a) 0.502 $8.36 Decoding of New Words 0.7 wpm 0.173 $24.29 Oral Reading Fluency 0.8 wpm 0.275 $15.25 Grade 1 results at end Year 3 at a “more mature” state of implementation Letter Knowledge 7.2 lpm 0.594 $17.66 $5.95 $12.78 Familiar Word Reading 1.0 wpm 0.207 $17.07 Decoding of New Words 1.2 wpm 0.274 $12.89 Oral Reading Fluency 3.8 wpm 0.232 $15.23 Grade 2 results at end Year 3, at a “more mature” state of implementation Letter Knowledge 16.5 lpm 0.664 $43.22 $13.02 $15.73 Familiar Word Reading 5.4 wpm 0.672 $12.86 Decoding of New Words 4.4 wpm 0.524 $16.50 Oral Reading Fluency 7.8 wpm 0.421 $20.53 Source: Year 1 and Year 3 average scores and effect sizes were estimated on the basis of IE data. (a) “lpm” refers to “letters read correctly in one minute”; “wpm” to “words read correctly in one minute.” Source: Spratt et al (2013) An IBTCI (2015) midline evaluation of an EGRA intervention in Mozambique found the difference in net gains between learners in treatment and their counterparts in comparison schools to be significant for familiar word recognition and reading fluency. Relative to comparison school performance, learners in Grade 2 improved performance on familiar word recognition by 14.0 percent and text reading fluency by 15.5 percent for every US dollar spent. 122 ANNEX 10: EARLY GRADE READING ASSESSMENT – BASELINE Malawi Early Grade Reading Assessment: Student Response Form Administrator Instructions and Protocol, May – June 2013 Chichewa Malangizo: Muyenera kukhazikitsa ubwenzi wabwino ndi wophunzira amene mukumuyesa kudzera mu nkhani zifupizifupi komanso zosangalatsa kuti aone mafunsowa ngati sewero chabe osati ntchito yovuta. Nkoyenera kuwerenga zigawo zokhazo zomwe zili mumabokosi mokweza, momveka bwino ndi modekha. Uli bwanji? Dzina langa ndi_________ndipo ndimakhala ku ________. (Chezani ndi wophunzira munjira yomwe ingathandize kuti amasuke). Kupempha chilolezo • Ndikuuze chifukwa chimene ndabwerera kuno. Ndimagwira ntchito ku Unduna wa za Maphunziro, za Sayansi ndi Luso. Ndikufuna kudziwa m’mene inu ophunzira mumaphunzirira kuwerenga. Mwa mwayi iwe wasankhidwa kuti ndicheze nawe. • Tichita sewero lowerenga. Ndikufunsa kuti undiwerengere malembo, mawu ndi nkhani mokweza. • Ndigwiritsa ntchito wotchi iyi kuti ndiwone nthawi yomwe utenge powerenga. • Awa simayeso, ndipo sizikhudzana ndi zotsatira za maphunziro ako. • Ndikufunsanso mafunso ena okhudzana ndi banja la kwanu monga, chiyankhulo chomwe mumayankhula kunyumba kwanu ndi zinthu zina zomwe muli nazo kwanu. • Palibe amene adziwe zimene tikambirane. • Uli ndi ufulu woyankha mafunso kapena ayi. Ngakhale tili mkati mwa kucheza uli ndi ufulu kukana kuyankha mafunso. • Ngati sukufuna kuti ndicheze nawe utha kubwerera m’kalasi. • Uli ndi funso tisanayambe? Tikhoza kuyamba? Chongani mukabokosika ngati ophunzira wavomereza kuyesedwa: INDE (Ngati wophunzira sanavomereze kuyesedwa, muthokozeni ndi kuitana ophunzira wina pogwiritsa ntchito chipepala chomwechi.) A. Tsiku la Mayeso Tsiku : _______ H. Kalasi ○ 1 = Sitandade 1 ○ 2 = Sitandade 2 ○ 3 = Sitandade 3 ○ 4 = Sitandade 4 Mwezi :_______ B. Dzina la Woyesa C. Dzina la Sukulu I. Dzina la Mphunzitsi D. Dera J. Sitilimu E. Boma K. Dzina la ophunzira L. Nambala yachinsinsi ya ophunzira F.Chigawo M. Zaka zakubadwa 123 G. Mtundu wa Sukulu : ○ 1 = Tsiku lonse ○ 2 = M’mawa ○ 3 = Masana N. Mwamuna kapena Mkazi O. Dzina la mudzi P. Dzina la mkulu wa pakhomo ○ 1 = Mwamuna ○ 2= Mkazi N. Nthawi Yoyambira 5710 ___ : ___ Gawo 1. Kudziwa Dzina la Lembo Onetsani ophunzira pepala la malembo mu buku la ophunzira.Nenani: Ili ndi tsamba la malembo a m’Chichewa. Ndiuze maina a malembo amene ungathe. Mwachitsanzo, dzina la lembo ili [lozani lembo la ‘S’] ndi ‘S’. Tiye tiyesere: Ndiuze dzina la lembo ili [lozani lembo la ‘U’] Ngati ophunzira ayankhe bwino nenani: Wakhoza dzina la lembo ili ndi ‘U’: Ngati ophunzira alephere kuyankha molondola, nenani: Dzina la lembo ili ndi ‘U’ Tsopano yesera lembo lina: Ndiuze dzina la lembo ili [lozani lembo la P]: Ngati mwana wayankha molondola, nenani: Wakhoza, dzina la lembo ili ndi ‘P’ Ngati mwana walephera kuyankha molondola, nenani: dzina la lembo ili ndi ‘P’ Kodi ukudziwa chomwe ukuyenera kuchita? Ndikanena kuti “Yamba” Chonde tchula dzina la lembo lili lonse mofulumira ndi mosamala. Yamba pano ndipo ndi kupitiriza motere [Lozani lembo loyamba mu mndandanda woyamba pamathero a chitsanzo ndipo lozetsani chala pa mzere woyamba. Ngati wafika pa lembo lomwe sukulidziwa, ndikuuza dzina lake.Ndikakuwuza udzipitiriza. Wakonzeka? Yamba tsopano. Yambani kuwerengera nthawi pamene ophunzira wawerenga lembo loyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pensulo polemba chizindikiro ichi ( / ). Werengerani lembo limene walikonza yekha ngati lolondola. Ngati mwachonga kale mayankho odzikonza yekha ngati olakwa, zunguzani mzere pa lembolo ndi kupitirira. Khalani chete pokhapokha akamapereka mayankho motere: ngati ophunzira adodoma kuyankha pa masekandi atatu, Perekani dzina la lembo, lozani lembo lotsatira ndi kunena, Pitiriza. Chongani lembo lomwe mwapereka kwa mwana. Ngati ophunzira apereke liwu la lembo osati dzina lalembo, mpatseni dzina lalembolo ndi kunena: Tandiuze dzina lalembo ili. Izi ziyenera kuchitika kamodzi kokha. PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI (60) nenani “lekeza pomwepo.” Lozerani lembo lomalizira kuwerenga ndi chizindikiro ichi (]). Lamulo loyamba: Ngati ophunzira alephere kupereka yankho lolondola limodzi mu mzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani mu kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. Chitsanzo : S u P 1 2 3 4 5 6 7 8 9 10 D i t i O T g C T m (10) H t O A r C n e h R (20) L e H p e A i o z U (30) h f i N T o o F d E (40) 124 e r P H r d T K t a (50) y w e L e E U N o d (60) W e A A S E n i m R (70) s t C V S N D t i L (80) A s J G e E i A C n (90) N a H S t U B y S o (100) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi) : Chongani m’kabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mzere oyamba. 125 Gawo 2. Maphatikizo a Malembo Ntchito iyi ndiyongomvera chabe. Ndikuuza mawu ndipo undiuze maphatikizo omwe ali m’mawuwo. Mwachitsanzo, m’mawu oti “nguluwe” muli maphatikizo awa: “ngu-lu-we”. Mu ntchito imeneyi ndikufuna kuti undiuze maphatikizo amene uwamve m’mawu. Nditchula mawuwa kawiri. Umvetsere kenako undiuze maphatikizo omwe ali m’mawuwo. Tiye tiyesere. Kodi maphatikizo omwe ali m’mawu oti “mayi”, “mayi” ndi chiyani? [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, maphatikizo a mawu oti “mayi” ndi “ma – yi”. Ngati mwana walephera kuyankha molondola, nenani: Mveranso kachiwiri: “mayi”. Maphatikizo omwe ali m’mawu oti “mayi” ndi “ma-yi.” Tsopano yesera ena: Kodi maphatikizo omwe ali m’mawu oti “khwanya”, “khwanya” ndi chiyani?. [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, maphatikizo a mawu oti “khwanya” ndi “khwa - nya ”. Ngati mwana walephera kuyankha molondola, nenani: Mveranso kachiwiri: “khwanya”. Maphatikizo omwe ali m’mawu oti “khwanya” ndi “khwa” ndi “nya.” Kodi ukudziwa chomwe uyenera kuchita? [Ngati ophunzira anene kuti ayi, muuzeni kuti]: Yesetsa m’mene ungathere. Werengani ndi kutchula mawu oyenera kachiwiri. Lolani yankho lokhalo lili ndi liwu lolondola. Ngati ophunzira akanike kuyankha m’masekondi atatu, chongani “Palibe yankho” ndipo pitirizani kutchula mawu otsatira. Tchulani momveka bwino koma musatsindike kwambiri paphatikizo loyamba la mawu ena aliwonse. Langizo loyamba : Ngati ophunzira alephere kuyankha molondola kapena kulephera kuwerenga mawu asanu oyambirira, nenani kuti “Zikomo”, ndipo musapitirize ntchiyoyi ndipo mukatero chongani m’kabokosi kali pamapeto a tsamba lino ndi kuyamba ntchito yotsatirayo. Kodi ndi maphatikizo ati omwe ali mmawu awa? [Bwerezani mawuwo kawiri] Ana A – na o wakhoz a o walakwa o sakudziwa o palibe yankho Boola Bo-o – la o wakhoz a o walakwa o sakudziwa o palibe yankho Mwamuna Mwa – mu na o wakhoz a o walakwa o sakudziwa o palibe yankho Bola Bo – la o wakhoz a o walakwa o sakudziwa o palibe yankho Mkaka Mka – ka o wakhoz a o walakwa o sakudziwa o palibe yankho Nama Na – ma o wakhoz a o walakwa o sakudziwa o palibe yankho Kakamiza Ka – ka – mi – za o wakhoz a o walakwa o sakudziwa o palibe yankho Mbola Mbo – la o wakhoz a o walakwa o sakudziwa o palibe yankho Mnkhwani Mnkhwa–ni o wakhoz a o walakwa o sakudziwa o palibe yankho 126 Kankha Ka – nkha o wakhoz a o walakwa o sakudziwa o palibe yankho Gawo 3. Kutchula liwu loyamba Ntchito iyi siyofunika kuwerengera nthawi ndipo PALIBE TSAMBA LA WOPHUNZIRA. Werengani mawu aliwonse kawiri ndipo mufunse ophunzira kuti atchule liwu loyamba m’mawu amenewa. kumbukirani kutchula maliwu moyenera : /p/ osati /pu/ monga: /p/, ----- “puh” kapena “pe.” Nenani: Ntchito iyi ndiyomvera chabe. Ndikufuna kuti undiuze liwu loyamba m’mawu ena aliwonse. Mwachitsanzo, m’mawu oti ‘galu’, liwu loyamba ndi “/g/”. Mu ntchito imeneyi, ndifuna undiuze liwu loyamba limene ukulimva m’mawu ena aliwonse. Nditchula mawuwo kawiri. Umvetsere mawuwo, kenaka undiuze liwu loyamba lomwe likumveka m’mawuwo. Tiye tiyesere. Kodi liwu loyamba m’mawu oti “mayi”, “mayi” ndi chiyani? [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, liwu loyamba m’mawu oti “mayi” ndi /mmmmm/ [Ngati ophunzira sanayankhe molondola, nenani]: mvetsera kawiri: “mmmayi”. Liwu loyamba m’mawu oti “mayi” ndi /mmmmm/. Tsopano yesera mawu ena: Kodi ndi liwu loyamba m’mawu oti “nzimbe”, “nzimbe” ndi chiyani? Ngati mwana wayankha molondola, nenani: Wakhoza, liwu loyamba m’mawu oti “nzimbe”ndi “/n/” Ngati mwana walephera kuyankha molondola, nenani: mveranso kaciwiri: liwu loyamba la m’mawu oti “nzimbe” ndi /n/ Kodi ukudziwa chomwe uyenera kuchita? [Ngati wophunzira anene kuti ayi, muzeni kuti]: Yesetsa m’mene ungathere. Werengani ndi kutchula mawu oyenera kawiri. Lolani yankho lokhalo lili ndi liwu lolondola. Ngati ophunizra akanike kuyankha m’masekondi atatu, chongani “Palibe yankho” ndipo pitirizani kutchula mawu otsatira. Tchulani momveka bwino koma musatsindike kwambiri liwu loyamba la mawu ena ali wonse. Langizo loyamba: Ngati ophunzira alephere kuyankha molondola kapena kulephera kuwerenga mawu asanu oyambirira, nenani kuti “Zikomo”, ndipo musapitirize ntchiyoyi ndipo mukatero chongani m’kabokosi kali pamapeto a tsamba lino ndi kuyamba ntchito yotsatirayo. Tchula liwu loyamba m’mawu awa ndi chiyani [Tchulani mawuwo] Kala /k/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Dona /d/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Khala /kh/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Atate /a/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Bala /b/ o wakhoza o walakwa o sakudziwa ○ palibe yankho 127 Mana /mmm/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Gada /g/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Wada /www/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Nola /n/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Gwada /g/ o wakhoza o walakwa o sakudziwa ○ palibe yankho Gawo 4. Kuwerenga Maphatikizo Onetsani wophunzira pepala la maphatikizo kuchokera m’buku la ophunzira.Nenani, Awa ndi maphatikizo a malembo. Ndikufunsa kuti uwerenge maphatikizo ochuluka mmene ungathere. Mwachitsanzo, phatikizo ili ndi: “jo”. Tiye tiwerenge phatikizo ili: [lozani phatikizo loti “bwe”]: [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, phatikizo ili ndi “bwe“ [Ngati ophunzira alephere kuyankha molondola, nenani]: phatikizo ili ndi “bwe” Yesa phatikizo lina: werenga phatikizo ili [ lozani phatikizo loti “nu”] [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, phatikizo ili ndi “nu” [Ngati ophunzira alephere kuyankha molondola, nenani]: phatikizo ili ndi “nu“ Ndikanena kuti yamba, uwerenge maphatikizo mofulumira ndi mosamala. Werenga maphatikizo ali pa mzere uli wonse. Ndikhala chete kukumvetsera pokhapokha ukafuna chithandizo. Kodi ukudziwa zomwe ukuyenera kuchita? Ngati wakonzeka tiye tiyambepo. ` Yambani kuwerengera nthawi pamene ophunzira wawerenga phatikizo loyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pensulo polemba chizindikiro ichi ( / ). Werengerani phatikizo lomwe wazikonza yekha ngati lolondola. Ngati mwachonga kale mayankho odzikonza yekha ngati olakwa, zunguzani mzere pa phatikizolo ndi kupitiriza. Khalani chete pokapokha akamapereka mayankho motere: ngati ophunzira adodoma kuyankha pa masekondi atatu, lozani phatikizo lotsatira ndi kunena, pitiriza. Izi ziyenera kuchitika kamodzi kokha.Chongani phatikizo lomwe mwapereka kwa mwana. PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI nenani “lekeza pomwepo.” Lozerani phatikizolomalizira kuwerenga ndi chizindikiro ichi. Lamulo loyamba: Ngati ophunzira alephere kupereka yankho lolondola limodzi mu mzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani mu kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito Chitsanzo : jo bwe nu 1 2 3 4 5 6 7 8 9 10 ka mi po ra bwa Dza mnya na da li (10) 128 nja thu da ki fu Ngi ko tsi i mphu (20) mfu fa o se pi Lu mda mse dzi tsa (30) ma ye re na me Pa mkha wo si ntha (40) dya nyu u wa ri Ka mwa ba ku go (50) e le tu sa nkho Nga fi wi la nda (60) te mba ndi ti zi Zo va ya no mu (70) phu mbo Be cha kwa Mbi tho za ne chi (80) yo yi pe ke mle Kwe ndo wu nkha ta (90) tso ngo ni A kho Bwi lo nzi ndu mo (100) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi: Chongani m’kabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mzere oyamba. GAWO 5: KUWERENGA MAWU ODZIWIKA Onetsani ophunzira pepala la malembokuchokera m’buku la ophunzira. Nenani, Awa ndi mawu a m’Chichewa. Ndipo ndikufuna iwe undiwerengere mawu ambiri omwe ungathe. Mwachitsanzo, mawu awa: “khama”. Tiye tiwerenge mawu awa: [lozani mawu oti “ona.”]: [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, mawu awa ndi “ona” [Ngati ophunzira alephere kuyankha molondola, nenani]: mawu awa ndi “ona”. Yesa mawu ena: werenga mawu awa [ lozani mawu oti “bakha”] [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, mawu awa ndi “bakha” [Ngati ophunzira alephere kuyankha molondola, nenani]: mawu awa ndi “bakha” Ndikanena kuti yamba, uwerenge mawu mofulumira ndi mosamala. Werenga mawuwo pa mzere uli wonse. Ndikhala chete kukumvetsera pokhapokha ukafuna chithandizo. Kodi ukudziwa zomwe uchite? Ngati wakonzeka tiye tiyambepo. Yambani kuwerengera nthawi pamene ophunzira wawerenga mawu woyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwika pogwiritsa ntchito pensulo polemba chizindikiro ichi (/ ). Werengerani mawu odzikonza yekha ngati olondola. Ngati mwachonga kale mayankho odzikonza yekha ngati olakwa, zunguzani mzere pa lembolo ndi kupitiriza. Khalani chete pokapokha akamapereka mayankho motere: ngati ophunzira adodoma kuyankha pa masekondi atatu, werengani mawuwo ndi kunena, pitiriza. Izi ziyenera kuchitika kamodzi kokha.Chongani mawu omwe mwapereka kwa mwana. PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI (60)nenani “lekeza pomwepo.” Lozerani mawu omalizira kuwerenga ndi chizindikiro ichi (]). Lamulo loyamba: Ngati ophunzira alephere kuwerenga mawu amodzi mu mzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani m’kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. Chitsanzo : khama ona bakha 129 1 2 3 4 5 Atate chiwala Amayi zovala chakudya (5) Zina atate nyumba lata ndi (10) Fisi malangizo Mutu mbalame mnyamata (15) Pamanda agogo Tsiku chimanga bwino (20) Monga mbewu Zinthu anthu mitengo (25) Kalulu ambiri kwambiri ana abambo (30) Mbozi kwa zakudya mphunzitsi koma (35) Izi kudziwa Lina mlonda kusamala (40) Kuti zipatso nkhalango iwo zambiri (45) Mlendo ena mbatata Iye akulu (50) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi: Chongani m’kabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mzere oyamba. 130 Gawo 6. Kuwerenga Mawu Opeka Onetsani wophunzira pepala la malembo kuchokera m’buku la ophunzira.Nenani, Awa ndi mawu ongopeka m’Chichewa. Ndipo ndikufuna undiwerengere mawu omwe ungathe. Mwachitsanzo, “biva”. Yesera kuwerenga mawu awa: [lozani mawu oti “lufa”]: [Ngati wophunzira anene kuti “aga” nenani]: Wakhoza, mawu awa ndi “aga“ [Ngati wophunzira alephere kuwerenga mawu woti “aga”nenani] Mawu awa timatchula kuti “aga” Yesera mawu ena: werenga mawu awa [lozani mawu woti “kete“]. [Ngati wophunzira anene kuti”kete” molondola, nenani]: Wakhoza, mawu awa ndi “kete” [Ngati wophunzira alephere kutchula “kete” molondola nenani]: “Mawu awa timatchula kuti “kete” Ndikanena kuti yamba, uwerenge mawu mofulumira ndi mosamala. Uwerenge mawuwo kuyambira mzere woyamba. Ndikhala chete kumvera pamene ukuwerenga, ukalephera kuwerenga mawu ena ndikuthandiza. Ngati wakonzeka yamba. Yambani kuwerengera nthawi pamene ophunzira wawerenga lembo loyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pensulo polemba chizindikiro ichi ( / ). Werengerani ngati cholondola pamene wophunzira wadzikonza yekha. Ngati munachonga kale mayankho wodzikonza yekha ngati olakwa, zunguzani mzere pa mawuwo ndi kupitirira. Khalani chete wophunzira akamawerenga, ngati wophunzira wadodoma kuwerenga mawu pa masekondi atatu, werengani mawuwo ndipo lozani mawu otsatira ndikumuuza kuti “ pitiriza”. Izi ziyenera kuchitika kamodzi kokha Chongani mawu omwe mwapereka kwa wophunzira. Ngati wophunzira awerenga mawu asanu molakwitsa,asapitilize ndipo chongani mkabosi komwe kali patsamba lotsatira . PAKATHA MASEKONDI MAKUMI ASANU (60 )NDI LIMODZI NENANI “lekeza pomwepo.” Lozerani mawu omalizira kuwerenga ndi chizindikiro ichi (]) Lamulo loyamba: Ngati wophunzira walephere kuwerenga mawu a mumzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani m’kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. . Chitsanzo : lufa aga kete 1 2 3 4 5 Aza Leta geba upa atu (5) Omo Mnkhawi mvuvu bwazo goju (10) Nthibe Aza Suule mpholi nkhiki (15) Tchefe Juje udo mng’ene nkhwena (20) Booli Chizi thyata eze ngogo (25) zefa Mnapa mphwika pwika sati (30) thobi Uto khuda tapuli ono (35) ndwigo Faano Fese bzyata nyanu (40) zeepi Iso Patu ilu deeni (45) popo Phena Laafi tetu ntchuka (50) 131 Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi: Chongani m’kabokosi ngati ntchitoyi sinapitirizidwe chifukwa wophunzira analibe mayankho olondola mu mzere woyamba. 132 GAWO 7A. KUMVETSERA NKHANI Iyi ndi nkhani yayifupi. Ndifuna iwe undiwerengere mokweza, mofulumira koma mosamala. Ukatha kuwerengako ndikufunsa mafunso pa zomwe wawerenga. Yamba kuwerenga. Yambani kuwerengera nthawi pamene wophunzira wawerenga mawu oyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pensulo polemba chizindikiro ichi ( / ). Werengerani ngati cholondola pamene wophunzira wadzikonza yekha. Ngati munachonga kale mawu wodzikonza yekha ngati olakwa, lembani mzere mozungulira mawuwa ndi kupitirira. Khalani chete wophunzira akamawerenga, ngati wophunzira wadodoma kuwerenga pa mphindi zitatu, muwerengereni mawuwo kenak lozani mawu otsatira ndikumuuza kuti “ pitiriza”.Izi ziyenera kuchitika kamodzi kokha Chongani mawu omwe mwapereka kwa wophunzira. PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI (60) NENANI “lekeza pomwepo.” Lozerani mawu omalizirakuwerenga ndi chizindikiro ichi (]) Lamulo loyamba: Ngati wophunzira walephera kuwerenga mawu a mumzere woyamba, nenani “Zikomo”siyira pomwepa kuwerenga. Ndipo chongani m’kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. GAWO 7B. KUWERENGA NDI KUMVETSA NKHANI Pakatha masekandi 60 kapena wophunzira akatsiriza kuwerenga ndime m’masekandi osaposera 60, chotsani ndimeyo patsogolo pa ophunzira ndipo werengani funso loyamba. Mpatseni wophunzira masekandi 15 kuti ayankhe funsolo.Chongani yankho la wophunzira ndi kumuwerenga funso lotsatira. Werengani mafunso a mzere uliwonse mpaka pamene ophunzira walekeza kuwerenga. 133 Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi) : Chongani m’kabokosi ngati ntchitoyi sinapitirizidwe chifukwa wophunzira analibe mayankho olondola mu mzere woyamba Tsopano ndikufunsa mafunso angapo okhudza nkhani yomwe wawerenga. wakhoza walakwa sakudziw a Palibe yankho Lachisanu m’mawa Mada anakonzeka kupita ku sukulu. 6 Kodi nkhaniyi inachitikira kuti ? [Nkhaniyi imachitikira ku sukulu. Tsiku lotsekera sukulu] Tsikuli lidali lotsekera sukulu. Mafumu ndi makolo anafika ku sukulu ya Kaliza kuti adzawonerere luso lowerenga. 22 Nanga chimachitikira pa tsikuli ndi chiyani? [Ophunzira a Sitandade 1 amawonetsa luso lowerenga.] Iyeyu adali ndi nkhawa chifukwa adali mtsikana wamng’ono ndipo anali kuyamba kumene sitandade 1. 36 Kodi n’chifukwa chiyani Mada anali ndi nkhawa? [Mada anali ndi nkhawa chifukwa anali mtsikana wamng’ono. Kunali kuyamba kumene sitandade 1] Mada anawerenga mopatsa chidwi poyerekeza ndi msinkhu wake. Anthu adasangalala kwambiri ndipo anamusupa ndalama. 49 Tchulani chifukwa chimene mbiri ya Mada inapita patali? [Mada amawerenga mopatsa chidwi poyerekeza ndi msinkhu wake.] Mbiri ya Mada idapita patali. 54 Kodi anthu amamusupa chiyani Mada ? [Anthu adamusupa Mada ndalama] 134 Gawo 8. Kumvetsa Nkhani Ntchito iyi siyofunika kugwiritsa ntchito TSAMBA LA WOPHUNZIRA. (Werengani ndimeyi mokweza kawiri mopatsa chidwi.) Ndiwerengera ndime yayifupi kawiri kenaka ndidzakufunsa mafunso angapo. Chonde umvetsere bwino pamene ndikuwerenga nkhaniyi. Uyenera kuyankha mafunsowa m’mene ungathere. Kodi ukudziwa chomwe ukuyenera kuchita? Kodi uli wokonzeka? Tiyeni tiyambe tsopano. Tsiku lina ndimapita ku mtsika kukagula nyama. Mphepete mwamsewu ndinaona chikwama ndipo ndinachitola. Mkati mwa chikwamacho munali ndalama ndi makadi a ku banki. Nditawauza mayi anga iwo anandilangiza kukapereka chikwamacho kwa Mfumu. Tsiku lina mayi anga anayitanidwa kwa Mfumu. Kumeneku tinakumana ndi abambo ena omwe anali mwini chikwama chija. Bambowa anathokoza ndi ndalama zokwana K5000.00 ndi kulonjeza kupereka chithandizo pa maphunziro anga. Tsopano ndikufunsa mafunso angapo okhudza nkhani yomwe ndawerenga. wakhoza walakwa sakudziw a palibe yankho Kodi nkhaniyi idachitika kuti? [Inachitika kumudzi, mphepete mwa msewu, popita ku msika] Kodi mkati mwa chikwama munali chiyani? [munali ndalama ndi makadi a ku banki] Chifukwa chiyani chikwama anakachipereka kwa Mfumu? [kuti chisungike chinthu a mfumu amayenera kudziwa] Kodi kwa mfumu kunabwera ndani? [Kunabwera, mwini wa chikwama] Ndi mphatso yanji yomwe mwini chikwama uja anapereka? [mphatso ya ndalama zokwana K5000.00 ndi chithandizo pa maphunziro] Nthawi yomaliza kuyesa ophunzira: ____ : _____ (maola 24) 135 ANNEX 11: EARLY GRADE READING ASSESSMENT – MIDLINE Malawi Early Grade Reading Assessment: Student Response Form Administrator Instructions and Protocol, April – May 2015 Chichewa Malangizo: Muyenera kukhazikitsa ubwenzi wabwino ndi wophunzira amene mukumuyesa kudzera mu nkhani zifupizifupi komanso zosangalatsa kuti aone mafunsowa ngati sewero chabe osati ntchito yovuta. Nkoyenera kuwerenga zigawo zokhazo zomwe zili mumabokosi mokweza, momveka bwino ndi modekha. Kupempha chilolezo Chongani mukabokosika ngati ophunzira wavomereza kuyesedwa: INDE (Ngati wophunzira sanavomereze kuyesedwa, muthokozeni ndi kuitana ophunzira wina pogwiritsa ntchito chipepala chomwechi.) A. Tsiku la Mayeso Tsiku : _______ H. Kalasi ○ 1 = Sitandade 2 Mwezi :_______ 2 = Sitandade 4 B. Dzina la Woyesa C. Dzina la Sukulu I. Dzina la Mphunzitsi Uli bwanji? Dzina langa ndi_________ndipo ndimakhala ku ________. (Chezani ndi wophunzira munjira yomwe ingathandize kuti amasuke). • Ndikuuze chifukwa chimene ndabwerera kuno. Ndimagwira ntchito ku Unduna wa za Maphunziro, za Sayansi ndi Luso. Ndikufuna kudziwa m’mene inu ophunzira mumaphunzirira kuwerenga. Mwa mwayi iwe wasankhidwa kuti ndicheze nawe. • Ndikufuna kuti tikambirane pa zimenezi koma ngati sukufuna utha kubwerera m’kalasi. • Tichita sewero lowerenga. Ndikufunsa kuti undiwerengere malembo, mawu ndi nkhani mokweza. • Ndigwiritsa ntchito wotchi iyi kuti ndiwone nthawi yomwe utenge powerenga. • Awa simayeso, ndipo sizikhudzana ndi zotsatira za maphunziro ako. • Ndikufunsanso mafunso ena okhudzana ndi banja la kwanu monga, chiyankhulo chomwe mumayankhula kunyumba kwanu ndi zinthu zina zomwe muli nazo kwanu. • Sindilemba dzina lako ndipo palibe amene adziwe zimene tikambirane. • Ndibwerezanso kuti uli ndi ufulu woyankha mafunso kapena ayi. Ngakhale tili mkati mwa kucheza uli ndi ufulu kukana kuyankha mafunso. • Uli ndi funso tisanayambe? Tikhoza kuyamba? 136 D. Dera J. Sitilimu E. Boma K. Nambala ya Chinsinsi ya Ophunzira F. Chigawo L. Zaka zakubadwa G. Mtundu wa Sukulu: ○ 1 = Tsiku lonse ○ 2 = M’mawa ○ 3 = Masana M. Mwamuna kapena Mkazi ○ 0 = Mwamuna ○ 1= Mkazi N. Nthawi Yoyambira 5710 ___ : ___ Gawo 1. Kudziwa Dzina la Lembo Onetsani ophunzira pepala la malembo mu buku la ophunzira. Nenani: Yambani kuwerengera nthawi pamene ophunzira wawerenga lembo loyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pensulo polemba chizindikiro ichi ( / ). Werengerani lembo limene walikonza yekha ngati lolondola. Ngati mwachonga kale mayankho odzikonza yekha ngati olakwa, zunguzani mzere pa lembolo ndi kupitirira. Khalani chete pokhapokha akamapereka mayankho motere: ngati ophunzira adodoma kuyankha pa masekandi atatu, Perekani dzina la lembo, lozani lembo lotsatira ndi kunena, Pitiriza. Chongani lembo lomwe mwapereka kwa mwana. Ngati ophunzira apereke liwu la lembo osati dzina lalembo, mpatseni dzina lalembolo ndi kunena: Tandiuze DZINA lalembo ili. Izi ziyenera kuchitika kamodzi kokha. PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI nenani “lekeza pomwepo.” Chongani lembo lomalizira ndi chizindikiro ichi (I) PAKUTHA PA MASEKONDI 60 NENANI “ lekeza pomwepo”). Lamulo loyamba: Ngati ophunzira alephere kupereka yankho lolondola limodzi mu mzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani mu kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. Chitsanzo : F v L 1 2 3 4 5 6 7 8 9 10 T i J N S n A t e h (10) l z a V B o H r N A (20) A C f C S a S o E U (30) e N t O a e C t o Wachita bwino. Tsopano tiye tipite ku gawo lotsatira. Ili ndi tsamba la malembo a alifabeti. Ndiuze maina a malembo amene ungathe. Mwachitsanzo, dzina la lembo [lozani lembo la ‘F’] ndi F Tiye tiyesere: ndiuze dzina la lembo ili [lozani lembo la ‘V’] Ngati ophunzira ayankhe bwino nenani: Wakhoza dzina la lembo ili ndi ‘Vii’: Ngati ophunzira alephere kuyankha molondola, nenani: Dzina la lembo ili ndi ‘Vii’ Tsopano yesera lembo lina: ndiuze dzina la lembo ili [lozani lembo la L]: Ngati mwana wayankha molondola, nenani: Wakhoza, dzina la lembo ili ndi “ “ELL” Ngati mwana walephera kuyankha molondola, nenani: dzina la lembo ili ndi “ ELL” Kodi ukudziwa chomwe ukuyenera kuchita? Ndikanena kuti “Yamba” Chonde tchula dzina la lembo lili lonse mofulumira ndi mosamala. Yamba pano ndipo ndi kupitiriza motere [Lozani lembo loyamba mu mndandanda woyamba pamathero a chitsanzo ndipo lozetsani chala pa mzere woyamba. Ngati wafika pa lembo lomwe sukulidziwa, ndikuuza dzina lake.Ndikakuwuza udzipitiriza. Wakonzeka? Yamba ts an 137 O (40) d L E d G E N o m t (50) h e K w T i L g y H (60) e i e t H I S e T f (70) R y W p U s i l e I (80) R o a E d n D a s I (90) r C n U r T P t m h (100) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi): Chongani mukabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mzere oyamba. Gawo 2. Maphatikizo a Malembo Werengani ndi kutchula mawu oyenera kachiwiri. Lolani yankho lokhalo lili ndi liwu lolondola. Ngati ophunizra akanike kuyankhe mumasekondi atatu, onetsani kuti “Palibe yankho” ndipo pitirizani kutchula mawu otsatira. Tchulani momveka bwino koma musatsindike kwambiri paphatikizo loyamba la mawu ena ali wonse. Langizo loyamba: Ngati ophunzira alephere kuyankha molondola kapena kulephera kuwerenga mawu asanu oyambirira, nenani kuti “Zikomo”, ndipo musapitirize ntchiyoyi ndipo mukatero chongani m’kabokosi kali pamapeto a tsamba lino ndi kuyamba ntchito yotsatirayo. Kodi ndi maphatikizo ati amene ali mu mawu awa “_______”? [bwerezani mawuwo kawiri] Wakhoza = 2 Walakwa/ sakudziwa = 1 Palibe yankho = 0 Bola Bo –la o wakhoza o Walakwa/ sakudziwa o palibe yankho Mkaka Mka – ka o wakhoza o Walakwa/ sakudziwa o palibe yankho Mwamuna Mwa – mu – na o wakhoza o Walakwa/ sakudziwa o palibe yankho Wachita bwino. Tsopano tiye tipite ku gawo lotsatira. Ntchito iyi ndiyongomvera chabe. Ndikuuza mawu ndipo undiuze maphatikizo omwe ali mu mawuwo. Mwachitsanzo, mu mawu oti “ola” muli maphatikizo awa: “o – la”. Mu ntchito imeneyi ndikufuna kuti undiuze maphatikizo amene uwamve m’mawu. Nditchula mawuwa kawiri. Umvere kenako undiuze maphatikizo omwe ali mu mawuwo. Tiye tiyesere. Undiuze maphatikizo omwe ali m’mawu oti “mayi”? “mayi.” [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, maphatikizo a mawu oti “mayi” ndi “ma – yi”. Ngati mwana walephera kuyankha molondola, nenani: Mveranso kachiwiri: “mayi”. Maphatikizo omwe ali mu mawu oti “mayi” ndi “ma-yi.” Tsopano yesera ena: kodi ndi maphatikizo ati amene ali m’mawu oti “khwanya”? “khwanya”. [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, maphatikizo a mawu oti “khwanya” ndi “khwa - nya ”. Ngati mwana walephera kuyankha molondola, nenani: Mveranso kachiwiri: “khwanya”. Maphatikizo omwe ali mu mawu oti “khwanya” ndi “khwa - nya.” Kodi ukudziwa chomwe uyenera kuchita? [Ngati ophunzira anene kuti ayi, muuzeni kuti]: Yesetsa mmene ungathere. (mawu 5) 138 Ana A – na o wakhoza o Walakwa/ sakudziwa o palibe yankho Boola Bo-o – la o wakhoza o Walakwa/ sakudziwa o palibe yankho Kakamiza Ka – ka – mi – za o wakhoza o Walakwa/ sakudziwa o palibe yankho Mnkhwani Mnkhwa – ni o wakhoza o Walakwa/ sakudziwa o palibe yankho Kankha Ka-nkha o wakhoza o Walakwa/ sakudziwa o palibe yankho Nama Na – ma o wakhoza o Walakwa/ sakudziwa o palibe yankho Mbola Mbo - la o wakhoza o Walakwa/ sakudziwa o palibe yankho Chongani mukabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mawu asanu oyamba: Gawo 3. Kutchula liwu loyamba Ntchito iyi siyofunika kuwerengera nthawi ndipo PALIBE TSAMBA LAWOPHUNZIRA. Werengani mawu aliwonse kawiri ndipo mufunse ophunzira kuti atchule liwu loyamba m’mawu amenewa. kumbukirani kutchula maliwu moyenera : /p/ osati /pu/ monga: /p/, ----- “puh” kapena “pe.” Nenani: Werengani ndi kutchula mawu oyenera kawiri. Lolani yankho lokhalo lili ndi liwu lolondola. Ngati ophunizra akanike kuyankha mu masekondi atatu,onetsani kuti “Palibe yankho” ndipo pitirizani kutchula mawu otsatira. Tchulani momveka bwino koma musatsindike kwambiri liwu loyamba la mawu ena ali wonse. Langizo loyamba: Ngati ophunzira alephere kuyankha molondola kapena kulephera kuwerenga mawu asanu oyambirira, nenani kuti “Zikomo”, ndipo musapitirize ntchiyoyi ndipo mukatero chongani m’kabokosi kali pamapeto a tsamba lino ndi kuyamba ntchito yotsatirayo. Tchula liwu loyamba mu mawu awa: Kodi liwu loyamba “_______”? “_______”? [Tchulani mawuwo] Wachita bwino. Tsopano tiye tipite ku gawo lotsatira. Ntchito iyi ndiyomvera chabe. Ndikufuna kuti undiuze liwu loyamba m’mawu ena aliwonse. Mwachitsanzo, m’mawu oti ‘galu’, liwu loyamba ndi “/g/”. Mu ntchito imeneyi, ndifuna undiuze liwu loyamba limene ukulimva mu mawu ena aliwonse. Nditchula mawuwo kawiri. Umvere mawuwo, kenako undiuze liwu loyamba lomwe likumveka m’mawuwo. Tiye tiyesere. Kodi liwu loyamba m’mawu oti “mayi”? “mayi” ndi chiyani? [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, liwu loyamba mu mawu oti “mayi” ndi /mmmmm/ [Ngati ophunzira sanayankhe molondola, nenani]: mvetsera kawiri: “mmmayi”. Liwu loyamba mu mawu oti “mayi” ndi /mmmmm/. Tsopano yesera mawu ena: Kodi ndi liwu liti lomwe lili mmawu oti “nzimbe”? “nzimbe”. Ngati mwana wayankha molondola, nenani: Wakhoza, liwu loyamba mu mawu oti “nzimbe”ndi “/n/” Ngati mwana walephera kuyankha molondola, nenani: mveranso kaciwiri: liwu loyamba la mu mawu oti “nzimbe” ndi /n/ Kodi ukudziwa chomwe uyenera kuchita? [Ngati wophunzira anene kuti ayi, muzeni kuti]: Yesetsa mmene ungathere. 139 Wakhoza = 2 Walakwa/ sakudziwa = 1 Palibe yankho = 0 Atate /a/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Bala /b/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Dona /d/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Kala /k/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Khala /kh/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Wada /www/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Gwada /g/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Gada /g/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Mana /mmm/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Nola /n/ o wakhoza o Walakwa/ sakudziwa o palibe yankho Chongani mukabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mawu asanu oyamba: Gawo 4. Kuwerenga Maphatikizo Onetsani ophunzira pepala la maphatikizo mu buku la ophunzira.Nenani, Yambani kuwerengera nthawi pamene ophunzira wawerenga phatikizo loyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pensulo polemba chizikiro ichi ( / ). Werengerani yozikonza yekha Wachita bwino. Tsopano tiye tipite ku gawo lotsatira. Awa ndi maphatikizo a malembo. Ndikufunsa kuti uwerenge maphatikizo ochuluka mmene ungathere. Mwachitsanzo, phatikizo ili ndi: “go”. Tiye tiwerenge phatikizo ili: [lozani phatikizo loti “kwa”]: [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, phatikizo ili ndi “kwa“ [Ngati ophunzira alephere kuyankha molondola, nenani]: phatikizo ili ndi “kwa” Yesa phatikizo lina: werenga phatikizo ili [lozani phatikizo loti “se”] [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, phatikizo ili ndi “se” [Ngati ophunzira alephere kuyankha molondola, nenani]: phatikizo ili ndi “se“ Ndikanena kuti yamba, uwerenge maphatikizo mofulumira ndi mosamala. Werenga maphatikizo ali pa mzere uli wonse. Ndikhala chete kukumvetsera pokhapokha ukafuna chithandizo. Kodi ukudziwa zomwe ukuyenera kuchita? Ngati wakonzeka tiye tiyambepo. 140 ngati yolondola. Ngati mwachonga kale mayankho odzikonza yekha ngati olakwa, zunguzani mzere pa phatikizolo ndi kupitiriza. Khalani chete pokapokha akamapereka mayankho motere: ngati ophunzira adodoma kuyankha pa masekondi atatu, lozani phatikizo lotsatira ndi kunena, pitiriza. Izi ziyenera kuchitika kamodzi kokha.Chongani phatikizo lomwe mwapereka kwa mwana. PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI nenani “lekeza pomwepo.” Chongani phatikizolomalizira ndi chizindikiro ichi (I) PAKUTHA PA MASEKONDI 60 NENANI “lekeza pomwepo”). Lamulo loyamba: Ngati ophunzira alephere kupereka yankho lolondola limodzi mu mzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani mu kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito Chitsanzo : go kwa se 1 2 3 4 5 6 7 8 9 10 pe ye da ngi mbe yi ti no pa le (10) chi ka ni dya zo li ku ngo dzi ndo (20) e wu lo kwa si wi phu ri se nzi (30) nkho fa go mi zi ra mfu mse po ya (40) sa tho la mbo mda fi mo ta te na (50) nda nja mu pi ntha u na wa mnya lu (60) va tsa i kho tu tsi da tso nga za (70) mle me ko yo ne cha mkha mwa bwa thu (80) ndu mba A mbi fu wo dza nkha mphu ba (90) ndi ke re Be ma ki nyu kwe bwi o (100) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi: Chongani mukabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mzere oyamba. Gawo 5. Kuwerenga Mawu Odziwika Onetsani ophunzira pepala la malembo m’buku la ophunzira.Nenani, Yambani kuwerengera nthawi pamene ophunzira wawerenga mawu woyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwika pogwiritsa ntchito pensulo polemba chizikiro ichi (/). Werengerani yodzikonza yekha ngati yolondola. Ngati mwachonga kale mayankho odzikonza yekha ngati olakwa, zunguzani mzere pa lembolo ndi kupitiriza. Khalani chete pokapokha akamapereka mayankho motere: ngati ophunzira adodoma kuyankha pa masekondi atatu, werengani mawuwo ndi kunena, pitiriza. Izi ziyenera kuchitika kamodzi kokha.Chongani mawu omwe mwapereka kwa mwana. Wachita bwino. Tsopano tiye tipite ku gawo lotsatira. Awa ndi mawu a m’Chichewa. Ndipo ndikufuna iwe undiwerengere mawu ambiri omwe ungathe. Mwachitsanzo, mawu awa: “gona”. Tiye tiwerenge mawu awa: [lozani mawu oti “chili.”]: [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, mawu awa ndi “chili” [Ngati ophunzira alephere kuyankha molondola, nenani]: mawu awa ndi ”chili.” Yesa mawu ena: werenga mawu awa [ lozani mawu oti “fodya”] [Ngati ophunzira ayankhe molondola, nenani]: Wakhoza, mawu awa ndi “fodya” [Ngati ophunzira alephere kuyankha molondola, nenani]: mawu awa ndi “fodya” Ndikanena kuti yamba, uwerenge mawu mofulumira ndi mosamala. Werenga mawuwo pa mzere uli wonse. Ndikhala chete kukumvera pokhapokha ukafuna chithandizo. Kodi ukudziwa zomwe uchite? Ngati wakonzeka tiye tiyambepo. 141 PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI nenani “lekeza pomwepo.” Chongani mawu omalizira ndi chizindikiro ichi (I) PAKUTHA PA MASEKONDI 60 NENANI “lekeza pomwepo”). Lamulo loyamba: Ngati ophunzira alephere kupereka yankho lolondola limodzi mu mzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani mu kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. Chitsanzo : gona chili fodya 1 2 3 4 5 ena chimanga fisi kalulu pamanda (5) kusamala Mutu mnyamata malangizo nyumba (10) atate zina ndi kudziwa nkhalango (15) koma izi akulu agogo mlendo (20) tsiku kwambiri mbalame mbatata ana (25) lata mbewu chakudya mbozi anthu (30) iwo amayi zinthu zambiri zakudya (35) zovala Iye lina bwino chiwala (40) ambiri abambo adali mlonda kuti (45) kwa monga mphunzitsi mitengo zipatso (50) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi: Chongani mukabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mzere oyamba. Gawo 6. Kuwerenga Mawu Opeka Onetsani wophunzira pepala la malembo m’buku la ophunzira.Nenani, Yambani kuwerengera nthawi pamene ophunzira wawerenga lembo loyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pennsulo polemba chizindikiro ichi ( / ). Werengerani ngati cholondola pamene wophunzira wadzikonza yekha. Ngati munachonga kale mayankho wodzikonza yekha ngati olakwa, zunguzani mzere pa mawuwo ndi kupitirira.. Khalani chete wophunzira akamawerenga, ngati wophunzira wadodoma kuyankha pa masekondi atatu, werengani mawuwo ndipo lozani mawu otsatira ndikumuza kuti “ pitiriza”. Chongani mawu omwe mwapereka kwa wophunzira. Ngati wophunzira awerenga mawu asanu molakwitsa,asapitilize ndipo chongani mkabosi komwe kali patsamba lotsatira . PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI NENANI “lekeza pomwepo.” Chongani mawu omalizira ndi chizindikiro ichi (/) Lamulo loyamba: Ngati wophunzira walephere kuwerenga mawu a mumzere woyamba, nenani “Zikomo”siyilani pomwepo ntchitoyi ndipo chongani m’kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. Chitsanzo : yono ndodi biva Wachita bwino. Tsopano tiye tipite ku gawo lotsatira. Awa ndi mawu ongopeka m’Chichewa. Ndipo ndikufuna undiwerengere mawu omwe ungathe. Mwachitsanzo, “yono”. Yesera kuwerenga mawu awa: [lozani mawu oti “ndodi”]: [Ngati wophunzira anene kuti “ndodi” nenani]: Wakhoza, mawu awa ndi “ndodi“ [Ngati wophunzira alephere kuwerenga mawu woti “ndodi”nenani] Mawu awa timatchula kuti “ndodi” Yesera mawu ena: werenga mawu awa [lozani mawu woti “biva“]. [Ngati wophunzira anene kuti”biva” molondola, nenani]: Wakhoza, mawu awa ndi “biva” [Ngati wophunzira alephere kutchula “biva” molondola nenani]: “Mawu awa timatchula kuti “biva” Ndikanena kuti yamba, uwerenge mawu mofulumira ndi mosamala. Uwerenge mawuwo kuyambira mzere woyamba. Ndikhala chete kumvera pamene ukuwerenga, ukalephera kuwerenga mawu ena ndikuthandiza. Ngati wakonzeka yamba. 142 1 2 3 4 5 iso tapuli patu omo udo (5) popo eze mphwika ilu nkhiki (10) phena uto bwazo ntchuka ngogo (15) soola ndwigo mng’ene sati goju (20) thyata nthibe pwika nkhwena faano (25) upa tetu bzyata mnkhawi leta (30) booli fese juje geba khuda (35) atu ono chizi laafi mpholi (40) tchefe nyanu aza thobi zeepi (45) Suule mvuvu mnapa deeni zefa (50) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi: Chongani m’kabokosi ngati ntchitoyi sinapitirizidwe chifukwa wophunzira analibe mayankho olondola mu mzere woyamba. Gawo 7a. Kumvetsera nkhani Yambani kuwerengera nthawi pamene wophunzira wawerenga mawu oyamba. Yendetsani pensulo ndi kuchonga moyenera yankho lolakwa pogwiritsa ntchito pensulo polemba chizindikiro ichi ( / ). Werengerani ngati cholondola pamene wophunzira wadzikonza yekha. Ngati munachonga kale mawu wodzikonza yekha ngati olakwa, lembani mzere mozungulira mawuwa ndi kupitirira. Khalani chete wophunzira akamawerenga, ngati wophunzira wadodoma kuwerenga pa mphindi zitatu, muwerengereni mawuwo kenak lozani mawu otsatira ndikumuuza kuti “ pitiriza”. Chongani mawu omwe mwapereka kwa wophunzira. Izi ziyenera kuchitika kamodzi kokha. PAKATHA MASEKONDI MAKUMI ASANU NDI LIMODZI NENANI “lekeza pomwepo.” Chongani mawu omalizira ndi chizindikiro ichi (/) Lamulo loyamba: Ngati wophunzira walephere kuwerenga mawu a mumzere woyamba, nenani “Zikomo”siyila pomwepa kuwerenga. Ndipo chongani m’kabokosi komwe kali pamapeto ndi kupitiriza ndi ntchito ina. Wachita bwino. Tsopano tiye tipite ku gawo lotsatira. Iyi ndi nkhani yayifupi. Ndifuna iwe undiwerengere mokweza, mofulumira koma mosamala. Ukatha kuwerengako ndikufunsa mafunso pa zomwe wawerenga. Yamba kuwerenga. 143 Gawo 7b. Kuwerenga ndi kumvetsa nkhani Pakatha masekandi 60 kapena wophunzira akatsiriza kuwerenga ndime m’masekandi zosaposera 60, chotsani ndimeyo patsogolo pa ophunzira ndipo werengani funso loyamba. Mpatseni wophunzira masekandi 15 kuti ayankhe funsolo, chongani yankho la wophunzira ndi kumuwerenga funso lotsatira. Werengani mafunso a mzere uliwonse mpaka pamene ophunzira walekeza kuwerenga. Tsopano ndikufunsa mafunso angapo okhudza nkhani yomwe wawerenga. Wakhoza = 2 Walakwa = 1 Palibe yankho = 0 Lidali tsiku lachisanu pamene sukulu yathu ya Kapeni idasewera mpira ndi ya Chimutu. 13 Kodi ndi sukulu ziti zinkasewera mpira? (Kapeni ndi Chimutu) Tidakonzekera kwambiri ndi cholinga choti tipambane. Chifukwa chiyani a Kapeni anakonzekera 144 Nawonso ochemelera sadalekelere 22 kwambiri? (kuti apambane) Mpira udayamba. Mwadzidzidzi, oyimbira mpira adayimba wezulo ndipo nthawi yomweyo ochemelera a Chimutu adalowa m’bwalo akuvina ndi kuimba. 40 Kodi chidachititsa a Chimutu kuti alowe m’bwalo akuvina ndi kuimba ndi chiyani? (amasangalalira chigoli, sukulu yawo idagoletsa chigoli, oyimbira adayimba wezulo) Osewera athu sadakhutire ndi chigolicho chifukwa adaona kuti oyimbirayo sadatsatire malamulo. 51 Kodi oyimbira mpira adaonetsa khalidwe lanji? (lokondera, losadziwa) Ngakhale zidali choncho masewero adapitilira ndipo potsiriza sukulu yathu idapambana. 61 Ukuganiza kuti ndi chifukwa chiyani mpira udapitilira? (A Kapeni amadzidalira, a Kapeni adakonzekera kwambiri, aphunzitsi adawalimbikitsa) Lembani nthawi yomwe yatsala pa wotchi pamapeto (nambala ya masekandi) : Chongani m’kabokosi ngati ntchitoyi sinapitirizidwe chifukwa wophunzira analibe mayankho olondola mu mzere woyamba Gawo 8. Kumvetsa Nkhani 145 Ntchito iyi siyofunika kugwiritsa ntchito TSAMBA LA WOPHUNZIRA. (Werengani ndimeyi mokweza kawiri mopatsa chidwi.) Dzina langa ndine Madalitso. Ndimaphunzira ku Kwerani pulayimale sukulu. Kuyambira Lolemba mpaka Lachisanu ndimayenera kuvala yunifolomu. Tsiku lina ndikusewera chipako ndi anzanga, ananding’ambira yunifolomu. Ndinadandaula kwambiri. Ndinadzimvera chisoni ndipo ndinapita kunyumba ndikulira. Nditafika kunyumba, ndinafotokoza zomwe zinachitika ndipo anandilonjeza kuti andigulira ina Tsopano ndikufunsa mafunso angapo okhudza nkhani yomwe wawerenga. Wakhoza = 2 Walakwa = 1 palibe yankho = 0 Kodi ndi sukulu yiti yomwe Madalitso amaphunzira? [Madalitso amaphunzira ku Kwerani pulayimale sukulu] Ndi chifukwa chiyani Madalitso akudandaula? [Yunifolomu yake yang’ambidwa, azivala chiyani popita ku sukulu, a phunzitsi akamubweza.] Kodi Madalitso akuliranji? [Madalitso amaopa kuti makolo ake akamukalipira] Madalitso anamva bwanji ndi zomwe makolo analonjeza? [Anakondwera, anavinavina] Kodi ubwino wa yunifolomu ndi chiyani? [Imadziwitsa komwe mwana akuphunzira, amaoneka okongola.] Chongani mukabokosi ngati ntchitoyi sinapitirizidwe chifukwa ophunzira analibe mayankho olondola mu mawu asanu oyamba: Ntchito iyi siyofunika kugwiritsa ntchito TSAMBA LA WOPHUNZIRA. Ndiwerengera ndime yayifupi kawiri kenaka ndidzakufunsa mafunso angapo. Chonde umvetsere bwino pamene ndikuwerengera nkhaniyi. Uyenera kuyankha mafunsowa mmene ungathere. Kodi ukudziwa chomwe ukuyenera kuchita? Kodi uli wokonzeka? Tiyeni tiyambe tsopano. 146 Gawo 9. Kucheza ndi ophunzira Funsani ophunzira funso lililonse momveka bwino monga mmene amachitira pocheza. Musawerenge mayankho onse kwa ophunzira mokweza. Dikirani ophunzira kupereka yankho ndipo mulilembe pa mpata womwe waperekedwa kapena kulemba mzere wozungulira chizindikiro cha yankho lomwe wophunzira wapereka. Ngati palibe malangizo ena otsutsana,yankho limodzi ndi limene likuloledwa. 1a Kodi chiyankhulo chomwe umaphuzirira kusukulu ndi chimenenso mumayankhula kunyumba? Ngati ayi, pitani ku funso 1b ..................... ……….0 Inde ........................................................... ……….1 Sakudziwa/Palibe yankho........................... ……….9 1b [Ngati yankho la funso 1a likhale Ayi] kodi ndi chiyankhulo chiti chimene umayankhula kunyumba? [Mayankho angapo ndi ololedwa] Chichewa ................................................... ……….1 Tumbuka .................................................... ……….2 Yao ............................................................. …….....3 Chingelezi ................................................... ……….4 zina (fotokozani): ........................................ …….…5 Sakudziwa/Palibe yankho .......................... .………9 Kodi kunyumba kwanu kuli zinthu ngati izi: Inde Ayi Sakudziwa 2 wailesi? 2 1 9 3 telefoni kapena telefoni ya m’manja? 2 1 9 4 magetsi? 2 1 9 5 televizyoni? 2 1 9 6 filiji? 2 1 9 7 chimbudzi cha mnyumba ? 2 1 9 8 njinga ? 2 1 9 9 njinga ya moto ? 2 1 9 10 galimoto, galimoto ya lole, thilakita kapena bwato la injini, ngolo, golosale, chigayo? 2 1 9 11 Kodi unapitapo kusukulu ya mkaka usalowe kalasi yoyamba? Ayi .............................................................. ……….0 Inde .......................................................... ….……1 Sakudziwa/Palibe yankho........................... .……..99 147 12 Kodi unali kalasi iti chaka chatha? Sindinali pa sukulu .................................... .………0 Sitandade 1 ................................................ .………2 Sitandade 2 ................................................ ….……3 Sitandade 3 ................................................ ….……4 Sitandade 4 ................................................ ….……5 Sakudziwa/Palibe yankho .......................... ….......99 13 Kodi chaka chatha unajombapo kusukulu kupyola sabata imodzi? Ayi .............................................................. …….....0 Inde .......................................................... ……….1 Sakudziwa/Palibe yankho .......................... ….......99 14 Kodi uli ndi mabuku owerenga a sukulu? Ayi ……………………………………………………..0 Inde …………………………..………...1 Sakudziwa/Palibe yankho .......................... ….......99 15 Kupatula mabuku a kusukulu, kodi pali mabuku ena, nyuzipepala kapena zinthu zina zowerenga kunyumba kwanu? Ayi ………………………………………….………….0 Inde ........................................................... ……….1 Sakudziwa/Palibe yankho .......................... ….......99 [Ngati inde, Funsani funso 15] chonde Perekani zitsanzo. (sikoyenera kulemba mayankho) 16 [Ngati inde kufunso 6] kodi mabuku amenewa kapena zinthu zimenezi zili mu chiyankhulo kapena ziyankhulo zanji ? [lolani mayankho ochuluka] Chingelezi ................................................... …..…...1 Chichewa .................................................... …..…...2 Tumbuka .................................................... …….....3 Zina (fotokozani): ........................................ ……….8 Sakudziwa/Palibe yankho .......................... ….......99 17 Kodi kunyumba kwanu umakhala ndi yani ? Makolo anga ............................................... …..…...0 Amayi anga ................................................ …….....1 Atate anga .................................................. ……….2 Agogo ......................................................... ……….3 Amalume .................................................... ……….4 Azakhali ...................................................... .………5 Achimwene ................................................. ….……6 148 Achemwali .................................................. .………7 Ena ( fotokozani) ........................................ ....…….8 18 Kodi amayi ako kapena okuyang’anira ako analekezera pati sukulu? Palibe ......................................................... .………0 Sukulu ina ................................................... ..…......1 Anatsiriza sukulu ya pulaimale ................... ..……..2 Anafika ku sukulu ya sekondale ................. ...…….3 Anatsiriza sukulu ya sekondale .................. ……….4 Sukulu ya za umisili .................................... ...….….5 Sukulu ya ukachenjede .............................. ..……...6 Zina (fotokozani) ......................................... ……….8 Sakudziwa/Palibe yankho .......................... ….......99 19 Kodi abambo ako kapena okuyang’anira ako analekezera pati sukulu? Palibe ......................................................... .………0 Sukulu ina ................................................... ..……...1 Anatsiriza sukulu ya pulaimale ................... .…..…..2 Anafika ku sukulu ya sekondale ................. ..…..….3 Anatsiriza sukulu ya sekondale .................. ……….4 Sukulu ya za umisili .................................... .…..….5 Sukulu ya ukachenjede .............................. …..…..6 Zina (fotokozani): ........................................ …..…..8 Sakudziwa/Palibe yankho .......................... ….......99 Nthawi yomaliza kuyesa ophunzira: ____ : _____ (maola 24) 149 ANNEX 12: LEARNER QUESTIONNAIRE Malawi EGRA Mid-Line Impact Evaluation Learner Questionnaire April-May 2015 Instructions: The enumerator should read each of the questions to the learner as is. He/she can also read the response choices (unless the question specifies that learners should not be prompted). Once the learner has selected an option, the letter associated with that option should be circled. Most questions should have only one response. However, in some cases, a question will specify that multiple responses are allowed. In those cases, the enumerator should circle the letters corresponding with all response options that apply. All regular text can be read to the respondents, and all italic text includes instructions to the enumerator. COMPLETE THESE QUESTIONS BEFORE STARTING THE ASSESSMENT OR QUESTIONNAIRE Student EMIS ID (Don’t know/Refused to Answer, enter 9999): Student Name: Date: Time Started: Time Ended: Enumerator Name: Survey and Logistics Manager Name: Technical Manager Name: School Name: School EMIS ID: Class Standard: Standard 2 Standard 4 (circle one) Learner ID Number (01 to 16, from sample): Questionnaire Number (EMIS + Standard + Learner number): learner background 1. What is your age?_________________________________________ 2. For how many years have you been attending school at this school? (Don’t prompt learners; let them answer, and then choose the best response based on their reply – you might need to compare this response to the learner’s age to make sure they are old enough to have been there that long.) a. Less than one year = 0 b. One year = 1 c. Two years = 2 d. Three years = 3 e. Four years = 4 f. More than four years = 5 g. Don’t know/Refuse to answer = 9999 150 3. In which class were you last year? a. Not in school = 0 b. Standard 1 = 1 c. Standard 2 = 2 d. Standard 3 = 3 e. Standard 4 = 4 f. Don’t know/Refuse to answer = 9999 4. Are you repeating your current standard this year? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 5. How often did you miss school because you were sick this academic year? a. Almost never = 1 b. Occasionally = 2 c. A lot = 3 d. Don’t know/Refuse to answer = 9999 6. Do you usually go to a clinic or hospital when you are sick? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 7. How often have you seen the doctor or nurse or visited a health clinic this academic year? a. Almost never = 1 b. Occasionally = 2 c. A lot = 3 d. Don’t know/Refuse to answer = 9999 reading 8. Does anyone at home read to you? a. No = 0 (Skip to QUESTION 9) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 9) 9. How often does someone at home read to you? (Don’t prompt learners; let them answer without reading the answer choices) a. Hardly ever = 1 b. Only sometimes = 2 c. 2-3 times a week = 3 d. Every day = 4 e. Don’t know/Refuse to answer = 9999 10. Do you read on your own at home? (Don’t prompt learners; let them answer without reading the answer choices) 151 a. No, never = 0 b. Yes, occasionally = 1 c. Yes, regularly = 2 d. Don’t know/Refuse to answer = 9999 11. Does anyone at home help you with your homework? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 12. Does anyone in your household know how to read? a. No = 0 (Skip to Question 13) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to Question 13) 13. Who in your household knows how to read? (Don’t prompt learners; let them answer without reading the answer choices; select all that apply; multiple answers possible) a. Brother b. Sister c. Grandmother d. Grandfather e. Uncle f. Aunt g. Cousin h. Mother i. Father j. Other, please specify__________________________ k. Don’t know/Refuse to answer 13. How do you feel about reading? a. Happy = 1 b. Neutral = 2 c. Unhappy = 3 d. Don’t know/Refuse to answer = 9999 14a. Do you ever take books home from school? a. No = 0 (Skip to QUESTION 15) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 15) • 14b. Do you read the books you take home from school? a. No = 0 (Skip to QUESTION 15) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 15) • 14c. Why do you not read the books you take home from school? 152 a. I don’t know how to read = 1 b. I don’t have electricity; so, I can’t see the books = 2 c. I don’t have time = 3 d. Other, please specify_______________________________________ = 4 meal information 15. Do you eat breakfast every day? a. No = 0 b. Yes = 1 (Skip to QUESTION 17) c. Don’ know/Refuse to answer = 9999 16. About how many days per week do you eat breakfast? (Don’t prompt learners; let them answer without reading the answer choices) a. Less than once per week = 1 b. One to two times per week = 2 c. Three to four times per week = 3 d. Five to six times per week = 4 e. Don’t know/Refuse to answer = 9999 17. Do you eat breakfast at home or at school? a. Home = 1 (Skip to QUESTION 19) b. School = 2 c. Both – home and school = 3 d. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 19) 18. What do you usually eat at breakfast? (Don’t prompt learners; let them answer without reading the answer choices; multiple responses possible; circle all that apply) a. Porridge = 1 b. Tea = 2 c. Nsima = 3 d. Sweet potatoes = 4 e. Fruit = 5 f. Other, please specify:____________________________ = 6 g. Don’t know/Refuse to answer = 9999 19. Do you eat lunch every day? a. No = 0 b. Yes = 1 (Skip to QUESTIONS 21) c. Don’t know/Refuse to answer = 9999 (Skip to QUESTIONS 21) 20. About how many days per week do you eat lunch? (Don’t prompt learners; let them answer without reading the answer choices) a. Less than once per week = 1 b. One to two times per week = 2 c. Three to four times per week = 3 153 d. Five to six times per week = 4 e. Don’t know/Refuse to answer = 9999 21. What do you usually eat for lunch? (Don’t prompt learners; let them answer without reading the answer choices; multiple responses possible; circle all that apply) a. Rice = 1 b. Nsima/rice and vegetables = 2 c. Sweet potatoes = 3 d. Nsima/rice and chicken = 4 e. Nsima/rice with beef/goat = 5 f. Nsima/rice with usipa 6 g. Other, please specify:___________________________ = 7 h. Don’t know/Refuse to answer = 9999 22. Do you eat lunch at home, bring lunch from home with you to school, or does the school give you lunch? i. Eat at home = 1 j. Bring lunch to school = 2 k. Eat lunch at school = 3 l. Don’t know/Refuse to answer = 9999 23. Are there some days when you don’t eat? a. No = 0 (Skip to QUESTION 25) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 25) 24. How many days in the past week did you not eat any food? (If the learner does not understand the question, you can ask “From this same day last week to today, how many days did you not eat any food?” Don’t prompt learners; let them answer without reading the answer choices) a. Once = 1 b. Twice = 2 c. Three times = 3 d. Four times = 4 e. Five times = 5 f. Six times = 6 g. Seven times = 7 h. Don’t know/Refuse to answer = 9999 25. How often do you feel hungry at school? (Don’t prompt learners; let them answer without reading the answer choices) a. Never = 0 b. Not very often = 1 c. A few times a week = 2 d. Every day = 3 e. Don’t know/Refuse to answer = 9999 26. Do you get tired at school? a. No = 0 (Skip to QUESTION 28) 154 b. Sometimes = 1 c. Yes = 2 d. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 28) 27. When are you most tired? a. When school starts = 1 b. In the middle of the school day = 2 c. When school is finished = 3 d. Don’t know/Refuse to answer = 9999 feelings about school 28. What do you like about coming to school? (Don’t read these options to the learner. If the learner is slow to respond, wait up to 8 seconds before asking “Are there things you like about coming to school? If so, what are they?” (The learner may not give these exact responses, but circle all those that are close to what he/she indicates. Select all that apply; multiple responses possible): a. Seeing my friends b. Learning new things c. Seeing my teacher d. School meals e. I like everything f. Other, please specify______________________________________________ g. I don’t like anything h. Don’t know/Refuse to answer 29. What do you not like about coming to school? (Don’t read these options to the learner. If the learner is slow to respond, wait up to 8 seconds before asking “Are there things you do not like about coming to school? If so, what are they?” (The learner may not give these exact responses, but circle all those that are close to what he/she indicates. Select all that apply; multiple responses possible): a. Other children are mean b. It’s boring c. I don’t understand the lessons d. The teacher is mean e. There’s no latrine or it’s too dirty f. I have to sit on the floor – no desk g. I can’t see the textbooks or don’t have textbooks h. I’m too tired i. I’m hungry j. It’s hard to pay attention k. I don’t feel well l. Other children fight too much m. I like everything n. Other, please specify______________________________________________ o. Don’t know/Refuse to answer 30. Do you feel happy or sad about coming to school? (Don’t prompt learners; let them answer without reading the answer choices) a. Happy = 1 155 b. Neutral = 2 c. Sad = 3 d. Don’t know/Refuse to answer = 9999 31. How would you describe your Chichewa teacher’s personality? (Don’t prompt learners; let them answer without reading the answer choices. If the learner answers does not match any of the choices, you can read the answer choices to the learner.) a. Nice/happy = 1 b. Neutral/neither happy nor unhappy = 2 c. Mean/unhappy = 3 d. Don’t know/Refuse to answer = 9999 32. How much do you think you learn at school? (Don’t prompt learners; let them answer without reading the answer choices) a. Not anything = 0 b. Not much = 1 c. Some = 2 d. A lot = 3 e. Don’t know/Refuse to answer = 9999 33. Do you think school is boring? (Don’t prompt learners; let them answer without reading the answer choices) a. No = 0 b. Sometimes = 1 c. Yes = 2 d. Don’t know/Refuse to answer = 9999 school environment 34. Do you feel comfortable using the latrine at school? a. No = 0 b. Yes = 1 (Skip to QUESTION 36) c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 36) 35. Why do you not feel comfortable using the latrine? (Don’t prompt learners; let them answer without reading the answer choices; select all that apply; multiple responses possible) a. It’s dirty b. It’s smelly c. I’m afraid other children/boys/girls will come in while I’m using it d. A snake (any animal/insect) may be in there e. Other, please specify:_____________________________________________ f. Don’t know/Refuse to answer 36. How long does it take you to walk to school? a. A short time (Less than 30 minutes) = 1 b. A medium amount of time (30 minutes to 1 hour) = 2 c. A long time (More than an hour) = 3 d. Don’t know/Refuse to answer = 9999 156 37. Do you ever get teased at school?: a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 38. Do you feel safe walking to school? a. No = 0 b. Yes = 1 (Skip to QUESTION 40) c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 40) 39. If you don’t feel safe walking to school, what kind of things make you feel unsafe? (Don’t prompt learners; let them answer without reading the answer choices; select all that apply; multiple responses possible) a. Animals b. Snakes c. Difficult-to-walk-on roads/paths (example – muddy, lots of rocks, many cars passing, etc.) d. Bad men or boys e. Other kids who are mean f. I’m afraid of getting lost g. Other, please specify:_____________________________________________ h. Don’t know/Refuse to answer 40. Do you ever get punished at school? a. No = 0 (Skip to END) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to END) 41. If yes, what do you get punished for? (Don’t prompt learners; let them answer without reading the answer choices; select all that apply; multiple responses possible) a. Making too much noise/talking b. Showing up late c. Fighting with other children d. Answering a question incorrectly e. Not paying attention f. Other, please specify:_____________________________________________ g. Don’t know/Refuse to answer 42. If yes, how do you get punished? ((Don’t prompt learners; let them answer without reading the answer choices; select all that apply; multiple responses possible) a. Send learner out of classroom b. Sweep or clean the classroom or school grounds c. Corporal punishment d. Kneel or stand on one leg for a long time e. Bring grass or reeds f. Stay after school and do school work g. Other (specify)__________________________________________________ h. Don’t know/Refuse to answer 157 Thank you for your participation! You have been very helpful! 158 ANNEX 13: TEACHER QUESTIONNAIRE Malawi EGRA Mid-Line Impact Evaluation Teacher Questionnaire April-May, 2015 The Malawi Ministry of Education, Science and Technology (MoEST) with funding from USAID are conducting a nationwide assessment of student reading ability in Standards 2 and 4. Your school was selected through a process of statistical sampling to take part in this study. We would like your help in this. But you do not have to take part if you do not want to, and you are free to opt out of any questions you do not feel comfortable answering. If you decide to take part, your name will not be mentioned anywhere in the survey data or report. The results of our analysis will be used by the Ministry of Education, Science and Technology to help identify additional support that is needed to help ensure that all children in Malawi become good readers. Additionally, your school will receive a report of the results that you can use to help you better address the needs of children in your school. If you agree to help with this study, please read the consent statement below, sign your name, and answer the questions I will ask you as completely and accurately as you can. It should take us no more than one hour. CONSENT STATEMENT: I understand and agree to participate in this reading research study by filling out this questionnaire as completely and accurately as possible. TEACHER SIGNATURE:_______________________________________________ Please answer all questions truthfully. Ask teacher to have attendance and progress record books for the entire year as well as the inventory book for the class with them for the interview. Date: Time Started: Time Ended: Enumerator Name: Survey and Logistics Manager Signature: Technical Manager Signature: School Name: School EMIS ID: Questionnaire ID: Location Type: Urban Rural Peri-Urban (circle one) Type of School: Coed All Boys All Girls (circle one) Instructions: The enumerator should read each of the questions to the teacher as is. He/she can also read the response choices (unless the question specifies that teachers should not be prompted). Once the teacher has selected an option, the letter associated with that option should be circled. Most questions should have only one response. In some cases, a question will specify that multiple responses are allowed. In those cases, the enumerator should circle the letters corresponding with all response options that apply. All regular text can be read to the respondents, whereas all italic text is meant for the enumerator clarification only. GENERAL BACKGROUND INFORMATION 1. Division:_________________________________________________________ 2. District: _________________________________________________________ 159 3. Zone: ___________________________________________________________ 4. Teacher’s Name: __________________________________________________ 5. Class level: a. Standard 2 = 2 b. Standard 4 = 4 TEACHER BACKGROUND INFORMATION 6. How old did you turn on your last birthday (Don’t know/Refuse to answer = 9999):_______ 7. How many years have you been teaching? (Don’t know/Refuse to answer = 9999):_______ 8. How many years have you been teaching in this school? (Don’t know/Refuse to answer = 9999):_______ 9. What is your highest academic qualification? (Don’t prompt) a. JCE = 1 b. MSCE = 2 c. Diploma = 3 d. Other (specify:__________________) = 4 e. Don’t know/Refuse to answer = 9999 10. Are you a trained teacher? a. No = 0 (Skip to QUESTION 12) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to question 12) 11. How many years have you been teaching as a trained teacher? (Don’t know/Refuse to answer = 9999):_______(Skip to QUESTION 13 after this question) 12. If you are not a trained teacher, what is your teaching status? (Don’t prompt unless the teacher does not understand the question; then you can list) a. Voluntary teacher = 1 b. Learner teacher = 2 c. Teaching assistant = 3 d. Other, please specify_____________________ = 4 Don’t know/Refuse to answer = 9999 CLASS BACKGROUND INFORMATION 13. How many times each week do you use each of the following methods to measure/assess your learners’ reading progress? (Don’t know/Refuse to answer = 9999) (Enter 0-5 for each) a. Written evaluations:________ b. Individual learner oral evaluations:________ c. Whole class oral evaluations:________ d. Small group oral evaluations:________ 160 e. Checking learners’ exercise books:________ f. Checking learners’ homework:________ g. Other methods (please describe):________________________ #:________ 14a. In this school, what are the most important things that prevent learners from learning? (Don’t prompt; circle all that apply; multiple responses possible): h. Classes too large i. Learners don’t have textbooks j. There’s not enough time in the school day k. Learners don’t understand the language of instruction l. There are too many subjects in the curriculum for the time available m. Teachers don’t have access to the teaching materials they need n. There are too many languages for learners to learn at one time o. Learners shouldn’t have to learn English so early p. Learners don’t attend school regularly q. Teachers don’t have enough training r. Teachers don’t understand English enough to be able to teach it s. Learners do not have enough to eat t. Learners are taking care of younger siblings or helping parents with work u. The distance to school is too far for children to travel v. The school is lacking in other resources, please list________________ w. Other, please specify:______________________________________ x. Don’t know/Refuse to answer 14b. What percent of reading instruction in Standards 1-4 is in the local familiar language (if something other than Chichewa)? (Don’t know/Refuse to answer = 9999): a. Standard 1:__________ b. Standard 2:__________ c. Standard 3:__________ d. Standard 4:__________ SCHOOL RESOURCES 15. Does your school or classroom have a library? a. No = 0 (Skip to QUESTION 18) b. Yes, a classroom library = 1 (check to see if you see books there, and ask a follow up question if not) c. Yes, a school library = 2 (Skip to QUESTION 17) d. Yes, both classroom and school libraries e. Don’t know/Refuse to answer = 9999(Skip to QUESTION 18) 16. How often do your learners use the classroom library? (Don’t prompt) a. Every day = 1 b. Every other day = 2 c. Three – Four times a week = 3 d. Once a week = 4 e. Once or twice a month = 5 f. Only occasionally = 6 161 g. Never = 7 h. Don’t know/Refuse to answer = 9999 17. How often do your learners use the school library? (Don’t prompt) a. Every day = 1 b. Every other day = 2 c. Three – Four times a week = 3 d. Once a week = 4 e. Once or twice a month = 5 f. Only occasionally = 6 g. Never = 7 h. Don’t know/Refuse to answer = 9999 18. Excluding textbooks, do you have sufficient teaching and learning resources (TALULAR)? a. No = 0 b. Yes = 1 (check the room for them, and ask follow-up question if you don’t see any) c. Don’t know/Refuse to answer = 9999 19. How many reading textbooks do you have for your class? (Don’t know/Refuse to answer = 9999): a. English = _______ (count them to verify, if possible) b. Chichewa = _______ (count them to verify, if possible) 20. How many reading textbooks do you hand out to learners? (Don’t know/Refuse to answer = 9999): a. English = _______ (count them to verify, if possible) b. Chichewa = _______ (count them to verify, if possible) (if both numbers match those in 19, skip to QUESTION 21b) 21a. Why do you not hand them all out? (Don’t prompt) c. There are not enough for each learner to have one. d. Learners do not take good care of the books/destroy them e. Learners tend to lose the books f. Other, please specify______________________________________________ g. Don’t know/Refuse to answer = 9999 21b. Do learners from your class ever take textbooks or library books home from school? h. No = 0 i. Yes = 1 j. Don’t know/Refuse to answer = 9999 22. In what ways do the staff in your school work together to identify strategies for increasing learner success in learning in school?_____________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ____________ 162 COMMUNITY INVOLVEMENT IN THE SCHOOL 23. Does your school have a functioning Parent Teacher Association? a. No = 0 (Skip to QUESTION 28) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 28) 24. How often does it meet? (Don’t prompt) a. On an “as-needed” basis = 1 b. Weekly = 2 c. Twice per month = 3 d. Monthly = 4 e. Every other month = 5 f. Quarterly (once per term) = 6 g. Twice per year = 7 h. Annually = 8 i. Less than once/year = 9 j. Don’t know/Refuse to answer = 9999 25. What sorts of activities does the PTA do to support the school? (Don’t prompt; multiple responses possible) a. Manage/help with construction of school buildings b. Help, manage, or fundraise to construct teacher houses c. Dig wells/toilets or manage this process d. Donate materials and resources for construction e. Cook f. Fundraise g. Volunteer at schools; please specify in what way(s)__________________ h. Mobilize the community to be more involved in the school i. Encourage parental participation in their learner’s education j. Discuss/implement ways of reducing absenteeism k. Discuss/implement ways of reducing dropouts l. School maintenance m. Other(s), please specify_______________________________________ n. Don’t know/Refuse to answer 26. Do you have meetings with groups of parents of your learners (outside of PTA meetings)? a. No = 0 (Skip to QUESTION 28) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 28) 27. How often do you have meetings with groups of your learners’ parents (outside of PTA meetings)? (Don’t prompt) a. Once per school year =1 b. Twice per school year =2 c. Three times per school year = 3 d. Four or more times per school year = 4 163 e. Don’t know/Refuse to answer = 9999 28. Do you ever invite parents to participate in their learners’ classrooms or become engaged in extra￾curricular activities? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 29. In what other ways, if any, does the community (including local individuals and businesses) get involved with your school? a. A – Way (no codes, just list) B - When did involvement begin (year) C - Has this support helped the school (No = 0, Yes = 1, Don’t know = 9999) D - If so, in what ways (see below list for codes) 1 - 2 - 3 - Codes for 29D: It didn’t benefit the school at all = 0 Better facilities = 1 More resources for teachers = 2 More resources for learners = 3 More motivation on the part of staff = 4 More motivation on the part of learners = 5 Better quality teaching = 6 Longer school day = 7 Learners are able to read better = 8 Learner are able to learn better in other learning areas = 9 Learners are getting better scores on their tests = 10 Better or more regular attendance = 11 Other, please list__________________________________ = 12 Don’t know/Refuse to answer = 9999 SUPPORT FROM OUTSIDE ORGANIZATIONS 30. Has your school received support from the EGRA Project? a. No = 0 (Skip to QUESTION 33) b. Yes = 1 c. Don’t know/Refuse to answer = 9999(Skip to QUESTION 33) 31a. What types of support has the school received from the EGRA Project? (Read through each option and mark with a “Yes,” “No,” or “9999” for Don’t know/Refuse to answer; multiple responses possible): d. Have you received more textbooks for use in class?:_______ e. Do learners have textbooks to take home now?:_______ 164 f. Have you received sample lesson plans or help with your lesson plans:_______ g. Has EGRA helped to get more parents involved in school?:_______ h. Has EGRA extended the length of your school day?:_______ i. Has EGRA provided you with training?:_______ j. Has EGRA provided other teachers in your school with training?:_______ k. Has EGRA provided you with coaching?:_______ l. Has EGRA provided any other support?, please m. specify______________________ 31b. Has EGRA been sending you text (SMS) messages? a. No = 0 (Skip to QUESTION 32) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 • 31c. When did you start receiving the SMS messages? a. In the last month = 1 b. 2-3 months ago = 2 c. 4-6 months ago = 3 d. t type o respond = 9999text (SMS) messages? e. still compare with baseline. as optionsable?hink they have enough resources7-12 months ago = 4 f. More than a year ago = 5 g. Don’t know/Refuse to answer = 9999 • 31d. How often do your receive the SMS messages? a. At least once per day = 1 b. Once every few days = 2 c. Once a week = 3 d. A few times a month = 4 e. Once a month = 5 f. Less than once a month = 6 g. Don’t know/Refuse to respond = 9999 • 31e. What is the topic of the SMS messages? (Read choices; select all that apply; multiple responses possible): a. School management information b. Updates on teaching practices c. Information on how to get parents involved in school d. Other, please specify_____________ 31. What effect(s) has the EGRA Project had on your school? (Don’t prompt; circle all that apply; multiple responses possible): a. It didn’t benefit the school at all b. Better facilities c. More resources for teachers d. More resources for learners e. More motivation on the part of staff f. More motivation on the part of learners 165 g. Better quality teaching h. Longer school day i. Learners are able to read better j. Learner are able to learn better in other learning areas k. Learners are getting better scores on their tests l. Better or more regular attendance m. Other, please list__________________________________ n. Don’t know/Refuse to answer = 9999 32. Are there any other donor or nonprofit organizations involved in providing any kind of support/training/assistance to the school? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 INSERVICE TRAINING/PROFESSIONAL DEVELOPMENT 33. How many days of any type of in-service training or professional development have you attended during the last three (3) years? (Don’t know/Refuse to answer = 9999):_______(If 0, skip to QUESTION 40) 34. How many days of MTPDS in-service training or professional development in teaching reading have you attended during the last three (3) years? (Don’t know/Refuse to answer = 9999):_______ 35. How many days of EGRA in-service training or professional development in teaching reading have you attended during the last three (3) years? (Don’t know/Refuse to answer = 9999):_______ (If 0, skip to QUESTION 39) 36. What were the most useful aspects of the EGRA reading trainings? (Don’t know/Refuse to answer = 9999): __________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 37. What were the least useful aspects of the EGRA reading trainings? (Don’t know/Refuse to answer = 9999): __________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 38. How many days of in-service training or professional development in another method of teaching reading have you attended during the last three (3) years? (Don’t know/Refuse to answer = 9999):_______ 166 CLASSROOM-BASED COACHING 39. Of the following list of possible supervision and/or coaching providers, please indicate the approximate number of hours each provider supervised/coached you in the past three (3) years and the last full term and then rate each coaching provider on a scale of 1-5 with 1 being least useful and 5 being most useful. (Don’t know/Refuse to answer = 9999) (Codes: 0 = doesn’t apply, 1=hurtful or discouraging, 2=not helpful, 3=somewhat helpful, 4=helpful, 5=very helpful): Coaching provider A - Approximate number of hours in past 3 years B – Approximate number of hours in the last full term C - Rating 1-5 1 - Head teacher 2 - MoEST inspector 3 - PEAs 4 - Divisional inspector 5 - MTPDS staff 6 – EGRA staff 7 – Teacher Training College Staff 8– Mentor Teacher 9 – Other, please specify _____________________ 40. What were the most useful aspects of the coaching sessions? (Don’t know/Refuse to answer = 9999):________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ __________________________________________________________________________________ ______________________________________________________ 41. What were the least useful aspects of the coaching sessions? (Don’t know/Refuse to answer = 9999):________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 42. Was this training enough for you to be able to use these methods correctly in your classroom? (Don’t prompt) a. No = 0 b. Somewhat = 1 167 c. Yes = 2 d. Don’t know/Refuse to answer = 9999 43. Do you feel you need more training? a. No = 0 (Skip to END) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 46 ) 44. In which topics would you like to receive more training? (Don’t know/Refuse to answer = 9999):______________________________________________________ __________________________________________________________________________ ____________________________________________________________ ___________________________________________________________________ QUESTIONS THAT MAY REQUIRE SOME RESEARCH 45. How many learners are enrolled in your class? (Don’t know/Refuse to answer = 9999):________ 46. How many are girls? (Don’t know/Refuse to answer = 9999):________ 47. Is the teacher regularly maintaining an attendance register? (Look at his/her attendance register). a. No = 0 (Skip to QUESTION 50) b. Yes = 1 48. If the teacher is maintaining an attendance register, make one “X” per column below for the number of absences during the Wednesday of the third week of the school year, the third week of January, and the most recent week (For any instances where numbers are not available, write – 9999, and if the school was closed on one of those Wednesdays, go to Thursday in that same week): Approximate number of absences A – Wednesday of the Third Week of the School Year B – Wednesday of the Third Week of January C – Wednesday of the most recent full week 0 = 0 1 = 1 – 15 2 = 16 – 30 3 = 31 – 50 4 = 51 – 75 49. Is the teacher regularly maintaining a progress record book? (Look at his/her grade book). a. No = 0 b. Yes = 1 168 50. How many learners have stopped attending or dropped out of your class during this academic year (since the third week of school)? (if necessary, count the number of learners at week 3 and count the number that are recorded as somewhat regular attendance during the past two weeks, and subtract)(Don’t know/Refuse to answer and No attendance records available = 9999):________ (If 0, Skip to QUESTION 54). 51. About how many of the learners who have stopped attending or dropped out have moved or transferred to another school during this academic year? (Don’t know/Refuse to answer = 9999):__________ 52. Why do you think learners drop out in your class? (Do not prompt; multiple responses possible; select all that apply) a. Lack of parental encouragement/support to attend school b. Need for the learner to earn money/sell things/work outside of the home c. Need for the learner to work inside of the home (caring for younger siblings or elderly family members or doing other chores d. Need for the learner to work on the family farm or tending to family animals e. Learners live too far away from the school f. Learners don’t do well in school or repeat grades often g. Learners get married young h. Learners become pregnant (teen pregnancy) i. Learners come from poor families and have insufficient food or resources j. Learners are not interested in school k. Learners do not have good role models showing them the value of education l. Learners become ill m. Learners move/migrate n. Learners lose their parents/become orphans o. Other(s), please specify___________________________________________ p. Don’t know/Refuse to answer = 9999 53. How many of your learners are repeating this standard? (Response of 0-100 should be recorded directly. For values more than 100, enter 101; enter 9999 only for Don’t know/Refuse to answer):_______ (If 0, Skip to END) 54. How many of your learners have been in this standard level for more than two years? (Any response of 0-100 should be recorded directly. For values more than 100, enter 101; enter 9999 only for Don’t know/Refuse to answer):_______ 55. What do you think are the main reasons learners in your class have had to repeat a standard? (Don’t prompt; select all that apply; multiple responses possible) a. They don’t study b. They don’t have textbooks c. There are too many learners in the class d. They don’t pay attention e. There isn’t enough time in the school day f. I can’t effectively teach this many learners g. Some of the learners are too young h. They can’t study at home because there is no electricity i. They can’t study at home because they don’t have materials to take home j. Other, please specify:_________________________________________ 169 k. Don’t know/Refuse to answer 170 ANNEX 14: HEAD TEACHER QUESTIONNAIRE Malawi EGRA Impact Evaluation Mid-Line Assessment Head Teacher Questionnaire May 2015 The Malawi Ministry of Education, Science and Technology (MoEST) with funding from USAID are conducting an impact evaluation of student reading ability in Standards 2 and 4. Your school was selected through a process of statistical sampling to take part in this study. We would like your help in this. But you do not have to take part if you do not want to, and you are free to opt out of any questions you do not feel comfortable answering. If you decide to take part, your name will not be mentioned anywhere in the survey data or report. The results of our analysis will be used by the Ministry of Education, Science and Technology to help identify additional support that is needed to help ensure that all children in Malawi become good readers. Additionally, your school will receive a report of the results that you can use to help you better address the needs of children in your school. This interview will take approximately one hour to complete. If you agree to help with this study, please read the consent statement below, sign on the line, and answer the questions I will ask you as completely and accurately as you can. CONSENT STATEMENT: I understand and agree to participate in this reading research study by filling out this questionnaire as completely and accurately as possible. HEAD TEACHER SIGNATURE:______________________________________________________ Please answer all questions truthfully. Date: Time Started: Time Ended: Enumerator Name: Survey and Logistics Manager Signature: Technical Manager Signature: School Name: EMIS ID: Questionnaire ID: Division: District: Zone: Location Type: Urban Rural Peri-Urban (circle one) Type of School: Coed All Boys All Girls (circle one) Designation of School: Junior Primary Full Primary (circle one) Instructions: The enumerator should read each of the questions to the head teacher as is. He/she can also read the response choices (unless the question specifies that the head teacher should not be prompted). Once the head teacher has selected an option, the letter associated with that option should be circled. Most questions should have only one response. However, in some cases, a question will specify that multiple responses are allowed. In those cases, the enumerator should circle the letters corresponding with all response options that apply. All regular text can be read to the respondents, and all italic text includes instructions to the enumerator. respondent background • 1a. Respondent name:______________________________________ • 171 1b. Respondent age:________________________________________ • 2. What is your position at this school? b. Head Teacher (HT) = 1 (Skip to QUESTION 3) c. Deputy Head Teacher (DHT) = 2 d. Other, please specify_______________________________ = 3 • 2a. Is the Head Teacher male or female? e. Male = 1 f. Female = 2 • 2b. What is the sex of the person being interviewed (observe, do not ask) g. Male = 1 h. Female = 2 • 5. How many years have you been in this position (as HT or DHT)? (Don’t know/Refuse to answer = 9999):______ (please write the number of years) • 6. How many years have you been in this position at this school? (Don’t know/Refuse to answer = 9999):______ (please write the number of years) 7. What is your highest academic qualification? (Do not prompt; select the answer that matches the response provided) a. JCE = 1 b. MSCE = 2 c. Diploma = 3 d. Degree = 4 e. Other, please specify:__________________ = 5 f. Don’t know/Refuse to answer = 9999 8. Are you a trained teacher? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 SCHOOL background 9. What is the length of the school day for each of the following standards? (Don’t know/Refuse to • answer = 9999) (List in hours and minutes; example – 2½ hours = 2 hours 30 minutes): a. Standard 1:_______Hours ________Minutes b. Standard 2:_______Hours ________Minutes c. Standard 3:_______Hours ________Minutes d. Standard 4:_______Hours ________Minutes • 8a. Does this school operate on shifts? 172 a. No = 0 (Skip to QUESTION 10) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 10) • 8b. Which standards are offered during shift one? (multiple responses possible) a. Standard 1 b. Standard 2 c. Standard 3 d. Standard 4 e. None f. Don’t know/Refuse to answer • 9. Which standards are offered during shift two? (multiple responses possible) e. Standard 1 f. Standard 2 g. Standard 3 h. Standard 4 i. None j. Don’t know/Refuse to answer • 10. How many classes are there at this school for each of the following standards? (Don’t know/Refuse to answer = 9999): a. Standard 1:__________________ b. Standard 2:__________________ c. Standard 3:__________________ d. Standard 4:__________________ 11. In which standards, if any, does your school teach in English? (Select all that apply; multiple responses possible): a. We don’t teach English in Standards 1-4 (Skip to QUESTION 13) b. Standard 1 c. Standard 2 d. Standard 3 e. Standard 4 f. Don’t know/Refuse to answer • 12. Does your school teach pupils how to read in English in any of the following standards? (Select all that apply; multiple responses possible): a. We don’t teach pupils to read in English in Standards 1-4 b. Standard 1 c. Standard 2 d. Standard 3 e. Standard 4 f. Don’t know/Refuse to answer • 13. Does your school teach students how to read in Chichewa in the following standards? (Select all that apply; multiple responses possible): a. We don’t teach pupils to read in Chichewa in Standards 1-4 173 b. Standard 1 c. Standard 2 d. Standard 3 e. Standard 4 f. Don’t know/Refuse to answer • • RESOURCES 14. Do all of your pupils have the prescribed number of textbooks? b. No = 0 c. Yes = 1 (Skip to QUESTION 16) d. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 16) • 15. Why not? (Do not prompt; select all that apply; multiple responses possible). a. The ministry did not provide more textbooks b. The donor organization did not provide enough textbooks c. We have more textbooks, but they are in too poor of condition to hand out d. We don’t like to hand out all textbooks because we want to keep some in good condition e. Other, please specify____________________________________________________ f. Don’t know/Refuse to answer 16. Has your school received textbooks or materials in the local familiar language? a. No = 0 (Skip to QUESTION 18) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 18) • 17. Who provided/provides pupils with textbooks in the local familiar language? (Do not prompt; select all that apply; multiple responses possible). a. MoEST = 1 b. MTPDS = 2 c. EGRA = 3 d. Read Malawi = 4 e. UNICEF = 5 f. Other, please specify____________________________ = 6 g. Don’t know/Refuse to answer = 9999 • 18. Is there a clean, safe water supply available on school premises? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 • 19. Does the school have electricity (Grid or Solar)? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 • 20. Does your school have a school feeding program? a. No = 0 (Skip to QUESTION 24) 174 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 24) • 21. If yes, what time does the feeding occur in the school day? a. Before school starts = 0 b. In the middle of the day = 1 c. After school = 2 d. Don’t know/Refuse to answer = 9999 • 22. Is school feeding offered every school day? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 • • 23. How long has the school been participating in the school feeding program? (Do not prompt) a. Less than one year = 0 b. One year = 1 c. Two years = 2 d. Three years = 3 e. Four years = 4 f. Five years = 5 g. More than five years = 6 h. Don’t know/Refuse to answer = 9999 • TEACHER INFORMATION 24. How many Standard 1-Standard 4 teachers are there at this school? (Don’t know/Refuse to answer = 9999):_______ 25. How many of the Standard 1-Standard 4 teachers at this school are trained? (Don’t know/Refuse to answer = 9999):_______ • 26. How many Standard 1-4 teachers from this school have participated in an EGRA training on how to teach reading since 2013? (Don’t know/Refuse to answer = 9999):_______ (If the answer if “0,” Skip to QUESTION 31) 27. Among those who participated in this training, on average, how many EGRA trainings has each of the Standard 1-Standard 4 teachers participated in the past two years? (Don’t know/Refuse to answer = 9999):_______ 28. How many of the Standard 1-Standard 4 teachers are using the EGRA methods in their teaching? (Don’t know/Refuse to answer = 9999):_______ (If the answer if “0,” Skip to QUESTION 31) • 29. How many of the Standard 1-Standard 4 teachers do you think feel confident about using the EGRA teaching methods? (Don’t know/Refuse to answer = 9999):_______ 175 30. How many of the Standard 1-Standard 4 teachers do you think need additional training on applying early grade reading methods in the classroom? (Don’t know/Refuse to answer = 9999):______ 31. How many of the Standard 1-Standard 4 teachers have participated in training in another approach to teaching reading? (Don’t know/Refuse to answer = 9999):_______ (If the answer if “0,” Skip to QUESTION 38) • 32. Which organization(s) organized these trainings? (Do not prompt; select all that apply; multiple responses possible): a. DTED b. MIE c. Read Malawi d. UNICEF e. World Vision (NASFEM) f. MTPDS g. Plan Malawi h. Tikwere i. Save the Children j. SIG (Ministry of Education Program) k. Other, please specify____________________________ l. Don’t know/Refuse to answer 33. Among those who have participated in such trainings, on average, how many non-EGRA reading trainings has each of the Standard 1-Standard 4 teachers participated in during the past two years? (Don’t know/Refuse to answer = 9999):_______ 34. How many of the Standard 1-Standard 4 teachers are using these other methods of teaching reading in their classrooms? (Don’t know/Refuse to answer = 9999):_______ 35. How many Standard 1 to Standard 4 teachers were absent yesterday (or on the last school day)? (Don’t know/Refuse to answer):_______ • 36. How many Standard 1 to Standard 4 teachers often arrive late or after the start of classes? (Don’t know/Refuse to answer = 9999):_______ 37. Do you maintain records of teacher absences? (If yes, ask to see them and provide an estimate of the numbers of absences for all teachers in Standard 1-Standard 4 for the entire year). (If no, mark with an 8888; Don’t know/Refuse to answer = 9999):_______ • 38. How often do you or someone else from your school review teacher lesson plans? (Do not prompt) a. Never = 0 b. Once per year = 1 c. Once every 2-3 months = 2 d. Once per month = 3 e. Once every two weeks = 4 f. Every week = 5 g. Once a day = 6 h. Don’t know/Refuse to answer = 9999 • 176 39. In a term, how many times are teachers provided with supervision or coaching in their classrooms by someone in this school? (Do not prompt) a. Never = 0 b. One time = 1 c. Two times = 2 d. Three times = 3 e. Four or more times = 4 f. Other, please specify_____________________________________ = 5 g. Don’t know/Refuse to answer = 9999 • INFORMATION ON PUPILS 40. Rank the three primary reasons, not including transfers, in this school for the Standard 2 dropouts? (Do not prompt; mark the greatest reason with a 1, the second greatest with a 2, and the third greatest with a 3.Leave all other reasons blank after answer first three.): a. Limited availability of teachers:_______ b. Employment/helping with family work:_______ c. Taking care of siblings or other relatives:_______ d. Fees:_______ e. Long distances travel:_______ f. Marriage:_______ g. Poor school facilities:_______ h. Pregnancy:_______ i. Sickness or injury:_______ j. Violence:_______ k. Not motivated/Don’t see importance of education:_______ l. Difficultly understanding the curriculum/Poor performance:_______ m. Other, please list____________________________ :_______ n. Don’t know/Refuse to answer (Write 9999 if selected):_______ • 41. Rank the three primary reasons, not including transfers, in this school for the Standard 4 dropouts? (Do not prompt; mark the greatest reason with a 1, the second greatest with a 2, and the third greatest with a 3): a. Limited availability of teachers:_______ b. Employment/helping with family work:_______ c. Taking care of siblings or other relatives:_______ d. Fees:_______ e. Long distances travel:_______ f. Marriage:_______ g. Poor school facilities:_______ h. Pregnancy:_______ i. Sickness:_______ j. Violence or Injury:_______ k. Not motivated/Don’t see importance of education:_______ l. Difficultly understanding the curriculum/Poor performance:_______ m. Other, please list____________________________ :_______ n. Don’t know/Refuse to answer (Write 9999 if selected):_______ • 177 42. Are dropout rates higher or lower for boys or girls? a. Higher for girls = 1 (Explain in 43) b. Higher for boys = 2(Explain in 43) c. About the same for both sexes = 3 (Skip to QUESTION 44) d. It varies by standard level = 4 (Explain in 43) e. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 44) • 43. Why do dropout rates vary by sex or standard level? __________________________________ • ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 44a. What, if anything has been done (by you, as the head teacher or deputy head teacher, the school as a whole, the Parent-Teacher Association, and the Community) to reduce dropouts at your school?_____________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 44b.What else would you like to be doing to reduce dropouts in your school if the resources were available?____________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 45. What is the average repetition rate (percent) for pupils in the following standards? (Don’t know/Refuse to answer = 9999): a. Standard 1:__________ b. Standard 2:__________ c. Standard 3:__________ d. Standard 4:__________ • 45a. What is the main reason for pupils' repetition in Standard 2? (Do not prompt) a. They don’t study = 1 b. They don’t have textbooks = 2 c. There are too many pupils in the class = 3 d. They don’t pay attention = 4 e. There isn’t enough time in the school day = 5 f. I can’t effectively teach this many pupils = 6 g. Some of the pupils are too young = 7 h. They can’t study at home because there is no electricity = 8 i. They can’t study at home because they don’t have any materials to take home = 9 j. Other, please specify___________________________________________ = 10 k. Don’t know/Refuse to answer = 9999 45b. What is the main reason for pupils' repetition in Standard 4? (Do not prompt) a. They don’t study = 1 b. They don’t have textbooks = 2 c. There are too many pupils in the class = 3 d. They don’t pay attention = 4 e. There isn’t enough time in the school day = 5 178 f. I can’t effectively teach this many pupils = 6 g. Some of the pupils are too young = 7 h. They can’t study at home because there is no electricity = 8 i. They can’t study at home because they don’t have any materials to take home = 9 j. Other, please specify___________________________________________ = 10 k. Don’t know/Refuse to answer = 9999 • 45c.What, if anything has been done (by you, as the head teacher or deputy head teacher, the school as a whole, the Parent-Teacher Association, and the Community) to reduce repetition at your school?_____________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 45d.What else would you like to be doing to reduce repetition in your school if the resources were available?____________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 46. Are boys or girls more likely to repeat a standard? e. Boys are more likely to repeat a standard = 1 Why?_____________________________ f. Girls are more likely to repeat a standard = 2 Why?_____________________________ g. They are equally likely to repeat a standard = 3 h. It varies by standard level = 4, Explain_______________________________________ i. Don’t know/Refuse to answer = 9999 • 47. How many pupils with disabilities are there in the school? (Don’t know/Refuse to answer = 9999):_______ • 48. How, if at all, does the school cater to pupils with disabilities? (Don’t know/Refuse to answer = 9999):______________________________________________________________________ • ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ COMMUNITY INVOLVEMENT IN THE SCHOOL • 49. Does the school have a PTA? a. No = 0 (Skip to QUESTION 53) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 53) • 50. When did it begin operating? (Enter year) (Don’t know/Refuse to answer = 9999):_____ (If before 1990, choose 1990) 51. How often did the PTA meet in this academic year? (Do not prompt unless the Head Teacher is struggling with understanding the questions. Then, it is okay to list the answer choices). 179 a. Never = 0 b. Once a year = 1 c. Twice per year - 2 d. Once every 2-3 months = 3 e. Once a month = 4 f. Once a week = 5 g. Don't know/Refuse to answer = 9999 • 52. For which of the following does the PTA have decision making authority and/or responsibility? (Read each answer choice; select all that apply; multiple responses possible): a. School management b. Pupil learning challenges and solutions c. Curriculum d. Physical school improvement efforts e. Maintenance of infrastructure/equipment f. Financial issues/fund raising g. Procurement and/or distribution of textbooks h. Reading instruction in after-school programming i. Other, please specify____________________________________________________ j. Don’t know/Refuse to answer • 53. Does the school have a school management committee (SMC)? a. No = 0 (Skip to QUESTION 57) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 57) • 54. When did it begin operating? (Enter year) (Don’t know/Refuse to answer = 9999):_____ (If before 1990, choose 1990) 55. How often did the school management committee meet in this academic year? (Do not prompt) a. Never = 0 b. Once a year = 1 c. Twice per year - 2 d. Once every 2-3 months = 3 e. Once a month = 4 f. Once a week = 5 g. Don't know/Refuse to answer = 9999 • 56. For which of the following does the school management committee have decision making authority and/or responsibility? (Read each answer choice; select all that apply; multiple responses possible): a. School management b. Pupil learning challenges and solutions c. Curriculum d. Physical school improvement efforts e. Maintenance of infrastructure/equipment f. Financial issues/fund raising g. Procurement and/or distribution of textbooks h. Don’t know/Refuse to answer • 180 57. Do you ever invite parents to participate in their pupils’ classrooms or become engaged in extra￾curricular activities? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 58. Other than the PTA, school management committee, and parents, is the community (individuals, organizations, or businesses) involved in supporting the school and pupil learning? a. No = 0 (Skip to QUESTION 61) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 61) • 59. In what other ways, if any, does the community (including local individuals and businesses) get involved with your school? (Do not prompt; just select all those that apply) z. A – Way (see below list for codes; list only one code per box) B - When did involvement begin (year) C - Has this support helped the school (No = 0, Yes = 1, Don’t know = 9999) D - If so, in what ways (see below list for codes; multiple selections possible) 1 2 3 Codes for 59A: a. Helping with construction (i.e. molding bricks, constructing buildings) = 1 b. Digging wells/toilets = 2 c. Donating materials and resources for construction = 3 d. Cooking = 4 e. Fundraising = 5 f. Volunteering at schools; please specify in what way(s)___________ = 6 g. Other, please list in space above = 7 h. Don’t know/Refuse to answer = 9999 Codes for 59D: It didn’t benefit the school at all = 0 Better facilities = 1 More resources for teachers = 2 181 More resources for learners = 3 More motivation on the part of staff = 4 More motivation on the part of learners = 5 Better quality teaching = 6 Longer school day = 7 Learners are able to read better = 8 Learner are able to learn better in other learning areas = 9 Learners are getting better scores on their tests = 10 Better or more regular attendance = 11 Other, please list in space above = 12 Don’t know/Refuse to answer = 9999 60. Has community involvement increased or decreased over the past three years? a. It has decreased = 1 b. It has increased = 2 c. It has stayed the same = 3 d. Don’t know/Refuse to answer = 9999 • SUPPORT FROM OUTSIDE ORGANIZATIONS 61. Has your school received support from EGRA? a. No = 0 (Skip to QUESTION 63) b. Yes = 1 c. Don’t know/Refuse to respond = 9999 (Skip to QUESTION 63) • 61a. What types of support has the school received from the EGRA Project? (Do not prompt; select all that apply; multiple responses possible): b. We have received more textbooks for use in class c. Our pupils have textbooks to take home now d. We have received sample lesson plans or help with our lesson plans e. EGRA helped to get more parents involved in school f. EGRA extended the length of our school day g. EGRA provided me with training h. EGRA provided other teachers in my school with training i. EGRA provided me with coaching j. Other, please specify_________________________________________________ k. Don’t know/Refuse to answer • 61b. Has EGRA been sending your text (SMS) messages? a. No = 0 (Skip to QUESTION 62) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 182 • 61c. When did you start receiving the SMS messages? h. In the last month = 1 i. 2-3 months ago = 2 j. 4-6 months ago = 3 k. t type o respond = 9999text (SMS) messages? l. still compare with baseline. as optionsable?hink they have enough resources7-12 months ago = 4 m. More than a year ago = 5 n. Don’t know/Refuse to answer = 9999 • 61d. How often do your receive the SMS messages? a. At least once per day = 1 b. Once every few days = 2 c. Once a week = 3 d. A few times a month = 4 e. Once a month = 5 f. Less than once a month = 6 g. Don’t know/Refuse to respond = 9999 • 61e. What is the topic of the SMS messages? (Read choices; select all that apply; multiple responses possible): a. School management information b. Updates on teaching practices c. Information on how to get parents involved in school d. Other, please specify_______________________________________________ _____ 63. What effect has the EGRA Project had on your school? (Do not prompt; select all that apply; multiple responses possible): a. It didn’t benefit the school at all b. Better facilities c. More resources for teachers d. More resources for pupils e. More motivation on the part of staff f. More motivation on the part of pupils g. Better quality teaching h. Longer school day i. Students are able to read better j. Student are able to learn better in other learning areas k. Students are getting better scores on their tests l. Better or more regular attendance m. Other, please list____________________________________________________ n. Don’t know/Refuse to answer 64. Has either the MTPDS Project, the EGRA Project, or another organization worked to add an 183 hour to your school day for some Standards? a. Yes, the MTPDS Project added an hour = 1 b. Yes, the EGRA Project added an hour = 2 c. Yes, another organization or project added an hour = 3 d. Yes, we have added an hour for other reasons (please specify those reasons__________ _______________________________________________________________________) e. No, our school day has not been extended = 4 (Skip to QUESTION 64) f. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 64) • 63a. For which Standards has the school day been extended by an hour? (Don’t prompt; select all that apply; multiple answers possible; Don’t know/Refuse to answer = 9999) a. Standard 1 b. Standard 2 c. Standard 3 d. Standard 4 e. Standard 5 f. Standard 6 g. Standard 7 h. Standard 8 i. Don’t know/Refuse to answer • 63b. How many days per week does the school day last an extra hour? a. One = 1 b. Two = 2 c. Three = 3 d. Four = 4 e. Five = 5 f. It varies by standard level = 6 g. Don’t know/Refuse to answer = 9999 65. Are there any other individuals, organizations, or businesses that are involved in providing any kind of support/training/assistance to the school? Please include support or training received from Airtel, World Vision, UNICEF, FAWEMA, World Bank, and any other organizations. a. No = 0 (Skip to QUESTION 69) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 69) 66. Which other donor or nonprofit organizations are these, when did they begin providing support for this school, what type of support are they providing, has the support helped, and if so, in what ways (Do not read options; just mark those that the respondents lists; multiple responses possible): A – Donor or Nonprofit Organization B - Year Support Began C - Type of Support D – Has this support helped the school (No = 0, Yes = 1, Don’t know = 9999) E – In what ways (see codes below; multiple responses possible; separate with commas) 1 – Concern Universal 184 2 – DFID 3 – FAWEMA 4 – Mary’s Meals 5 – Plan Malawi 6 – Save the Children 7 – UNICEF 8 – World Vision 9 – Yoneco 10 – Other, specify _______________ ___ 11 – Other, specify _______________ ___ 12 – Other, specify _______________ ___ 13 – Other, specify _______________ ___ Codes for 65E: It didn’t benefit the school at all = 0 Better facilities = 1 More resources for teachers = 2 More resources for learners = 3 More motivation on the part of staff = 4 More motivation on the part of learners = 5 Better quality teaching = 6 Longer school day = 7 Learners are able to read better = 8 Learner are able to learn better in other learning areas = 9 Learners are getting better scores on their tests = 10 Better or more regular attendance = 11 Other, please list__________________________________ = 12 67. What has been the most helpful type of support your school has received? (Don’t know/Refuse to answer = 9999):____________________________________________________________ • ___________________________________________________________________________ ___________________________________________________________________________ • 68. What is the least helpful type of support your school has received? (Don’t know/Refuse to answer = 9999):_______________________________________________________________ • ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 185 69. What additional support, if any, does your school most need in order to increase reading scores? (Don’t know/Refuse to answer = 9999):____________________________________________ • ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ • ___________________________________________________________________________ RESPONDENT ROLE AND THOUGHTS 70. For how many hours per week do you provide instructional support to your teachers? (Don’t know/Refuse to answer = 9999):_______ • 71. What are the reasons you don’t provide more instructional support? (Don’t prompt; select all that apply; multiple responses possible): a. I think that amount of support is enough b. I have to teach classes too often c. I have too many administrative duties d. I don’t feel comfortable providing instructional support e. The teachers don’t like it when I provide instructional support f. Other, please specify__________________________________________________ g. Don’t know/Refuse to answer • 72. Have you participated in any training on instructional support? a. No = 0 (Skip to QUESTION 74) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 74) • 73. Who provided the training on instructional support? (Don’t prompt; select all that apply; multiple responses possible): a. MoEST DTED (DEMs, PEAs, etc.) b. MIE c. MTPDS d. EGRA e. Read Malawi f. UNICEF g. World Vision h. Other, please specify_________________________________________________ i. Don’t know/Refuse to answer • • • 74. How many days have you participated in instructional support training in the past three years? (Don’t know/Refuse to answer = 9999):____________ 75. Have you participated in training or taken courses in school management in the past three years? a. No = 0 (Skip to QUESTION 78) b. Yes = 1 186 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 78) 76. Did you complete the MSSSP training? a. No = 0 b. Yes = 1 c. Don’t know/Refuse to answer = 9999 • 77. Have you participated in any other school management training? a. No = 0 (Skip to QUESTION 78) b. Yes = 1 c. Don’t know/Refuse to answer (Skip to QUESTION 78) 78. How many hours of non-MSSSP training did you receive from each of the following organizations? (Read out each organization; fill in the hours for all that apply or mark “0” if the head teacher did not receive any training from the specified organization): a. DTED_______________ b. MIE_________________ c. MTPDS______________ d. EGRA_______________ e. Read Malawi___________ f. UNICEF______________ g. World Vision___________ h. Other, please specify_________________________________________________ i. Don’t know/Refuse to respond 79. Have you received training (training of trainers) or taken courses on how to teach reading? a. No = 0 (Skip to QUESTION 80) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 80) 80. How many hours of training on how to teach reading did you receive from each of the following organizations? (Read out each organization; fill in the hours for all that apply or mark “0” if the head teacher did not receive any training from the specified organization): a. DTED_______________ b. MIE_________________ c. MTPDS______________ d. EGRA_______________ e. Read Malawi___________ f. UNICEF_____________ g. World Vision__________ h. Other, please specify_________________________________________________ i. Don’t know/Refuse to answer 81. Have you received text messages from the EGRA Project in the past two years? a. No = 0 (Skip to QUESTION 83) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 83) 187 82. When did you begin receiving the text messages from the EGRA Project? a. In the past couple of 1-3 months b. 4-6 months ago c. 7-9 months ago d. 10-12 months ago e. More than 12 months ago 83. Approximately how many text messages do you received per week? a. 0 text messages b. 1 text message c. 2 text messages d. 3 text messages e. 4 text messages f. 5 text messages g. 6 text messages h. 7 text messages i. More than 7 text messages • 84. Are you satisfied with the reading performance at Standard 4 in your school? a. No = 0 b. Yes = 1 (Skip to QUESTION 86) c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 86) 85. Why aren’t you satisfied with the reading performance at Standard 4?______________________ • ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ • ___________________________________________________________________________ 86. What things would you suggest to improve reading performance in your school?_____________ • ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ • ___________________________________________________________________________ • QUESTIONS THAT MAY REQUIRE SOME RESEARCH • 87. What is the total enrollment at the school for Standards 1-4? (Don’t know/Refuse to answer = 9999): a. Standard 1:__________________ b. Standard 2:__________________ c. Standard 3:__________________ d. Standard 4:__________________ • • 88. What is the pupil-teacher ratio across the following standards (including both trained and untrained teachers but not student trainees or substitutes)? Don’t know/Refuse to answer = 9999) (If it is 200 to 1, list 200, etc.): 188 a. Standard 1:__________________ b. Standard 2:__________________ c. Standard 3:__________________ d. Standard 4:__________________ • 89. Since the start of the current school year, was this school closed for any days other than holidays? a. No = 0 (Skip to QUESTION 91) b. Yes = 1 c. Don’t know/Refuse to answer = 9999 (Skip to QUESTION 91) • 90. How many days, other than holidays, was the school closed this academic year? (Don’t know/Refuse to answer = 9999):________ • 91. Why was the school closed for days other than holidays this year? (Do not prompt; select all that apply; multiple responses possible. Don’t know/Refuse to answer = 9999): a. Strike by teachers b. Examinations c. Funeral / Death d. Weather e. Teacher absences f. Elections g. Other(s), please specify___________________________________________________ h. Don’t know/Refuse to answer 92. What has been the average daily absentee rate (percentage) for pupils in the following standards this academic year: (Don’t know/Refuse to answer = 9999): a. Standard 1:__________ b. Standard 2:__________ c. Standard 3:__________ d. Standard 4:__________ 93. What is the dropout rate for all students in the following standards this academic year? (Don’t know/Refuse to answer = 9999): a. Standard 1:__________ b. Standard 2:__________ c. Standard 3:__________ d. Standard 4:__________ 94. Is the school located in a paternal or maternal lineage area? 1. Maternal (mother based) 189 ANNEX 15: CLASSROOM OBSERVATION INSTRUMENT Malawi EGRA Mid-Line Impact Evaluation Classroom Observation Protocol April-May, 2015 Instructions: Meet with the Head Teacher and tell him/her you want to observe a Standard 2 and Standard 4 classroom where the teacher has been teaching most of the year. For those classes, ask when the Chichewa and English reading classes are and when the breaks/recess and school feeding occurs in each class. You will need to determine your observation schedule based on this information, observing Std. 2 and Std. 4 each for 3 lessons for the same teacher. If a teacher is absent and no other class and teacher is available to be sampled, student teachers may be observed. We do not want to observe caretaker teachers. Enumerator: COMPLETE A SEPARATE PROTOCOL FOR EACH LESSON 1. Questionnaire ID:____________________________________________________________ 2. Enumerator Name:___________________________________________________________ 3. Survey and Logistics Manager Signature:___________________________________________ 4. Technical Manager Signature:___________________________________________________ 5. Division:____________________________________________________________________ 6. District:____________________________________________________________________ 7. Zone:______________________________________________________________________ 8. School:_____________________________________________________________________ 9. EMIS ID Number:___________________________________________________________ 10. Teacher name:_______________________________________________________________ 11. Teacher gender:______________________________________________________________ 12. Date:_______________________________________________________________________ 13. Class Standard:_______________________________________________________________ 14. Is teacher present when lesson is scheduled to begin? Yes______ No______ 15. Time lesson begins:____________________________________________________________ For 16 and 17, enter the number of boys/girls present when the lesson begins and then add those who come late at the end of the lesson – from #20. 16. Number of boys present:_______________________________________________________ 17. Number of girls present:________________________________________________________ 18. Number of adults helping in the classroom in addition to the teacher:_____________________ 19. Subject being taught: a. Reading (in Chichewa) b. English c. Reading in another language, please specify language________________ d. Other, please specify_________________________________________ 20. Number of learners that come to class late: (Complete table by entering under “minutes” columns a tick in the appropriate cell each time a pupil comes in late, then sum and record an “X” in each appropriate row) Time late in minutes No. of Learners A - 1 - 10 B - 11 - 20 C - 21 - 30 D - Total late 190 1 - Boys 2 - Girls 191 TEACHER BEHAVIOR OBSERVED 1 Opposite of behavior described or do not see the behavior described 2 See the behavior sometimes or partially correct 3 See the behavior done very well and consistently where appropriate 4 Not Applicable (Behavior is not relevant to the subject being taught) 25a. Uses a lesson plan 25b. Uses a scripted lesson plan 26. Introduces lesson by connecting to what learners have learned previously 27. Introduces lesson with advance organizer 28. Manages instructional time effectively 29. Demonstrates effective classroom management skills 30. Makes effective use of different instructional resources and strategies 31. Treats all students equally/fairly 32. Engages learners in carefully planned cooperative learning strategies 33. Asks probing, open-ended questions that encourage thinking and helps learners explicate their thinking 34. Provides learners with structured opportunities to apply their understanding and skills to everyday life and problems 35. Provides opportunities for learners to develop higher-order and critical thinking skills 36. Uses appropriate learning materials besides textbooks 37. Assesses pupil learning BIAS or MISTREATMENT 38. Avoids using gender biased language 192 1 Opposite of behavior described or do not see the behavior described 2 See the behavior sometimes or partially correct 3 See the behavior done very well and consistently where appropriate 4 Not Applicable (Behavior is not relevant to the subject being 39. Avoids using abusive language 40. Provides positive, encouraging feedback 41. Does not allow learners to use gender bias 42. Does not allow learners to use abusive language 43. Girls have equal access to desks, learning materials, etc. READING PRACTICE May need to mark Option 4 for many of these, if not observing a reading class 45. Engages learners in reading activities or games appropriate to their reading level 46. Encourages learners to “sound it out” when they don’t know a word 47. Avoids criticizing learners who don’t answer correctly or read poorly 48. When teacher or pupil(s) read a story, teacher asks learners pre-reading questions such as “What do you think the story will be about based on the pictures and/or title of the book?” 49. When teacher or learners read a story, teacher asks learners to make appropriate sounds or act something out, such as the roar a lion makes or the way a frog hops 50. Applies multiple methods to support comprehension, including games, group work, etc. 193 51. Encourages learners to help each other 52. Has individual learners read aloud 1 Opposite of behavior described or do not see the behavior described 2 See the behavior sometimes or partially correct 3 See the behavior done very well and consistently where appropriate 4 Not Applicable (Behavior is not relevant to the subject being 53. Provides instructions on how to decode syllables and words 54. Teaches learners meanings of new words 55. Asks learners questions to assess their understanding of something the learner(s) or teacher have/has read 57. Asks learners questions to assess their understanding of stories they hear 58. Asks learners to recognize letters and say letter names and/or sound 59. Learners retell a story they or the teacher read 60. Asks learners to recite the alphabet 61. Assigns reading for learners to do on their own during school time 62. Provides a variety of methods for learners to establish good writing skills PUPIL BEHAVIOR 63. Most learners are paying attention 64. Most learners are actively engaged in the lesson 65. Most learners are actively engaged when working in small groups or in pairs 66. Learners appear to understand what the teacher is saying 194 21. Number of textbooks being used by learners:________________________________________ 22. Number of reading materials on walls and around classroom (NOT painted walls): ___________ 23. Number of reading materials on the walls and around the classroom that appear to be recent:___ 24. Langua ge teacher uses A - Lo ca l o nl y B - Local + Chich ewa C - Loc al + Eng lish D - Chich ewa only E - Eng lish onl y F – Chich ewa + Engli sh Put X in approp riate box NOTES ABOUT THE LESSON:_________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ Time lesson ends: ______________ Length of Lesson:_______________ 195 ANNEX 16: SCHOOL CLIMATE INSTRUMENT Malawi EGRA Mid-Line Impact Evaluation School Climate Observation Protocol April-May, 2015 196 Strongly Agree Agree Disagree Strongly Disa ree Not Applicable Observed Conditions No improvement Slight improvement is needed Much improvement is needed Urgent improvement is needed Not applicable School grounds well maintained – without litter Rubbish bins are available to dispose of rubbish School has plantings to make the school more attractive There are no broken windows Buildings and classrooms have functioning locks Classrooms have space for the teacher and students to move around Class schedule for entire school is available in HT’s office or Teachers Room (only use not applicable if there is no HT office) A teachers’ lounge/room is available Teachers’ lounge/room is in good condition Classrooms have sufficient ventilation Classrooms have sufficient light Classrooms have electricity The school has clean water available for learners to drink/wash their hands Classrooms appear to have a range of learning materials available – not simply years-old posters or paintings on the wall Latrines are available Latrines are clean Latrines are available specifically for girls Latrines are available specifically for boys Latrines are available specifically for teachers Most or all classrooms have enough desks for all students There is a school (not classroom) library The library appears to be well stocked The library appears to be actively used by students and teachers The books in the library are in good condition Most textbooks appear to have been distributed to students (select not applicable if there are no textbooks in the school) 197 School Name:_____________________________________________________________________________ ___ School EMIS ID:________________________________________________________________________________ School Questionnaire ID:_________________________________________________________________________ Enumerator Name:_____________________________________________________________________________ Enumerator Signature:___________________________________________________________________________ Survey and Logistics Manager Signature:______________________________________________________________ Technical Manager Signature:______________________________________________________________________ NOTES:____________________________________________________________________________ __________________________________________________________________________________ ___________________ __________________________________________________________________________________ __________ __________________________________________________________________________________ __________ __________________________________________________________________________________ __________ Resources in this school are adequate for teaching the material Teachers/head teachers appear very engaged and interested in the development of learners Teachers/head teachers do not seem hostile or angry. No signs of physical punishment of students School staff speak to students in a friendly, supportive way No student fighting/bullying observed No teasing among pupils observed Few interruptions of class time When learners get into trouble, the teacher/head teacher gives them a chance to tell their side of the story The school has school feeding If observed, school feeding functions in an orderly way 198 __________________________________________________________________________________ __________ __________________________________________________________________________________ __________ __________________________________________________________________________________ _________ __________________________________________________________________________________ __________ __________________________________________________________________________________ __________ __________________________________________________________________________________ __________ __________________________________________________________________________________ __________ 199 ANNEX 17: HOUSEHOLD SURVEY Malawi Early Grade Reading Activity (EGRA) Impact Evaluation - Midline Assessment Household Survey May - June 2015 Hi, . My name is _____________, and I am from IKI, an independent data collection firm contracted by the Malawi Ministry of Education, Science and Technology (MoEST) with funding from USAID. We are conducting a study to assess the impact of a three-year project to support teachers and schools in their efforts to teach children to read. MoEST and USAID have contracted with our independent company to conduct this study. The first part of our study involved testing learner reading abilities at randomly selected schools in or near areas where the new education project is being implemented. [LEARNER’S NAME] was randomly selected to take part in this study. We visited his/her school a few weeks ago to assess his/her reading ability. But, now we want to understand more about the various factors that may be affecting [LEARNER’S NAME] ability to read. We would like your help in this. But you do not have to take part if you do not want to, and you are free to opt out of any questions you do not feel comfortable answering. If you decide to take part, your responses will be confidential. Your name will not be mentioned anywhere in the survey data or report. We will collect it only so that we can follow up with you or your family again in a few years, but it will only be known to myself and a handful of researchers. It will not be reported to MoEST or USAID. The results of our analysis will be used by the Ministry of Education, Science and Technology to help identify additional support that is needed to help ensure that all children in Malawi become good readers. Additionally, the school that [LEARNER’S NAME] attends will receive a report of the results that it can use to help better address the needs of children at that school. However, your learner’s name will not be included in this or any report and will be kept confidential. If you agree to help with this study, I will read you a consent statement and ask you to sign or make a mark on the line below and answer the questions I will ask you as completely and accurately as you can. This survey will take approximately one hour to complete. Do you have any questions? Do you provide your consent to begin? CONSENT STATEMENT: I understand and agree to participate in this reading research study by filling out this questionnaire as completely and accurately as possible. A unduna wa zamaphunziro, sayansi ndi luso ndi chthandizo chochokera ku USAID,akupanga kafukufuku wa pulojekiti yomwe ikuthandiza aphunzitsi ndi sukulu pophunzitsa ana kuwerenga. [LEARNER’S NAME] anasankhidwa kuti atenge nawo mbali mukafukufukuyu. Tinakamuyendera kusukulu yake sabata zingapo zapitazo kukamuona mmene iye amawerengera. Pakadali pano tikufuna tiwone zinthu zosiyanasiyana zimene zingakhudze mawerengede ake. Tikufuna thandizo lanu pa zimenezi. Koma ngati simukufuna kutenga nawo mbali mutha kutero komanso muli ndi ufulu osayankha. Ngati musankhe kutenga nawo mbali, mayankho anu akasungidwa mwachinsinsi. Dzina lanu silikaoneka mmalipoti amene ati akalembedwe. Dzinali lidzatengedwa kuti litithandize pa nthawi yomwe tidzakufuneninso inuyo kapena banja lanu kutsogoloku. Dzinalinso silikaperekedwa ku unduna wa zamaphunziro ndi luso kapena kubungwe la USAID. Koma zotsatira za kafukufukuyu zidzagwiritsidwa ntchito ndi a unduna wa zamaphunziro ndi luso kapena bungwe la USAID kuti akathe kupeza mmene angathe kuthandizapo kuti ana athe kukhala otha kuwerenga bwino. Poonjezerapo, sukulu yomwe [LEARNER’S NAME] amaphunzira idzalandira ripoti lomwe lingazathandize kuona mmene angawathandizire ana ngakhale palibe dzina la wophunzira wina aliyense lomwe lidzalembedwamo. Ndikuwerengerani chivomerezo ichi ndipo ngati mwavomera kuthandizapo mukafukufukuyi, ndikupemphani kuti musayinire m’musimu ndipo muyankhe mafunso onse moyenera. Ndamvetsetsa ndipo ndikuvomereza kutenga nawo mbali mukafukufukuyu wothandiza ana kudziwa kuwerenga poyankha mafunso onse ndinso moyenera. 200 SIGNATURE/Mark:____________________________________________________________________ ___________Visits: Visit 1 Date:______________________ Time:____________________________ Result:__________________________ Visit 2 Date:_____________________ Time:____________________________ Result:___________________________ Visit 3 Date:______________________ Time:____________________________ Result:__________________________ Final Visit Date:______________________ Time:__________________________ Result:_________________________ Total Number of Visits:______________________________________________________________________________ RESULT CODES a. Completed b. Nobody at home or no one who is capable of responding. c. Respondent asked to postpone the visit. d. Respondent refused to participate. e. Dwelling vacant or location not a dwelling. f. Dwelling destroyed. g. Dwelling not found. h. Other, please specify on line above. Total # Persons in the Household:____________________________________________________________________ ____ Type of Guardianship for the Learner: a. Double-parent household b. Single-parent household c. Guardianship d. Child-headed household e. Other, please specify________________________________________________________________________ __ Latitude:_______________________________________ Longitude:____________________________________________ 201 The following information should be prefilled by the enumerator: 1. District:_________________________________________________________________ ___________________ 2. Zone:__________________________________________________________________ ____________________ 3. Village:__________________________________________________________________ ___________________ 4. Name of Learner:_________________________________________________________________ ___________ 5. Standard of Learner:_________________________________________________________________ _________ 6. School Learner Attends:________________________________________________________________ _______ 7. School EMIS ID Number:________________________________________________________________ ______ 8. Enumerator:_____________________________________________________________ ___________________ 9. Survey and Logistics Manager Name:____________________________________________________________ 10. Technical Manager Name:__________________________________________________________________ ___ 11. Date:___________________________________________________________________ ___________________ 12. Time Start:___________________________________________________________________ _______________ 13. Respondent Name:__________________________________________________________________ _________ 14. Language of Interview a. Chichewa = 1 b. Citumbuka = 2 c. Ciyawo = 3 d. English = 4 202 e. Other, please specify_________________________________ 203 15. HH Roster 1 2 3 4 5 6 7 8 9 10 11 12 Please list each of the members who live in your household – including all the infants, children, adults, and elderly. Do not include those who live on your compound but not in your household. I only need the surname for the head of the household and the learner. Name (given name and surname ) Dzina What is [NAME’s ] relations hip to the head of househol d? Kodi (dzina) ndi wamkulu wapakho mo pano pali ubale wanji? SEE CODES What is [NAME’s] relationship to [LEARNER’S NAME]? SEE CODES Kodi pali ubale wanji pakati pa (dzina) ndi (dzina la ophunizira)? What is [NAME’ S] sex? (Only ask if not obvious) 0: Male 1: Female Kodi (dzina) ndi wamwa muna kapena wa mkazi? How old is [NAME] ? Kodi (dzina) ali ndi zaka zingati? GO TO NEXT PERSO N ON ROSTE R if person is under the age of 2 Did [NAME] attend school this year? (Only ask this question is the person is older than 2) Kodi (dzina) amapita kusukulu chaka chino? 0-No (SKIP TO COLUMN 9) 1-Yes What level of school did he or she attend this year? SEE COD ES Kodi (dzina ) anali kalasi yanji chaka chino? Did he or she repeat this year? Kodi wabwe reza chaka chino? 0 – No 1 - Yes If column 6 is coded with a “0”, ask what is the highest level of education completed by [NAME]? Kodi dzina anamaliza kalasi yanji maphunziro ake? SEE CODES If the answer in column 9 is coded as less than 1 AND column 6 is coded as “0” ask “why didn’t he/she attend school this year?”Ndi chifukwa chiyani sanapite ku sukulu? If column 9 is coded as 1-8 AND column 6 is coded as “0”, ask “why did he or she drop out of school?”Ndi chifukwa chiyani analekezera sukulu panjira? SEE CODES Can [NAME] read? Kodi (dzina) amatha kuwerenga? 0 – No 1 – A little (e.g. can read signs but not books) 2 - Yes If Col the Can [NA read pag in C Kod [DZ atha kuw tsam limo Chi A – HEAD B – [LEARNER NAME] C D E 204 Make a complete list of all concerned before going to other columns. F G H I J K L 205 15. HH Roster 1 2 3 4 5 6 7 8 9 10 11 1 to Please list each of the members who live in your household – including all the infants, children, adults, and elderly. Do not include those who live on your compound but not in your household. I only need the surname for the head of the household and the learner. Name (given name and surname) Dzina What is [NAME’s] relationshi p to the head of household ? Kodi (dzina) ndi wamkulu wapakho mo pano pali ubale wanji? SEE CODES What is [NAME’s] relationshi p to [LEARNE R’S NAME]? SEE CODES Kodi pali ubale wanji pakati pa (dzina) ndi (dzina la ophunizira )? What is [NAME’ S] sex? (Only ask if not obvious) 0: Male 1: Female Kodi (dzina) ndi wamwa muna kapena wa mkazi? How old is [NAME] ? Kodi (dzina) ali ndi zaka zingati? GO TO NEXT PERSO N ON ROSTE R if person is under the age of 2 Did [NAME] attend school this year? (Only ask this question is the person is older than 2) Kodi (dzina) amapita kusukulu chaka chino? 0-No (SKIP TO COLUMN 9) 1-Yes What level of school did he or she attend this year? SEE COD ES Kodi (dzina ) anali kalasi yanji chaka chino? Did he or she repeat this year? Kodi wabwe reza chaka chino? 0 – No 1 - Yes If column 6 is coded with a “0”, ask what is the highest level of education completed by [NAME]? Kodi dzina anamaliza kalasi yanji maphunziro ake? SEE CODES If the answer in column 9 is coded as less than 1 AND column 6 is coded as “0” ask “why didn’t he/she attend school this year?”Ndi chifukwa chiyani sanapite ku sukulu? If column 9 is coded as 1-8 AND column 6 is coded as “0”, ask “why did he or she drop out of school?”Ndi chifukwa chiyani analekezera sukulu panjira? SEE CODES Can [NAME] read? Kodi munthuyu amatha kuwereng a? 0 – No 1 – A little (e.g. can read signs but not books) 2 - Yes If C th C [N re pa in C K [D at k a lim m C M N O P Q R S T U V W 206 X A. CODES FOR (Q14-2) RELATIONSHIP TO HEAD 1=SPOUSE OF HEAD OF HOUSE 2=CHILD OF HEAD 3=GRANDCHILD OF HEAD OF HOUSE 4=NIECE/NEPHEW OF HEAD OF HOUSE 5=PARENT OF HEAD OF HOUSE 6=SIBLING OF HEAD OF HOUSE 7=MOTHER/FATHER-IN-LAW OF HEAD 8=SON/DAUGHTER-IN-LAW OF HEAD 9=BROTHER/SISTER-IN-LAW OF HEAD 10=GRANDFATHER/GRD.MOTHER OF HEAD 11=FATHER/MATHER-IN-LAW OF HEAD 12=STEPFATHER/STEPMOTHER OF HEAD 13=STEPBROTHER/STEPSISTER OF HEAD 14=OTHER RELATIVE 15=NON-RELATIVE B. CODES FOR (Q14-3) RELATIONSHIP TO THE LEARNER 1=FATHER OF LEARNER 2=MOTHER OF LEARNER 3=SIBLING OF LEARNER 4=GRANDFATHER OR GRANDMOTHER OF LEARNER 5=COUSIN OF LEARNER 6=UNCLE OF LEARNER 7=AUNT OF LEARNER 8=SISTER-IN-LAW OF LEARNER 9=BROTHER-IN-LAW OF LEARNER 10=GREAT GRANDFATHER OR GREAT GRANDMOTHER 11=STEPFATHER/STEPMOTHER OF LEARNER 12=NIECE/NEPHEW OF LEARNER 13=FAMILY FRIEND 14=OTHER RELATIVE C. CODES FOR LEVEL OF SCHOOL ATTENDED THIS YEAR and HIGHEST LEVEL OF EDUCATION COMPLETED (Questions 14-7, 14-9, and 14- 10) 0 = NONE 1 = NURSERY/PRESCHOOL 2 = STANDARD 1 3 = STANDARD 2 4 = STANDARD 3 5 = STANDARD 4 6 = STANDARD 5 7 = STANDARD 6 8 = STANDARD 7 9 = STANDARD 8 10 = JUNIOR FORM 1 11 = JUNIOR FORM 2 12 = SENIOR FORM 3 13 = SENIOR FORM 4 14 =VOCATIONAL TRAINING 15 =UNIVERSITY UNDERGRADUATE 16 =UNIVERSITY POST￾GRADUATE 17 =OTHER D. REASON PERSON DIDN’T ATTEND SCHOOL OR DROPPED OUT 1 = LIMITED AVAILABILITY OF TEACHERS 2 = EMPLOYMENT/HELPING FAMILY WITH WORK 3 = TAKING CARE OF SIBLINGS OR OTHER RELATIVES 4 = FEES/COST/COULDN’T AFFORD 5 = DISTANCE (THE CLOSEST SCHOOL WAS TOO FAR AWAY) 6 = MARRIAGE 7 = POOR SCHOOL FACILITIES 8 = PREGNANCY 9 = SICKNESS 10 = CONCERN OVER VIOLENCE/BULLYING/TEASING 11 = NOT MOTIVATED/DIDN’T VALUE EDUCATION 207 12 = CURRICULUM TOO DIFFICULT OR NOT PERFORMING WELL 13 = OTHER 208 BACKGROUND ON HOUSEHOLD – I would like to ask you some general background questions about your household. 15a. What is the current marital status of the household head? Kodi mkulu wa khomo lino ali pabanja? a. Married b. Divorced c. Widowed d. Never married e. Other living arrangements, please specify ________________________ f. Refuse to respond 16. Which languages are spoken in your household? Ndi zilankhulo ziti zimene zimalankulidwa pakhomo pano, ndipo zimalankhulidwa pafupipafupi bwanji? (Select all that apply; multiple responses possible) a. Chichewa b. Citumbuka c. Ciyawo d. English e. Other, please specify__________________________________________________________________ f. Refuse to respond 17. What language does [LEARNER’S NAME] most commonly use with his/her friends? Kodi ndi chiyankhulo chiti chimene [dzina la ophunzira] amalankhula kwambiri (chimagwiritsidwa ntchito kwambiri) ndi anzake? a. Chichewa = 1 b. Citumbuka = 2 c. Ciyawo = 3 d. English = 4 e. Other, please specify___________________________ = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 209 18. How long have you been living in your current village? Kodi m’mudzi muno mwakhalamo nthawi yayitali bwanji? (List in complete years unless the family has been living there for less than a year, then list months) a. < 1 year =1 b. 1-2 years =2 c. 3-4 years =3 d. 5 or more years = 4 (SKIP TO QUESTION 19) e. Don’t know = 8888 f. Refuse to respond = 9999 18.a Where did you live before? Kodi poyamba munkakhala kuti? (If the respondent does not know the zone, write down the other information and add the zone in later) g. District:______________________________________________________________________________ h. Zone:_______________________________________________________________________________ i. Village, town or city:___________________________________________________________________ j. School name that [LEARNER’S NAME] attended previously, if relevant:___________________________ HOUSING CONDITION AND HOUSEHOLD ASSETS – Now, I would like to ask about some assets you may have at your house. Tsopano ndikufunsani za katundu amene muli naye pakhomo panu pano 19. What is your household’s main source of drinking water? Kodi madzi akumwa mumatunga kuti? a. Piped water into dwelling = 1 (SKIP TO QUESTION 24) b. Piped water into yard/plot = 2 (SKIP TO QUESTION 24) c. Piped water into community/stand pipe = 3 d. Unprotected well = 4 (This is a dug well for which one of the following conditions is true: 1) the well is not protected from runoff water; or 2) the well is not protected from bird droppings and animals. If at least one of these conditions is true, the well is unprotected). e. Protected well =5 e. Borehole = 6 f. Spring = 7 g. River/stream = 8 210 h. Pond/lake = 9 i. Dam = 10 j. Rainwater = 11 k. Tanker truck/bowser = 12 l. Bottled water = 13 m. Other, please specify__________________________________ = 14 n. Don’t know = 8888 o. Refuse to respond = 9999 20. How long does it take for someone to go to the water source, get water, and return to the house? Kodi munthu amatenga nthawi yayitali bwanji kupita kotunga madzi, kutunga madzi ndikubwerera kunyumba? a. No time/water is on the premises = 0 b. 1-10 minutes = 1 c. 11-20 minutes = 2 d. 21-30 minutes = 3 e. 31-40 minutes = 4 f. 41-50 minutes = 5 g. 51-60 Minutes = 6 h. More than 60 minutes = 7 i. Don’t know = 8888 Refuse to respond = 9999 21. How often does someone go to get water? Kodi kotunga madzi kumapitidwa pafupipafupi bwanji? a. No time/the water is on the premises = 0 b. A few times a week = 1 c. Once a day = 2 211 d. 2-3 times a day = 3 e. More than 4 times a day = 4 f. Don’t know = 8888 g. Refuse to respond = 9999 22. Who is the primary person who goes to get the water? Kawirikawiri amapitapita kotunga madzi ndi ndani? a. Learner = 1 b. Other child from the household = 2 c. Adult man from the household = 3 d. Adult woman from the household= 4 e. Someone from outside the household = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 23. Who is the secondary person who goes to get the water? Kodi winanso amene amapitapita kotunga madzi ndi ndani? a. Learner = 1 b. Other child from the household = 2 c. Adult man from the household = 3 d. Adult woman from the household= 4 e. Someone from outside the household = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 212 24. What kind of toilet facility does your household use? Kodi pakhomo pano mumagwiritsa ntchito chimbudzi chantundu wanji? a. Flush toilet = 1 b. Traditional pit toilet = 2 c. Ventilated pit toilet (ventilated improved pit) = 3 d. Latrine = 4 e. No facility = 5 (Skip to question 26) f. Other, please specify______________________________ = 6 g. Don’t know = 8888 h. Refuse to respond = 9999 25. Do you share this facility with other households? Kodi chimbudzichi chimagwiritsidwanso ntchito ndi anthu a m’makomo ena? h. No = 0 i. Yes = 1 j. Don’t know = 8888 k. Refuse to respond = 9999 26. As of today, which of the following does your household have? Kodi khomo lino liri ndi zinthu izi? (Read response options to the respondent and select all that apply; multiple responses possible) a. A paraffin lamp Nyali ya palafini 213 b. A cell phone Foni ya m’manja c. A bicycle Njinga yopalasa d. A coffee/dining table Tebulo e. A bed with mattress bedi ndi matilesi f. A sofa Sofa g. A radio Wayilesi h. A television TV i. A tape player, CD player, or HiFi Wayilesi ya kaseti, CD j. An iron for pressing clothes Simbi k. An ox plow Makasu olimira ndi ngómbe l. Gold/silver jewelry Ndolo kapena zibangili za golide kapena siliva m. A motorcycle and motorized scooter Njinga ya moto n. A refrigerator Filiji o. A car or truck Galimoto p. A tractor Thirakita 27. As of today, Does anyone in your household own any animals and poultry Kodi alipo pakhomo pano amene ali ndi ziweto? (these can be animals and poultry kept for personal consumption/use or those raised for selling/producing products for sale)? a. No = 0 (SKIP TO QUESTION 29) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 29) d. Refuse to respond = 9999 (SKIP TO QUESTION 29) 28. As of today, How many of the following types of animals does your household own? Mwamitundu yaziweto izi, khomo lanu lino liri ndi zingati? (Read response options to the respondent and select all that apply; multiple responses possible) a. Cows Ng’ombe zazikazi _____________ 214 b. Pigs Nkhumba ______________ c. Sheep Nkhosa ____________ d. Goats Mbuzi _____________ e. Chickens Nkhuku __________ f. Horses Mahachi ____________ g. Donkeys Abulu ___________ h. Ox/Bull Ng’ombe zokoka ngolo, zamphongo ____________ i. Other 1, Zina 1, please specify type___________________ number_______________________ j. Other 2, Zina 2, please specify type___________________ number_______________________ k. Other 3, Zina 3please specify type___________________ number_______________________ l. Don’t know = 8888 m. Refuse to respond = 9999 29. What type of fuel does your household mainly use for cooking? Kodi mumaphika pachiyani? a. Electricity = 1 b. Gas = 2 c. Paraffin = 3 d. Charcoal = 4 e. Firewood = 5 f. Straw = 6 g. Other, please specify_______________ = 7 h. Don’t know = 8888 i. Refuse to respond = 9999 30. What is the main source of lighting in your house? Kodi nthawi zambiri mumaunikira chiyani? 215 a. Electricity = 1 b. Gas = 2 c. Paraffin = 3 d. Firewood = 4 e. Grass = 5 f. Candle = 6 g. Solar = 7 h. Torch = 8 i. Battery-lit light bulbs = 9 j. No lighting = 10 k. Other, please specify_______________ = 11 l. Don’t know = 8888 m. Refuse to respond = 9999 ENUMERATOR ASSET OBSERVATION (Enumerator should observe and fill out this section without asking unless the answer is not apparent, in which case it is okay to clarify). 31. What are the walls of the house made of? (Select all that apply; multiple responses possible) Kodi khoma la nyumbayi ndi lomangidwa motani? a. Mud = 1 b. Stone = 2 c. Stone and wood = 3 d. Bamboo = 4 e. Wood = 5 f. Concrete = 6 g. Burnt bricks = 7 h. Bricks = 8 i. Other, please specify____________________ = 9 216 j. Don’t know = 8888 (should only be used in extreme circumstances) 32. What is the roof made of? Kodi denga la nyumba yanu ndi la chiyani? a. Thatch/Palm Leaf (Natural Roofing) = 1 b. Sod (Natural Roofing) = 2 c. Rustic Mat (Basic Roofing) = 3 d. Palm/Bamboo (Basic Roofing) = 4 e. Slate = 5 f. Wood Planks (Basic Roofing) = 6 g. Cardboard (Basic Roofing) = 7 h. Metal/Aluminum sheets = 8 i. Wood = 9 j. Calamine/Cement Fiber = 10 k. Ceramic Tiles = 11 l. Cement = 12 m. Roofing Shingles = 13 n. Other (specify)Other, please specify___________________ = 14 o. Don’t know = 8888 (should only be used in extreme circumstances) 33. What is the main material of the floor? Kodi pansi pa nyumbayi panapangidwa ndi chiyani? a. Earth/sand = 1 b. Dung = 2 c. Wood planks = 3 d. Palm/bamboo = 4 e. Broken bricks = 5 217 f. Parquet or polished wood = 6 g. Cement = 7 h. Vinyl or asphalt strips = 8 i. Ceramic tiles = 9 j. Brick = 10 k. Other, please specify__________________ = 11 l. Don’t know = 8888 (should only be used in extreme circumstances) 34. Not including the bathroom, how many rooms does the house have? Osaphatikiza kubafa kapena ku chimbudzi, nyumba iyi ili ndi zipinda zingati? a. List number______________________ b. Don’t know = 8888 (should only be used in extreme circumstances) ACCESS TO HEALTH SERVICES – Now, I would like to talk about what you do when people in your household are sick. 35a. How long does it take for you to get to the closest clinic? Mumatenga nthawi yayitali bwanji kukafika kuchipatala chapafupi? a. Hours___________ Minutes __________ b. Don’t know = 8888 c. Refuse to respond = 9999 35b. How long does it take for you to get to the closest Health Service Center? Mumatenga nthawi yayitali bwanji kukafika kuchipatala chapafupi? d. Hours___________ Minutes __________ e. Don’t know = 8888 f. Refuse to respond = 9999 36. When a child in your household needs medical care, what do you typically do? Wina wapabanja pano akadwala, nthawi zambiri mumatani? a. Go to a clinic = 1 (SKIP TO QUESTION 38) b. Go to a Health Service Center = 2 (SKIP TO QUESTION 38) c. Go to a health surveillance assistant (HSA) =3 (SKIP TO QUESTION 38) 218 d. Go to the hospital = 4 (SKIP TO QUESTION 38) e. Take care of him/her at home = 5 f. Go to a traditional village healer = 6 g. Other, please specify________________________________________ = 7 h. Don’t know = 8888 i. Refuse to respond = 9999 37. Why do you not go to a clinic, Health Service Center, or hospital when someone is sick? (Select all that apply; multiple responses possible) Chifukwa chiyani simupita kuchipatala wina aliyense wapakhomo pano akadwala? a. We can’t afford the clinic fees (SKIP TO QUESTION 38e) b. We can’t afford the transportation (SKIP TO QUESTION 38e) c. It is too far (SKIP TO QUESTION 38e) d. It is difficult to get there (SKIP TO QUESTION 38e) e. Traditional healing/at home care is just as good or better than going to a clinic/hospital (SKIP TO QUESTION 38e) f. Don’t know (SKIP TO QUESTION 38e) g. Refuse to respond (SKIP TO QUESTION 38e) 38. How do members of your household usually travel to the nearest clinic/Health Service Center/hospital when they need to go? Kodi anthu a pabanja pano amayenda bwanji akafuna kupita kuchipatala chapafupi/kwa alangizi a zaumoyo? a We never go to a clinic, Health Service Center, or hospital = 1 b By foot = 2 c By bicycle = 3 d By personal motorcycle = 4 e By personal car = 5 f By getting a ride with a neighbor = 6 g By bus, van, or other public transport = 7 219 h Other, please specify_______________________ = 8 i There is no need to/we never travel to a clinic/hospital = 9 j The nearest clinic/hospital is too far away for us to travel to = 10 k Don’t know = 8888 l Refuse to respond = 9999 38a. What type of services have you received from the clinic or Health Service Center? Kodi ku chipatala mumalandira chithandizo chanji? a. First Aid b. Immunization for Measles/Mumps/Rubella c. Immunization for Tetanus d. Other vaccines/immunizations e. Treatment for Malaria f. Treatment or information for malnutrition g. Consultation/Advice h. Other, please specify_________________________ 220 38c. How often have you encountered any of these problems with your local public clinic or Community Health Service Center in the past three years? Kodi m’zaka zitatu zapitazi mwakumana ndi mavuto awa pafupipafupi bwanji kuchipatala chimene muli nacho pafupi a. Services are too expensive/unable to pay Kulephera kulipira 0 Never 1 Once or twice 2 A few times 3 Often 8888 Don’t know 9999 Refuse to respond b. Lack of medicine or other supplies Kusowa kwa mankhwala 0 Never 1 Once or twice 2 A few times 3 Often 8888 Don’t know 9999 Refuse to respond c. Lack of attention or respect from staff Kusowa ulemu kwa ogwira ntchito ku chipatala 0 Never 1 Once or twice 2 A few times 3 Often 8888 Don’t know 9999 Refuse to respond d. Absent doctors Kujomba kuntchito kwa madotolo 0 Never 1 Once or twice 2 A few times 3 Often 8888 Don’t know 9999 Refuse to respond e. Long wait times Kutalika kwa mizere yodikilira chithandizo m70e: Long waiting time 0 Never 1 Once or twice 2 A few times 3 Often 8888 Don’t know 9999 Refuse to respond f. Dirty facilities Uve 0 Never 1 Once or twice 2 A few times 221 3 Often 8888 Don’t know 9999 Refuse to respond 38d. How long did you wait at your last visit to the Health Service Center or clinic? Kodi munadikira nthawi yaitali bwanji nthawi yomaliza imene munapita kuchipatala? Hours:__________________ Minutes:__________________ 38e. Is there a Health Surveillence Assistant (HSA) in your village? Kodi m’mudzi mwanu muno muli alangizi a zaumoyo? a. No = 0 (SKIP TO QUESTION 39) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 39) d. Refuse to respond = 9999 (SKIP TO QUESTION 39) 38f. Has a Health Surveillence Assistant (HSA) ever visited your household or provided your household members with support? a. No = 0 b. Yes = 1 (SKIP TO QUESTION 38h) c. Don’t know = 8888 (SKIP TO QUESTION 38h) d. Refuse to respond = 9999 (SKIP TO QUESTION 38h 38g. Why not? (Record response exactly as said; write in 8888 for don’t know or 9999 for refuse to respond) ______________________________________________________________________________________________________________ ______________________________________________________________________ (SKIP TO QUESTION 39 after providing response) 38h. When did your household first get visited (or did you household members first receive support) by the Health Surveillence Assistant (if there are two different dates, record information on the earliest date)? a. In the past month b. Less than one year ago (but more than one month ago) c. Less than two years ago (but more than one year ago) d. Less than three years ago (but more than two years ago) e. More than three years ago 222 f. Don’t know = 8888 g. Refuse to respond = 9999 38i. On average, how often does your household receive support from a Health Surveillence Assistant (HSA)? a. Weekly b. Monthly c. Annually d. Only when someone is sick or pregnant e. Other, please specify___________________________________________________________________ f. Don’t know = 8888 g. Refuse to respond = 9999 38j. What type of services has your household received from the Health Surveillence Assistant (HSA)? (Read prompts to the respondent, and select all that apply) a. Community-Based Maternal Health (CBMH) b. Support for Nutrition (SUN) c. First Aid d. Immunization for Measles/Mumps/Rubella e. Immunization for Tetanus f. Other vaccines/immunizations g. Treatment for Malaria h. Treatment or information for malnutrition i. Consultation/Advice j. Other, please specify_________________________ 38k. Have you or anyone in your household experienced any of the following problems with the Health Surveillence Assistant in the past three years? (Multiple responses possible/select all that apply) 223 a. He/she did not provide preventative care or treatment on time (when needed) b. The nutrition support he/she provided was not sufficient for my household c. The treatment provided for an illness was not adequate d. His/her advice did not help my family e. Other, please specify________________________________________________ ACCESS TO OTHER SERVICES – Now, I would like to ask about your access to other types of services. 39. Over the past month, did anyone in your household purchase or pay for powder clothes soap? Mwezi wathawu, kodi munagulapo sopo waufa wochapira? a. No = 0 b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 39.a Does your family have access to finance/a loan should it need it for an emergency or any other reason? Kodi banja lanu lili ndi mwayi wopeza ndalama/ ngongole patafunika kutero? a. No = 0 (SKIP TO QUESTION 40a) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 40a) d. Refuse to respond = 9999 (SKIP TO QUESTION 40a) 224 39.b Has your household/family received a loan or or credit from any of the following sources in the past three years? Muzaka zitatu zapitazi kodi banja lanu lalandirapo ngongole kuchokera ku: (Read all answer choices, and select all that apply; multiple responses possible) a. A bank b. A microfinance institution c. A non-governmental organization d. Village banking e. A local lender/trader f. A friend or relative g. Other, please specify_________________________ h. Don’t know i. Refuse to respond 40a. Have you or anyone else in your household participated in an INVC community Care Group? a. No = 0 (SKIP TO QUESTION 40g) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 40g) d. Refuse to respond = 9999 (SKIP TO QUESTION 40g) 40b. When did you/he/she start participating in the INVC community Care Group? a. In the past month b. Less than one year ago (but more than one month ago) c. Less than two years ago (but more than one year ago) d. Less than three years ago (but more than two years ago) e. More than three years ago f. Don’t know = 8888 g. Refuse to respond = 9999 225 40c. What type of support has this person received from the INVC Care Group? a. I/he/she watched a video on nutrition = 1 b. I/he/she received training from the Care Group’s lead mother or lead father on maternal nutrition and diet, breastfeeding, complementary feeding, and/or hygiene = 2 c. Other, please specify________________________________________________________________ = 3 d. Don’t know = 8888 e. Refuse to respond = 9999 226 40e. Did you experience any of the following problems with the Care Group in the past three years? (Multiple responses possible/select all that apply) a. The training was not timely/did not come prior to my needing it b. The training was not comprehensive enough for me to adopt the practices taught c. There was insufficient follow-up after trainings to allow for me to ask questions d. Other, please specify________________________________________________ 40f. Is the member of your household who is part of the INVC Care Group the lead mother/father? a. No = 0 b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 40g. Have you or anyone else in your household participated in an INVC Farm Club? a. No = 0 (SKIP TO QUESTION 41) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 41) d. Refuse to respond = 9999 (SKIP TO QUESTION 41) 40h. When did you/he/she start participating in the INVC community Farm Club? a. In the past month b. Less than one year ago (but more than one month ago) c. Less than two years ago (but more than one year ago) d. Less than three years ago (but more than two years ago) e. More than three years ago f. Don’t know = 8888 g. Refuse to respond = 9999 40i. What type of support has this person/your household received from the Farm Club? 227 a. We have received soy seeds from the Farm Club b. We have received ground nut seeds from the Farm Club c. We have received an inoculant for soy bacteria d. We have received training on planting practices (planting in a row, seed spacing, etc.) e. We have received training on market pricing f. We have received training on how to write a business plan g. We have received reading materials for our children h. Other, please specify _____________________________________________ = 3 i. Don’t know = 8888 j. Refuse to respond = 9999 40k. Is the member of your household who is part of the Farm Club the lead farmer? a. No = 0 b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 40l. How often, if at all, have you experienced any of the following problems with the Farm Club in the past three years? (Multiple responses possible/select all that apply) Kodi m’zaka zitatu zapitazi mwakumanapo pafupipafupi bwanji ndi mavuto awa ku kalabu yanu ya zaulimi? a. The training was not timely/did not come prior to my needing it Kuchedwa kwa maphunziro a zaulimi Never = 0 Once or Twice = 1 A Few Times = 2 Often = 3 Don’t know = 8888 Refuse to respond = 9999 b. Training was not comprehensive enough for me to adopt the practices taught Never = 0 Once or Twice = 1 A Few Times = 2 228 Maphunziro a zaulimi sanali abwino kwenikweni Often = 3 Don’t know = 8888 Refuse to respond = 9999 c. Seeds were not delivered on time Mbeu sitinalandire nthawi yabwino Never = 0 Once or Twice = 1 A Few Times = 2 Often = 3 Don’t know = 8888 Refuse to respond = 9999 d. Quality of the seeds was low Mbeu sizinali zabwino Never = 0 Once or Twice = 1 A Few Times = 2 Often = 3 Don’t know = 8888 Refuse to respond = 9999 e. Other, please specify____________________ Never = 0 Once or Twice = 1 A Few Times = 2 Often = 3 Don’t know = 8888 Refuse to respond = 9999 229 41. Has your household received any other support, training, or assistance from any donor or nonprofit organizations in the past three years? Kodi banja lanu lalandirapo chithandizo kapena maphunziro kuchokera kwa ma donor kapena kumabungwe omwe sapanga phindu muzaka zitatu zapitazi? a. No = 0 (SKIP TO QUESTION 43) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 43) d. Refuse to respond = 9999 (SKIP TO QUESTION 43) 42. From which organizations did you receive support, assistance, or training? Chithandizochi/maphunzirowa anachokera ku bungwe liti? What year did they begin working with you, what type of support did they provide, has that support helped your household, and (if so) in what ways? Anayamba kugwira nanu ntchito muchaka chanji? Anapereka thandizo lotani? Nanga thandizolo lathandiza pakhomo pano munjira yanji? (Allow the respondent to list organizations, and write them in. But, if they don’t mention the ones listed below, ask specifically about them. Enter in column 4, the numbers for all ways listed – multiple responses possible.) Donor or Nonprofit Organization 1 - Year Support Began Chaka choyambira thandizo 2 - Type of Support Mtundu wa thandizo Use Codes Below 3 – Has this support helped your household (No = 0, Yes = 1, Don’t know = 99999999) Thandizoli lathandiza khomo lanu? 4 – In what ways (see codes below) Munjira yanji? Select all that apply; multiple responses possible A – INVC/FtF (Livelihoods/Agriculture) B – SSDI (Clinics/hospitals) C – WALA (Livelihoods/Agriculture) D – Millennium Villages E – CAMFED (Girls Scholarships) F – TIANA G - EGRA H – Literacy Boost I --ASPIRE J – World Bank K - UNICEF L – Irish Aid M – Concern International 230 J – Other, please specify______________ K – Other, please specify______________ 43. Does your family have a farm or grow any crops? Kodi pa banja pano muli ndi munda kapena mumalima mbeu zilizonse? a. No = 0 (Skip to question 52) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 52) d. Refuse to respond = 9999 (SKIP TO QUESTION 52) 44. How much land does your family farm? Kodi banja lanu limalima malo okula bwanji? a. _________ Amount (Enter “0” for none) b. _________Unit (Acres/Hectares/Football pitches) c. Don’t know = 8888 d. Refuse to respond = 9999 45. What types of crops does your family grow? Kodi banja lanu limabzyala mbeu zamtundu wanji? (Select all that apply; multiple responses possible) a. Maize Codes for 42-2: Financial assistance = 1 Health education = 2 Health/medical services (not including vaccinations)= 3 Vaccinations = 4 Nutritional training = 5 Agriculture inputs = 6 Agricultural training = 7 Business/micro-enterprise training = 8 Scholarships = 9 Cash transfers for education = 10 Cash transfers for something else = 11 (please specify in the space above) Maternal and child health assistance = 12 Another type of training not listed = 13 (please specify in the space above) Other = 13 (please specify in the space above) Codes for 42-4: It didn’t benefit our household at all = 0 It helped to increase our agricultural/dairy output = 1 It helped to improve our business = 2 It helped to increase our income = 3 It helped to improve our food security = 4 It helped to increase our nutrition = 5 It helped to improve our health = 6 It helpedto improve our education = 7 Other, please specify in the space = 8 231 b. Wheat c. Sorghum d. Groundnuts e. Cassava f. Sesame g. Tobacco h. Legumes/beans i. Millet j. Rice k. Sweet potatoes l. Yams m. Sugarcane n. Bananas o. Mangoes p. Eggplant q. Peppers r. Tomatoes s. Okra t. Greens u. Onions v. Potatoes w. Cabbage x. Other, please list___________________________ y. Don’t know z. Refuse to respond 232 46. Has the type of crops that your family grows changed in past two years? Kodi m’zaka ziwiri zapitazi mtundu wambeu zimene mumalima/mumabzyala wasitha? a. No = 0 (SKIP TO QUESTION 48) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 48) d. Refuse to respond = 9999 (SKIP TO QUESTION 48) 47. Why has the type of crops your family grows changed? Ndi chifukwa chiyani mbeu zimene banja lanu limalima zasintha? (Select all that apply; multiple responses possible) a. A NGO offered us new seed/told us to change crops b. A certain type of seed became easier/cheaper to access c. We gained access to additional land d. We gained access to a loan and were able to buy new seeds e. We were trained on the importance of crop diversification f. We wanted to add a new crop to the farm g. Rainfall patterns changed h. Don’t know i. Refuse to respond 48. Please estimate your production (in 50 kilogram bags) in your last harvest for each of the crops you mentioned: Chonde ganizirani ndi kunena mlingo wazokolora zanu (in 50 kg bags) mwakolora komaliza pa mbeu iliyonse itchulidweyi: a. Maize Chimanga____________ b. Wheat Tirigu____________ c. Sorghum Mapira____________ d. Groundnuts shelled Mtedza____________ e. Groundnuts unshelled Mtedza osaswa _________ 233 f. Cassava Chinangwa____________ g. Sesame ____________ h. Tobacco Fodya____________ i. Legumes/beans Za ntundu wa nyemba____________ j. Millet Nchewere____________ k. Rice Mpunga____________ l. Sweet potatoes Mbatata____________ m. Yams Zilazi/Mipama_______ n. Sugarcane Nzimbe____________ o. Bananas Nthochi____________ p. Mangoes Mango____________ q. Eggplant Mabiringanya____________ r. Peppers Tsabola____________ s. Tomatoes Tomato____________ t. Okra Therere____________ u. Greens Masamba ____________ v. Onions Anyezi ____________ w. Potatoes Mbatata____________ x. Cabbage Kabichi ____________ y. Soya Soya ____________ z. Other, please list___________________________ Amount____________ 49. Has your production increased or decreased overall in this year as compared to last year? Kodi zokolora zanu chaka chino zachuluka kapena kuchepa kuyerekeza ndi chaka chatha? a. Increased = 1 b. Stayed the same = 2 234 c. Decreased = 3 d. Don’t know = 8888 e. Refuse to respond = 9999 50. Did anyone in your household cultivate a dimba garden in [last completed dry season]? Kodi chaka chapitachi pabanja panu pano alipo amene analima mbeu za kudimba? a. No = 0 b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 SHOCKS SHOCKS 51. What types of shocks has your household experienced in the past year? Ndi ngozi za mtundu wanji zimene khomo lanu lino lakumana nazo muchaka chapitachi? 1 2 3 4 Did your household experience any of the following in the last 12 months? Kodi khomo lanu lino lakumanapo ndi zinthu monga izi mu miyezi 12 yapitayi? 0 = No GO TO NEXT EVENT 0 - No 1 = Yes Did this seriously affect your normal living conditions? Kodi izi zinakhudza kwambiri m’mene mumakhalira? 0 = No 1 = Yes What, if anything, did you do in response to this shock (such as borrow money, sell animals etc.)? Nanga munachita chiyani, ngati chilipo chimene munapanga pa nthawiyo, kuti methane ndi ngozi yadzidziyi (monga kubwereka ndalama, kugulitsa ziweto…) LIST UP TO 3 OPTIONS from following page What was the result of the shock? Chotsatira changozi yadzidzidziyi chinali chiyani? LIST UP TO 3 OPTIONS fromfollowing page CODE CODE i ii iii i ii iii a. Crop failure (due to flood, drought, or pest infestation) Kukanika kwa mbeu (chifukwa cha kusefukira kwa madzi, chilala kapena kugwa kwa tizirombo) 235 b. Inability to plant crop due to lack of seeds, lack of plowing services, insecurity. Kulephera kubzyala chifukwa chosowa mbeu, kusowa olima, kusowa chitetezo? b. Livestock died Kufa kwa ziweto c. Lost regular job Kutha kwa ncthito ya masiku onse d. Fire, theft or loss of property Moto, kuberedwa kapena kuluza katundu? e. Severe illness or injury Kudwala kwambiri kapena kuvulala f. Death of a household member Kumwalira kwa munthu wapakhomopo? g. Victim of violence/crime Kukhudzidwa ndi zipolowe/umbanda h. Any other event List: Chochitika chinachillichonse 236 RESULTS OF SHOCKS 1 = No results 2 = Child/children had to drop out of school to help support/ care for family or because fees/costs were too expensive 3 = Food insecurity/family had to reduce food/beverage consumption 4 = Family lost the house 5 = Children had to be absent more to work to support family or care for others 6 = Other, please specify in column CODES FOR RESPONSES TO SHOCKS 1 = No response 2 = Sold animals 3 = Sold jewelry or other assets 4 = Borrowed money 5 = Worked more at current job 6 = Started a new job 7 = Migrated 8. = Asked for help from a relative/friend 9 = Other, please specify 237 NUTRITION THIS SECTION SHOULD BE ANSWERED BY THE PERSON RESPONSIBLE FOR COOKING FOR THE HOUSEHOLD. PLEASE PROVIDE RESPONDENT ID FROM HOUSEHOLD ROSTER _____________ I would like to ask some questions about the food [LEARNER NAME] consumed yesterday. Please describe everything that [CHILD] had to eat yesterday during the day or the night, whether at home or outside the home. Pano ndikufunsani mafunso okhudza chakudya chimene [DZINA LA OPHUNZIRA] anadya dzulo. Chonde longosolani chilichonse chomwe [MWANA] anadya dzulo masana kapena usiku, kaya ndi panyumba pano kaya kunja kwa khomo lino. My first question is: Funso langa loyamba nali: 1. Think about when [CHILD] first woke up yesterday. Did [LEARNER’S NAME] eat anything at that time? IF NO >> (ii) IF YES: Please tell me everything that [CHILD] ate at that time. Mark a “1” next to each letter option that describes something the child ate. THEN PROBE: And anything else? UNTIL RESPONDENTSAYS NO. THEN >> (ii). Takumbukirani [DZINA LA OPHUNZIRA] atangodzuka kumene dzulo. Kodi [DZINA LA OPHUNZIRA] anadya kalikonse panthawi imeneyi? Ngati AYI >> (ii) Ngati INDE: Chonde ndiuzeni chilichonse chimene [DZINA LA OPHUNZIRA] anadya nthawi imeneyi? Mark a “1” next to each letter option that describes something the child ate. THEN PROBE: And anything else? UNTIL RESPONDENTSAYS NO. THEN ASK QUESTION 2. 2. When did [CHILD] next eat anything? What did [LEARNER’S NAME] eat at that time? THEN PROBE: And anything else? UNTIL RESPONDENT SAYS NO. Ndi nthawi iti ina imene [DZINA LA OPHUNZIRA] anadya kena kake? Kodi [DZINA LA OPHUNZIRA] anadya chiyani pa nthawi imeneyi? THEN PROBE: Kaliponso kena? UNTIL RESPONDENT SAYS NO. 3. REPEAT (2) UNTIL RESPONDENT SAYS [LEARNER’S NAME] WENT TO SLEEP UNTIL THE NEXT DAY 4. IF RESPONDENT MENTIONS MIXED DISHES LIKE A PORRIDGE, SAUCE OR STEW, PROBE: What ingredients were in that [MIXED DISH]? Kodi munasakanizamo chiyani muchakudya chimenechi? THEN PROBE: And anything else? China chili chonse? UNTIL RESPONDENT SAYS NO. 5. AS THE RESPONDENT RECALLS FOOD, UNDERLINE THE CORRESPONDING FOOD AND ENTER 1 IN THE COLUMN NEXT TO THE FOOD GROUP. IF FOODS ARE USED IN SMALL AMOUNTS FOR SEASONING OR AS A CONDIMENT, INCLUDE THEM UNDER THE CONDIMENTS FOOD GROUP. 6. ONCE THE RESPONDENT FINISHES RECALLING FOODS EATEN, READ EACH FOOD GROUP WHERE 1 WAS NOT ENTERED, ASK THE FOLLOWING QUESTION: “Yesterday, did [LEARNER’S NAME] eat or drink any [LIST SKIPPED FOOD ITEM]? NO…….- YES…..1 Dzulo, masana kapena usiku, kodi [CHILD] anadya/anamwa zina mwa [FOOD GROUP ITEMS]? 7. 7. ONLY ASK QUESTIONS 53, 54, and 55 if the respondent answers “No” to ALL Questions 52a – 52q. SEE TABLE ON NEXT PAGE NUTRTION CONTINUED 238 UTRITION CONTINUED mited variety of foodsst part of 6 as the question, but thefirst the tablet. Also, note that ally asks thatNUTRITION CONTINUED Other foods: please write down other foods (to the right of this box) that respondent mentioned but are not in the list below. This will allow the survey supervisor or other knowledgeable individuals to classify the food later. No = 0, Yes = 1, Don't know = 8888, Refuse to respond = 9999 52a Food made from grains, such as bread, rice, noodles, pasta, porridge, or [other local grain food] Zakudya zopangandidwa kuchokera ku mbewu za mgulu la chimanga, monga bredi, mpunga wophikaphika, ma,noodle, phala etc 52b Pumpkin, carrots, or sweet potatoes that are yellow or orange inside or [other local yellow/orange foods] Maungu, karoti, mphonda, mbatata zomwe zimakhala za ntundu wa chikasu (yelo) kapena wa orenji mkati? [Zina zakudya zamakolo za ntundu wa yelo kapena wa orenji mkati]? 52c White potatoes, white yams, manioc, cassava, [other local root crops] or any other foods made from roots Mbatata zoyera mkati, zilazi zoyera, manioc, chinangwa [zokolora zamizu] kapena zakudya zopangidwa kuchokera ku mizu? 52d Any dark green leafy vegetables such as [local dark green leafy vegetables] Zakudya zochokera kumasamba obiriwira kwambiri [zopezeka wamba za masamba obiriwira kwambiri]? 52e Ripe mangoes, ripe papayas or [other local vitamin A-rich fruits – usually orange in color] Mango okupsya, mapapaya okupsya [zipatso zopezeka mosavuta zomwe zili ndi vitamin A] 52f Any other fruits or vegetables zipatso zilizonse kapena masamba 52g Liver, kidney, heart, or other organ meats Chiwindi, imphyo, mtima kapena ziwalo zina za nyama 52h Any meat, such as beef, pork, lamb, goat, chicken, or duck Nyama iliyonse monga yang’ombe, nkhumba, mbuzi, nkhuku kapena bakha 52i Eggs Mazira 52j Fresh or dried fish, shellfish, or seafood Nsomba za fuleshi kapena zouma, shellfish kapena zakudya za mnyanja ya mchere 52k Any foods made from beans, peas, lentils, nuts, or seeds such as [local food names] 239 FOOD SECURITY/HUNGER – Now, I would like to ask about your household’s food security. Zakudya zilizonse zopangidwa kuchokera ku mtundu wanyemba (beans, peas, lentils, nuts, or seeds) monga nyemba, khobwe, nandolo, mtedaza, kabaifa etc 52l Cheese, yogurt, or other milk products Zakudya zopangidwa kuchokera ku mkaka monga cheese, yogati, ndi zina? 52m Any oil, fats, or butter, or foods made with any of these Zakudya zamafuta monga mafuta amene, bata kapena zopangidwa kuzamafuta? 52n Any sugary foods such as chocolates, sweets, candies, pastries, cakes, or biscuits Zakudya zotsekemera monga chokoleti, maswiti, makeke, pastry ndi mabisiketi? 52o Condiments for flavor, such as chilies, spices, herbs, or fish powder Zokometsera monga tsabola, herbs ndi pawudala wochokera ku nsomba? 52p Grubs, snails or insects Grubs, nkhono ndi zouluka zing’onozing’ono? 52q Foods made with red palm oil, red palm nut, or red palm nut pulp sauce Zakudya zochokera ku mafuta a 53 Did [child’s name] eat any solid, semi-solid, or soft foods yesterday during the day or at night? Kodi [Child’s name] anadya chakudya cholimba, cholimba pang’ono pena, chofewa dzulo nthawi yamasana kapena nthawi yausiku? (If no, SKIP TO 56) 54 IF ‘YES’ PROBE: What kind of solid, semi-solid, or soft foods did [child’s name] eat? (If none or refuse, SKIP TO 56) Ngat Inde, chinali chakudya chantundu wanji chimene [Child’s name] anadya? 55 How many times did [child’s name] eat solid, semi-solid, or soft foods other than liquids yesterday during day or at night Ndikangati kamene [Child’s name] anadya chakudya cholimba, cholimba pang’ono, chofewa dzulo nthawi yamasana kapena nthawi yausiku? 56. Now I will ask you several questions about your household’s food security. Please answer by telling the frequency with which each of the following conditions occurred (never, rarely, sometimes, or often). You can also tell me the exact number of times a specific incidence occurred. Tsopano ndikufunsani mafunso okhuza chakudya pakhomo panu pano. Chonde mundiuze kuti izi zimachitika pafupipafupi bwanji.(sizinachitikepo, mwakanthawi, nthawi zina, or kawirikawiri). Muthanso kundiuza kuti ndipafupipafupi bwanji pamene izi zachitika? 0 1 2 3 No Rarely (once or twice) Sometimes (3 to 10 times) Often (more than 10 times) a. In the past 4 weeks, did you worry that your household would not have enough food? M’masabata anayi apitawa, kodi munadandaulapo kuti pakhomo panu pano simukhala ndi chakudya chokwanira?     240 b. In the past 4 weeks, were you or any household member not able to eat the variety or kinds of foods you preferred? M’masabata anayi apitawa, kodi inu kapena pali wina wa pakhomo panu pano amene anadandaulapo kuti simukhala ndi chakudya chokwanira chifukwa cha kuchepekedwa?     c. In the past 4 weeks, did you or any household member have to eat some foods that you really did not want to eat because of a lack of resources to obtain other types of food? M’masabata anayi apitawa, kodi inu kapena pali wina wa pakhomo panu pano amene anadya chakudya chomwe iye samafuna kudya chifukwa cha kuchepekedwa kupeza chakudya chantundu wina?     d. In the past 4 weeks, did you or any household member have to eat fewer and/or smaller meals than you felt you needed because there was not enough food? M’masabata anayi apitawa, kodi inu kapena pali wina wa pakhomo panu pano amene anadya chakudya cha mlingo wochepa ndi m’mene amayenera kudyera chifukwa panalibe chakudya chokwanira?     241 FOOD SECURITY/HUNGER CONTINUED e. In the past 4 weeks, was there ever no food to eat of any kind in your household because of lack of resources to get food? M’masabata anayi apitawa, kodi pakhomo panu pano panalibiliretu chakudya chifukwa chosowa zokuyenerezani kuti mupeze chakudyacho?     f. In the past 4 weeks, did you or any household member go to sleep at night hungry because there was not enough food? M’masabata anayi apitawa, kodi inu kapena pali wina wa pakhomo panu pano amene anagona usiku osadya chifukwa choti kunalibe chakudya chokwanira?     g. In the past 4 weeks, did you or any household member go a whole day and night without eating anything because there was not enough food? M’masabata anayi apitawa, kodi inu kapena pali wina wa pakhomo panu pano amene anakhala tsiku lonse kapena usiku onse osadya chifukwa choti kunalibe chakudya chokwanira?     h. How do the last 4 weeks compare to the rest of the year in terms of food availability for your household? Mungafananizire bwanji mmasabata anayi apitawa ndi chaka chonse chapitachi pankhani yakapezekedwe ka chakudya pakhomo panu pano? 1=Last 4 weeks have been: BETTER 2=WORSE 3=SAME i. During the last 12 months, in which months was the food shortage most acute? Mumiyezi khumi ndi iwiri yapitayi, ndi miyezi iti imene munasowa chakudya kwambiri? MONTH (NUMBER) - LIST UP TO THREE IN RANK ORDER WITH THE WORST FIRST 242 243 HEALTH BACKGROUND ON LEARNER ASSESSED – This section should be answered by the adult who is most familiar with [LEARNER’S NAME]’s personal and educational background. PLEASE PROVIDE RESPONDENT ID FROM HOUSEHOLD ROSTER________. For the next few questions, I want to know more about [LEARNER’S NAME] and what he/she typically eats/how often he/she is sick. 57. Has [LEARNER’S NAME] gotten vaccinations? Kodi [DZINA LA OPHUNZIRA] analandirapo katemera? a. No = 0 b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 58. How many vaccinations has he/she received? Kodi [DZINA LA OPHUNZIRA] walandira katemera mungati? a. 1-3 = 1 b. 4-6 = 2 c. 7-10 = 3 d. Don’t know = 8888 e. Refuse to respond = 9999 59. When did he/she get the vaccinations? (Write 8888 for don’t know or 9999 for refuse to respond) Kodi [DZINA LA OPHUNZIRA] analandira katemera liti? Month:_______________ Year:______________ 60. Who provided the vaccinations? Kodi katemerayi anapereka ndani? a. A clinic = 1 b. The local Health Service Center = 2 c. The local dispensary = 3 d. A local health surveillance assistant (HSA) = 4 e. The hospital = 5 f. A traditional village healer = 6 g. Other, please specify______________________________________________= 7 h. Don’t know = 8888 i. Refuse to respond = 9999 244 61. Does [LEARNER’S NAME] have access to other child health services? Kodi [DZINA LA OPHUNZIRA] amalandira chithandizo chinanso chokhuza umoyo wa ana? a. No = 0 (Skip to question 81) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 63) d. Refuse to respond = 9999 (SKIP TO QUESTION 63) 62. What are these other child health services? (Select all that apply; multiple responses possible) Kodi amalandira chithandizo chanji? a. Dental care =1 b. Vision care =2 c. Vitamins =3 d. Worm medication =4 e. Malaria medication =5 f. Other =5 please specify_________________________________________________ g. Don’t know =6 h. Refuse to respond =7 63. Has [LEARNER’S NAME] been sick in the past month? Kodi [LEARNER’S NAME] anadwalapo mu mwezi wapitawu? a. No = 0 (SKIP TO QUESTION 63) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 63) d. Refuse to respond = 9999 (SKIP TO QUESTION 63) 64. How many days, if any, did [LEARNER’S NAME] stay home from school this past month because he/she was sick? Ndi masiku angati, ngati alipo amene [LEARNER’S NAME] anajomba kusukulu chifukwa choti amadwala? a. One = 1 b. Two = 2 c. Three = 3 d. Four = 4 e. Five = 5 245 f. 6 -10 days (2 weeks) = 6 g. 11-15 days (3 weeks) = 7 h. 16-20 days (4 weeks) = 8 i. More than 20 days (4 weeks) = 9 j. Don’t know = 8888 k. Refuse to respond = 9999 65. What type(s) of illness did [LEARNER’S NAME] have? Kodi [LEARNER’S NAME] amadwala matenda anji? (Select all that apply; multiple responses possible) a. Cold b. Flu c. Diarrhea d. Headache e. Malaria f. Tuberculosis g. Strep throat h. Bronchitis i. Pneumonia j. Other, please specify____________________________________ k. Don’t’ know = 8888 l. Refuse to respond = 9999 66. Did [LEARNER’S NAME] attend a clinic/Health Service Center/hospital when he/she became ill? Kodi [LEARNER’S NAME] anapita kuchipatala panthawi imene amadwala? a. No = 0 b. Yes = 1 (SKIP TO QUESTION 68) c. Don’t know = 8888 (SKIP TO QUESTION 68) d. Refuse to respond = 9999 (SKIP TO QUESTION 68) 67. Why didn’t [LEARNER’S NAME] attend a clinic/Health Service Center/hospital? Ndi chifukwa chiyani [LEARNER’S NAME] sanapite kuchipatala? (Select all that apply; multiple responses possible) a. I thought he/she would get better without medicine/assistance (he/she wasn’t very ill) b. He/she went to a traditional healer 246 c. A village health worker or Health Surveillence Assistant (HSA) came to our house d. The closest one is too far away e. It is too expensive f. He/she didn’t want to go g. There was nobody to take him/her h. Other, please specify__________________________________________ i. Don’t know j. Refuse to respond LEARNER SCHOOLING – Now, I would like to talk about [LEARNER’S NAME]’s schooling. 68. How long did [LEARNER’S NAME] attend a preschool, kindergarten, nursery, reception, or early childhood development school? Ndi nthawi yayitali bwanji imene [LEARNER’S NAME] anali kusukulu yamkaka? a. Never = 0 b. A few months = 1 c. One year = 2 d. Two years = 3 e. Three or more years = 4 f. Don’t know = 8888 g. Refuse to respond = 9999 69. How old was [LEARNER’S NAME] when he/she first attended Standard 1? Kodi [LEARNER’S NAME] anali ndi zaka zingati pamene amakayamba stitandade 1? a. Age__________ (If 6 or younger, SKIP TO QUESTION 72) b. Don’t know = 8888 (SKIP TO QUESTION 72) c. Refuse to respond = 9999 (SKIP TO QUESTION 72) 70. How long has [LEARNER’S NAME] been attending his/her current school? Kodi [LEARNER’S NAME] wakhala akupita kusukulu ya[SCHOOL NAME] kwa nthawi yayitali bwanji? a. Years_________ Months ________ (If more than 4 years, skip to question 72 b. Don’t know = 8888 c. Refuse to respond = 9999 71. What other primary school(s) did he/she attend? Ndi sukulu zina ziti za pulayimale zimene [LEARNER’S NAME] anaphunzirako? a. None/Not applicable 247 b. School name___________________________________ c. Don’t know = 8888 d. Refuse to respond = 9999 72. What is the mode used by the child to go to school most of the days? Kodi mwana wanu amayenda bwanji popita kusukulu? a. Walk b. Bicycle (by the child; some family member gives a ride; bike taxi) c. Bus d. Other, please specify __________________ e. Refuse to respond 73. How long does it take for him or her using the mode above to go to school? Kodi [LEARNER’S NAME] amatenga nthawi yaitali bwanji kukafika kusukulu? a. 0-20 minutes = 1 b. 21-40 minutes = 2 c. 41-60 minutes = 3 d. 61 minutes-90 minutes (1.5 hours) = 4 e. 1.5-3 hours f. More than 3 hours = 5 g. Don’t know = 8888 h. Refuse to respond = 9999 74. What do you expect learners from [LEARNER’S NAME’S] school to be able to accomplish after completing Standard 2? (Read each option. Multiple responses possible; select all that apply) Kodi mumayembekezera chiyani kwa ophunzira amene amaliza sitandard 2 pa sukulu imene [DZINA LA OPHUNZIRA] amapita? a. They should be able to recite the alphabet b. They should know the sounds of letters c. They should be able to understand short stories that are read to them d. They should be able to sound out simple words e. They should be able to read short sentences f. They should be able to read short, simple stories g. They should be able to read longer, more complext stories 75. What do you expect learners from [LEARNER’S NAME’S] school to be able to accomplish after completing Standard 4? (Read each option. Multiple responses possible; select all that apply) Kodi mumayembekezera chiyani kwa ophunzira amene amaliza sitandard 4 pa sukulu imene [DZINA LA OPHUNZIRA] amapita? a. They should be able to recite the alphabet b. They should know the sounds of letters 248 c. They should be able to understand short stories that are read to them d. They should be able to sound out simple words e. They should be able to read short sentences f. They should be able to read short, simple stories g. They should be able to read longer, more complex stories • 76. If [LEARNER’S NAME’S] does not meet these learning expectations, what do you think would be the reasons? (Don’t read these options - ONLY check those offered. Select all that apply; multiple responses possible) Mukuganiza kuti zifukwa zake zingakhale chiyani kuti [DZINA LA OPHUNZIRA] asakwanitse chiyembekezo chanu? a. He/she is too busy at home to do homework b. He/she does not study hard enough c. The child is not receiving enough support from home (with homework, reading, etc.) d. He/she is too hungry to concentrate e. He/she is not smart f. He/she misses too many days of school g. The teacher is doing a poor job of teaching h. The teacher seldom comes to school i. The classrooms are too crowded j. He/she doesn’t have the textbooks k. He/she doesn’t pay attention in class l. He/she doesn’t understand what the teacher is saying m. Other, please specify_________________________________________________ n. Don’t know o. Refuse to respond 77. If [LEARNER’S NAME’S] does not meet these learning expectations, what would you do, if anything? (Select all that apply; multiple responses possible) Mungatani, ngati [DZINA LA OPHUNZIRA] sangakwanitse chiyembekezo chanu, ngati pali chilichonse chimene mutha kuchita? a. Nothing b. Talk with teachers c. Talk with the head teacher d. Complain to the PTA e. Punish the child f. Offer to assist in the classroom g. Work with the child on his/her studies h. Pay for a tutor i. Withdraw him/her from school j. Other, please specify_________________________________________________ 78. Has [LEARNER’S NAME’S] repeated a grade? Kodi [LEARNER’S NAME] wabwerezapo kalasi iliyonse? a. No = 0 (SKIP TO QUESTION 80) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 80) 249 d. Refuse to respond = 9999 79. Which grade(s) was it/ were they and why? Anabwereza makalasi anji ndipo ndi chifukwa chiyani? Standard 1 - Repea ted (No = 0, Yes = 1) 2 – Number of years repeated (including current year, if applicable) 3 – Reason for repeating (see codes below; multiple selections possible) A - Standard 1 B - Standard 2 C - Standard 3 D - Standard 4 CODES for 79-3 265 Too many absences = 1 Poor quality teaching = 2 Classroom was too crowded = 3 No/not enough textbooks = 4 Child isn’t smart = 5 Child didn’t study/pay attention = 6 I didn’t know how to help him/her = 7 I didn’t have time to help him/her = 8 He/she was too hungry to learn = 9 Teacher didn’t like him/her = 10 Child didn’t sit for the exam = 11 Don’t know = 8888 Refuse to answer = 9999 80. Did [LEARNER’S NAME] miss one or more days of school in the past four weeks? Kodi [DZINA LA OPHUNZIRA] wajombapo kusukulu m’masabata anayi apitawa? a. No = 0 (SKIP TO QUESTION 82) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 82) d. Refuse to respond = 9999 (SKIP TO QUESTION 82) 81. Why did [LEARNER’S NAME] miss some school in the past four weeks? (Select all that apply; multiple responses possible) Ndi chifukwa chiyani [LEARNER’S NAME] anajomba ku sukulu masiku ena masabata anayi apitawa? (Select all that apply; multiple responses possible) a. He/she needed to stay home to complete domestic chores such as helping to care for younger children or elderly or sick relatives, cooking, cleaning, fetching water or wood, etc. [LEARNER’S NAME] amafunika kuti athandize ntchito zapakhomo pano monga kusamalira ana, okalamba kapena achibale amene akudwala, kuphika, kukonza pakhomo, kukatunga madzi, kukatola nkhuni. b. He/she needed to tend animals or work on the family farm or in the family business. [LEARNER’S NAME] amafunika koweta ziweto, kukagwira ntchito kumunda kapena kubizinesi. c. He/she did not want to go to school. [LEARNER’S NAME] sanafune kupita kusukulu. d. He/she was ill/sick. [LEARNER’S NAME] amadwala e. He/she needed to attend a funeral. [LEARNER’S NAME] anapita kumaliro. f. He/she was too hungry to go to school. [LEARNER’S NAME] anali ndi njala kwambiri oti sakanatha kupita kusukulu. g. He/she did not have any clothes to wear or his/her clothes were dirty. [LEARNER’S NAME] analibe zovala zina zoti nkuvala chifukwa zinali zakuda. h. He/she missed school for another reason; [LEARNER’S NAME] anajomba chifukwa cha zifukwa zina, chonde longosolani, please specify) 266 ______________________________________________________________ i. Don’t know j. Refuse to respond 82. How important do you think it is for the boys in your household to go to school? A bwanji Kodi mukuganiza kuti ndikofunikira bwanji kuti anyamata a pakhomo lanu lino adzipita kusukulu? a. It’s useless = 1 b. It’s not important = 2 c. It’s somewhat important = 3 d. It’s important = 4 e. It’s very important = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 83. How many years of schooling do you think boys should have? Kodi mukuganiza kuti anyamata amayenera kuphunzira sukulu mpaka kalasi yanji? (Enter number of years)_____________ 84. Why this many years and not more/less? (Select all that apply; multiple responses possible). Chifukwa chiyani mpaka kalasi imeneyi? a. They'll know how to read by then b. They need to begin working c. They probably won't get into secondary school/university d. They don't learn very much at school e. They don't like being in school f. They'll earn more income the more years they go to school (or can have a profession and be more successful) g. They'll get a better husband/wife h. They'll be a better parent/their children will be better educated i. They'll be better able to help support us when we're old/support (contribute to) the household/family or siblings' education j. Other, please specify_________________________________________________ k. Don't know l. Refuse to answer 85. How important do you think it is for the girls in your household to go to school? Kodi mukuganiza kuti ndikofunikira bwanji kuti atsikana a pakhomo lanu lino adzipita kusukulu? a. It’s useless = 1 b. It’s not important = 2 c. It’s somewhat important = 3 d. It’s important = 4 e. It’s very important = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 86. How many years of schooling do you think girls should have? Kodi mukuganiza kuti atsikana amayenera kuphunzira sukulu mpaka kalasi yanji? 267 (Enter number of years)_____________ 87. Why this many years and not more/less? (Select all that apply; multiple responses possible). Chifukwa chiyani mpaka kalasi imeneyi? a. They'll know how to read by then b. They need to begin working c. They probably won't get into secondary school/university d. They don't learn very much at school e. They don't like being in school f. They'll earn more income the more years they go to school (or can have a profession and be more successful) g. They'll get a better husband/wife h. They'll be a better parent/their children will be better educated i. They'll be better able to help support us when we're old/support (contribute to) the household/family or siblings' education j. Other, please specify_________________________________________________ k. Don't know l. Refuse to answer 88. How much responsibility do you feel parents (or guardians) have in ensuring that children learn and do well in school? Kodi mukuganiza kuti makolo ali ndi udindo wotani pakuonetsetsa kuti ana awo akuphunzira ndipo akuchita bwino kusukulu? a. No responsibility = 1 b. Very little responsibility = 2 c. Some/a medium amount of responsibility = 3 d. Quite a bit of responsibility = 4 e. A lot of responsibility = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 • 88a. I’m going to read you a list of things that may influence a learner’s success in school. I would like you to tell me how important you think each of these is to a learner’s success. Please rate the items on the list I will read to you on a scale from 1-3, with 1 being NOT AT ALL IMPORTANT, 2 being SOMEWHAT IMPORTANT and 3 being VERY IMPORTANT. ARE YOU READY? Ndikuwerengerani zinthu zosiyanasiyana zimene zitha kupangitsa kuti ophunzira achite bwino kusukulu. Ndikufuna mundiwuze kufunikira kwa zinthuzi ndi kotani kuti ophunzira achite bwino. Ngati zili zofunikira kwambiri mundiwuze kuti 3; ngati zili zofunikira pang’ono mundiwuze kuti 2; ngati zili zosafunikira mundiwuze kuti 1. (Read each option and wait for the respondent to provide a rating) a. Teachers attending school regularly Aphinzitsi osajombajomba kusukulu____ b. Parents or someone in the household reading to the learner most days Makolo kapena wina aliyense wapakhomo kumuwerengera ophunzira kawirikawiri____ c. How hard the learner works at his/her studies Kulimbikira kwa ophunzira pa maphunziro ake____ d. The quality of teaching Kaphunzitsidwe ka aphunzitsi____ 268 e. The learner having a textbook for each subject Ophunzira kukhala ndi bukhu la phunziro lililonse____ f. Parents (or other household members) teaching the learner the alphabet Makolo (kapena wina aliyense wa pakhomo) kuphunzitsa ophunzira kuwerenga zilembo____ g. Having books in the home Kukhala ndi mabukhu kunyumba____ h. Parents (or someone in the household) being involved in the school Makolo (kapena wina aliyense wa pakhomo) kutenga nawo mbali pazochitika zakusukulu____ i. Parents (or someone in the household) making sure the child goes to school every day and arrives on time Makolo (kapena wina aliyense wa pakhomo) kuwonetsetsa kuti mwana akupita kusukulu tsiku lililonse komanso nthawi yabwino____ j. Parents/guardians expecting the child to do well in school Makolo (kapena wina aliyense wa pakhomo) kukhala ndi chiyembekezo kuti ophunzira azichita bwino ku sukulu____ k. The number of learners in a classroom Nambala ya ana m’kalasi ____ l. How well a learner can read Kudziwa kuwerenga kwa ophunzira____ m. Teachers who are well trained Aphunzitsi ophunzitsidwa bwino ntchito yawo____ n. Someone in the household helping the child with homework Wina aliyense wa pakhomo kuthandiza ophunzira kuchita homework____ 89. What are the things you (or someone in your household) do or have done to help [LEARNER’S NAME] learn? (Don’t read the options but check all options the respondent offers; multiple responses possible) Kodi mwachitapo (inuyo kapena wina aliyense wapakhomo pano) chiyani kuti [DZINA LA OPHUNZIRA] aphunzire? a. Help with their homework b. Buy or borrow books for them to read c. Take them to the library d. Take them to a reading event e. Talk with their teacher or head teacher about the child’s learning progress f. Participate in the PTA g. Participate in the School Committee h. Regularly read to the child (can be in the past when the child was younger) i. Encourage child to read j. Communicate to your child that you have high expectations for him/her 90. Does [LEARNER’S NAME] ever do homework outside of school? Kodi [LEARNER’S NAME] amalemba homework akaweluka kusukulu? a. No = 0 (SKIP TO QUESTION 93) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 93) d. Refuse to respond = 9999 (SKIP TO QUESTION 93) 91. About how many hours per week does [LEARNER’S NAME] spend doing homework outside of school? [LEARNER’S NAME] amalemba homework akaweluka kusukulu pafupifupi kwa maola angati pa sabata?______________________________________ 269 92. Do you or anyone else in the ever help [LEARNER’S NAME] with his/her homework? If so, how often? Kodi inu kapena wina aliyense wapakhomo pano amamuthandiza [LEARNER’S NAME] kulemba homework? Mumamuthandi kawirikawiri, nthawi zina, kapena mwakanthawi? a. No = 0 b. Yes, rarely = 1 c. Yes, sometimes = 2 d. Yes, frequently = 3 e. Don’t know = 8888 f. Refuse to respond = 9999 93. Are there any books, magazines, etc. that [LEARNER’S NAME] can read at home? Kodi [LEARNER’S NAME] ali ndi mabuku, magazine etc amene amatha kuwerenga kunyumba? a. No = 0 (should skip to 98, right?) b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 94. How often, if ever, does [LEARNER’S NAME] Does anyone in the household read books, magazines, or newspapers on a regular basis? Alipo wina pakhomo pano amene amawerenga mabuku, nyuzi, kapena magazini pafupipafupi? a. No = 0 b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 95. Has anyone in your household ever read to [LEARNER’S NAME]? (Including family members who no longer live in the household) Alipo wina pakhomo pano amene amamuwerengera [LEARNER’S NAME]? a. No = 0 (SKIP TO QUESTION 98) b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 96. How often does someone usually read to [LEARNER’S NAME]? Ndipafupipafupi bwanji pamene amamuwerengera [LEARNER’S NAME]? a. Nobody reads to him/her anymore = 1 270 b. Once a month = 2 c. A few times a week = 3 d. Once a week = 4 e. More than once a week = 5 f. Don’t know = 9999 g. Refuse to respond = 9999 97. How old was [LEARNER’S NAME]? when someone in this household begin to read to him/her? Kodi [LEARNER’S NAME’s] anali ndi zaka zingati pamene wina wapakhomo pano anayamba kumamuwerengera? a. ___________age b. Don’t know = 8888 c. Refuse to respond = 9999 98. Does [LEARNER’S NAME] ever bring any books home from school? Kodi [LEARNER’S NAME’s] amabweretsako mabukhu kunyumba? a. No = 0 b. Yes = 1 c. Don’t know = 8888 d. Refuse to respond = 9999 99. COSTS FOR LEARNER TO ATTEND SCHOOL Now, I would like to discuss how your household spent for [LEARNER’S NAME] to attend school this year. Specifically, I would like to know how much you spent on: Mu chaka chasukulu chapitachi, kodi banja lanu lagwirtsa ntchito ndalama zingati pa zinthu izi? MK a. …shoes for [LEARNER’S NAME] to attend school? Nsapato kuti [LEARNER’S NAME’S] athe kupita kusukulu? b. …school uniforms and/or clothing, for [LEARNER’S NAME] to attend school? Yunifolomu kapena zovala kuti [LEARNER’S NAME’S] athe kupita ku sukulu? c. …textbooks for [LEARNER’S NAME]? Mabukhu a [LEARNER’S NAME] d. …pens or pencils for [LEARNER’S NAME] to attend school? Mabolopointi/mapensulo kuti [LEARNER’S NAME’S] athe kupita ku sukulu? e. …exercise books for [LEARNER’S NAME] to attend school? Makope kuti [LEARNER’S NAME’S] athe kupita ku sukulu? 271 f. …a school bag for [LEARNER’S NAME] to attend school? Chikwama chonyamulira makope kuti [LEARNER’S NAME’S] athe kupita ku sukulu? g. …part-time lessons h. …food or snacks for [LEARNER’S NAME] school? Zakudya zotenga/kugula ku sukulu kuti [LEARNER’S NAME] athe kupita ku sukulu? i. …transportation [LEARNER’S NAME] to and from school? Transipoti kuti [LEARNER’S NAME’S] athe kupita ku sukulu? j. …other costs for [LEARNER’S NAME] to attend school? Ndalama zina zilizonse kuti [LEARNER’S NAME] athe kupita ku sukulu? 100. About how many hours per day does [LEARNER’S NAME] spend doing chores/work around the house/compound? Ndi maola angati pa tsiku amene [LEARNER’S NAME] amagwira ntchito? a. ______ Hours b. Don’t know = 8888 c. Refuse to respond = 9999 101. Does [LEARNER’S NAME] work to earn money during the school year? Kodi [LEARNER’S NAME] amagwira ntchito nthawi yoti sukulu atsegulira ndipo ili mkati? a. No = 0 (SKIP TO QUESTION 104) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 104) d. Refuse to respond = 9999 (SKIP TO QUESTION 104) 102. What type of work does [LEARNER’S NAME] do? Kodi [LEARNER’S NAME] amagwira ntchito yamtundu wanji? a. Running family shop = 1 b. Selling small goods on street = 2 c. Begging = 3 d. Fishing = 4 e. Other, please specify________________________________________ = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 103. How many hours per day does [LEARNER’S NAME] work? Ndi maola angati pa tsiku amene [LEARNER’S NAME] amagwira ntchito? 272 d. ______ Hours e. Don’t know = 8888 f. Refuse to respond = 9999 COMMUNITY-SCHOOL INVOLVEMENT IN EDUCATION 104. Does [LEARNER’S NAME’S] school have a PTA or School Committee? Kodi pasukulu yomwe [LEARNER’S NAME’S] amaphunzira pali PTA kapena komiti yothandiza kuyendetsa sukuluyo (SMC)? a. No = 0 (SKIP TO QUESTION 107) b. Yes, a PTA = 1 c. Yes, a School Committee = 2 (SKIP TO QUESTION 106) d. Yes, both = 3 e. Don’t know = 8888 (SKIP TO QUESTION 107) f. Refuse to respond = 9999 (SKIP TO QUESTION 107) 105. Please describe the types of things the PTA at [LEARNER’S NAME’s] school does? Longosolani mitundu ya ntchito zimene a PTA amapanga pa sukulu yomwe [LEARNER’S NAME’S] amaphunzira? (Read the response options to the respondent. Select all that apply; multiple responses possible) a. Monitors teacher absences b. Buys, or raises money to buy learning materials (other than books) for the school c. Buys books for the classrooms or raises money to buy books d. Reads to learners e. Provides tutoring for learners who are having difficulty learning to read f. Tries to motivate the community to get involved in supporting the school g. Raises money and/or encourages parents and/or community members to repair/maintain the school and/or build new classrooms or teacher housing h. Helps organize book fairs i. Hosts after-school book clubs j. Works with the school staff to find ways to improve the school and the teaching￾learning process k. Helps set policy l. Other, please specify_________________________________________________ m. Don’t know n. Refuse to respond 106. What types of things does the School Committee at (LEARNER’S NAME) do? Longosolani mitundu ya ntchito a Komiti ya Sukulu a pasukulu yomwe [LEARNER’S NAME’S] amaphunzirapo amapanga? (Let them respond on their own for about 20-30 seconds and then ask about the items below for those they haven’t already given. Report all that apply; multiple responses possible) a. Monitors teacher absences b. Buys, or raises money to buy learning materials (other than books) for the school c. Buys books for the classrooms or raises money to buy books d. Reads to learners e. Provides tutoring for learners who are having difficulty learning to read f. Tries to motivate the community to get involved in supporting the school 273 g. Raises money and/or encourages parents and/or community members to repair/maintain the school and/or build new classrooms or teacher housing h. Helps organize book fairs i. Hosts after-school book clubs j. Works with the school staff to find ways to improve the school and the teaching￾learning process k. Helps set policy l. Other, please specify_________________________________________________ m. Don’t know n. Refuse to respond 107. Do you or others in the household feel welcome in (LEARNER’S NAME) school? Kodi inuyo kapena wina aliyense wapakhomo pano amamva kulandiridwa ku sukulu ya [DZINA LA OPHUNZIRA] a. No = 0 b. Yes = 1 (SKIP TO QUESTION 109) c. I/We have never gone to his/her school (skip to y) = 2 (SKIP TO QUESTION 109) d. Don’t know = 8888 (SKIP TO QUESTION 109) e. Refuse to answer = 9999 (SKIP TO QUESTION 109) 108. . Why do you or they not feel welcome in (LEARNER’S NAME) school? Ndi chifukwa chiyani inuyo kapena wina aliyense wapakhomo samamva kulandiridwa ku sukulu ya [DZINA LA OPHUNZIRA] (Don’t read options, but record all options they give; multiple responses possible) a. Because I/we can’t read b. Because I/we don’t know anything about schools – or I never went to school c. Because the teachers and head teachers at the school don’t want me/us there d. Because education is best left to the educators e. Because I don’t have time f. I can’t think of any way I can be helpful or make a difference g. I’d be involved if someone told me how I could be helpful h. Other, please list________________________________________________________ i. Don’t know j. Refuse to respond 109. Have you received support from the Early Grade Reading Activity (EGRA)? Kodi sukulu ya [DZINA LA OPHUNZIRA] ikutenga nawo mbali pa maphunziro othaniza kuti ophunzira adzitha kuwerenga m’makalasi angónoangóno, EGRA? a. No = 0 (Skip to Question 111) b. Yes =1 c. Don’t know = 8888 d. Refuse to respond = 9999 (Skip to Question 111) 110. What type of support have you or other community members received from EGRA? Kodi sukulu ya [DZINA LA OPHUNZIRA] ikuchita chiyani kapena yasintha chiyani chifukwa cha maphunziro amenewa, EGRA? a. EGRA has provided support to the school but not to our household or parents or community members = 1 274 b. EGRA has provided support to the PTA; please specify what type of support_____ • ________________________________________________________________________ _________________________________________________________ = 2 c. EGRA has provided support to the School Management Committee (SMC); please specify what type of support___________________________________________ • ________________________________________________________________________ _________________________________________________________ = 3 d. EGRA has encouraged me and other parents/guardians in our neighborhood to get involved in [LEARNER’S NAME’S] school = 4 e. EGRA has encouraged me and other parents/guardians in our neighborhood to read to [LEARNER’S NAME’S] = 4 f. EGRA has provided books for [LEARNER’S NAME’S] to bring home from school = 5 g. EGRA has sent me or another household member text/(SMS) messages related to school involvement or reading = 6 h. EGRA has started or provided support to start a community reading center for learner’s from [LEARNER’S NAME’S] school = 7 i. EGRA held or provided support for a reading fair at [LEARNER’S NAME’S] school = 8 j. EGRA trained village leaders on community sensitization to reading = 9 k. Other, please specify __________________ __________________________=10 l. Don’t know = 8888 m. Refuse to respond = 9999 111. Have you and/or any member of your family ever been invited to or asked to be involved in [LEARNER’S NAME’S] school in any way? Kodi inu kapena wina aliyense wa m’banja lanu anayitanidwapo kapena kupemphedwapo kutenga nawo mbali ku sukulu ya [LEARNER’S NAME’S] munjira ina iliyonse? a. No = 0 (SKIP TO QUESTION 113) b. Yes = 1 c. Don’t know = 8888 (SKIP TO QUESTION 113) d. Refuse to respond = 9999 (SKIP TO QUESTION 113) 275 112. Who invited you to be involved? Munayitanidwa ndi ndani? (Multiple responses possible; select all that apply) a. Headteacher b. Teacher c. PTA Member d. School Committee Member e. Letter from school f. Neighbor g. Friend h. Relative i. [LEARNER’s NAME] j. The EGRA Project k. The TIANA Project l. The Literacy Boost Project m. The ASPIRE Project n. Other, please list____________________________________________________ o. Don’t know or don’t remember 8888 p. Refuse to respond = 9999 113. Are you (and/or any member of the household) involved in the school in any way? Kodi inu kapena wina aliyense wa m’banja lanu amakhudzidwa ndi za kusukulu ya [LEARNER’S NAME’S] munjira ina iliyonse? a. No = 0 (SKIP TO END) b. Yes = 1 c. Don’t know = 8888 (SKIP TO END) d. Refuse to respond = 9999 (SKIP TO END) 114. When did you (and/or any member of your family) first become involved? Ndi liti limene inu kapena wina wapakhomo pano pamene anakhuzidwa koyamba ndi zochitikachitika za pasukulu ya [LEARNER’S NAME’S] koyamba a. This year = 1 b. Last year = 2 c. Two years ago = 3 d. Three years ago = 4 e. More than three years ago = 5 f. Don’t know = 8888 g. Refuse to respond = 9999 • 276 115. How are you (and/or someone in your household) involved? Kodi inuyo kapena wina aliyense wapabanja panu amakhuzidwa bwanji ndi za sukulu kapena za mkalasi ya [LEARNER’S NAME] (Multiple responses possible; select all that apply) a. Help in [LEARNER’S NAME’s] classroom b. In PTA c. On School Committee d. In a group helping to increase support for reading e. Host after-school book club f. Donate books, magazines g. Donate food for school meals h. Encourage families to send their girls to school or to let them stay in school i. Encourage families to send their disabled child(ren) to school or to let them stay in school j. Provide financial support to families who can’t afford to children to school k. Provide (buy and/or make) learning materials for use in the classroom l. Helped to construct, maintain and/or refurbish a building (e.g., classroom, teacher housing, latrine) m. Help in school garden n. Other, please specify_________________________________________________ o. Don’t know = 8888 p. Refuse to respond = 9999 • 116. Approximately how much time do you spend each month on these activities? Pafupifupi mumagwiritsa ntchito nthawi yochuluka bwanji pamwezi pa zochitikachitikazi? a. No time = 0 b. About 1 hr = 1 c. About 2 hrs = 2 d. About 3 hrs = 3 e. About 4 hrs = 4 f. Five or more hours = 5 g. It varies from one month to the next = 6 h. Less than monthly = 7 i. Don’t know or don’t remember = 8888 j. Refuse to respond = 9999 277 117. What motivated you to get involved in [LEARNER’s NAME’s] school or classroom? Chinakupangitsani ndi chiyani kuti muzitenga nawo mbali pazochitikachitika pa sukulu kapena mkalasi ya [LEARNER’S NAME’S]? (Select all that apply; multiple responses possible) a. I was asked to b. I thought I could help improve some things at the school c. I think the emphasis on reading will help (LEARNER’S NAME) be more successful in school and in life/work d. The school/project said it is important that all parents and community members get involved in the school e. The school/project said it is important that all parents and community members to get involved in supporting reading f. Other, please specify_________________________________________________ g. Don’t know h. Refuse to respond Time Interview Ended:_______________________________________________________ Thank you for your participation! You have been very helpful! Zikomo chifukwa chakutenga nawo mbali mukafukufukuyu!