Republic of Zambia Ministry of Education _________________________________________________ Learning at Taonga Market at Grade 3: An Evaluation of Interactive Radio Instruction In IRI centres and IRI Community Schools in 2006 __________________________________________________________ CONTACT INFORMATION This evaluation was conducted by the Department of Open and Distance Learning of the Ministry of Education with technical assistance and financial support from QUESTT, a USAID-funded project. Department of Open and Distance Education Ministry of Education P.O. Box 50093 Lusaka Zambia Office phone: +260-1-254 330 QUESTT (Quality Education Services Through Technology) Project Education Development Centre 5 Kasisi Close Private Bag 542x, RW Lusaka, Zambia Office phone: +260-1-257 520 Grade 3 Evaluation Report, 2006 i TABLE OF CONTENTS EXECUTIVE SUMMARY............................................................................................................III 1.0 INTRODUCTION ..............................................................................................................1 1.1 BACKGROUND .............................................................................................................................................. 1 1.2 PURPOSE OF THE EVALUATION ................................................................................................................ 2 2.0 EVALUATION METHODOLOGY .................................................................................. 3 2.1 SAMPLING DESIGN ...................................................................................................................................... 3 2.2 ASSESSMENT OF LEARNING ACHIEVEMENT .......................................................................................... 4 2.2.1 Test Planning and Development ........................................................................................................... 4 2.2.2 Piloting and Reviewing the Tests........................................................................................................... 4 2.2.3 Live Test Administration ........................................................................................................................ 5 3.0 FINDINGS AND DISCUSSION ....................................................................................... 6 3.1 DEMAND FOR IRI......................................................................................................................................... 6 3.2 PROFILE OF LEARNERS IN IRI SCHOOLS.................................................................................................. 7 3.2.1 Sex and age of learners............................................................................................................................ 7 3.3 ATTENDANCE ............................................................................................................................................... 9 3.4 PERFORMANCE IN MATHS, ENGLISH, SCIENCE AND SOCIAL STUDIES........................................... 11 3.4.1 Overall and subtest performance......................................................................................................... 11 3.4.2 Mean scores by sex ........................................................................................................................... 12 3.4.3 Mean scores by age ........................................................................................................................... 12 3.4.4 Percent mean scores by school type and orphan status ............................................................. 13 3.5 OTHER CHARACTERISTICS AFFECTING PERFORMANCE AT GRADE 3.............................................. 13 4.0 DISCUSSION AND RECOMMENDATIONS ................................................................16 4.1 SUMMARY..................................................................................................................................................... 16 4.1.1 Demand for IRI...................................................................................................................................... 16 4.1.2 Characteristics of the IRI Population ................................................................................................. 16 4.1.3 Attendance of IRI Learners.................................................................................................................. 17 4.1.4 Achievement ........................................................................................................................................... 17 4.2 RECOMMENDATIONS................................................................................................................................. 18 REFERENCES ..............................................................................................................................20 APPENDIX A: GRADE 3 TEST....................................................................................................21 APPENDIX B: T-TEST TABLES .................................................................................................40 Grade 3 Evaluation Report, 2006 i LIST OF ACRONYMS DEBS .............................................................................District Education Board Secretary DODE........................................................ Directorate of Open and Distance Education DRCC........................................................................District Resource Centre Coordinator EBS .................................................................................Educational Broadcasting Services ECZ ..................................................................................Examinations Council of Zambia EDC....................................................................................Education Development Centre DODE................................................................................Education Development Centre GRZ..................................................................... Government of the Republic of Zambia IRI ............................................................................................Interactive Radio Instruction IRLC................................................................................Interactive Radio Learning Centre LTM............................................................................................ Learning at Taonga Market MOE.....................................................................................................Ministry of Education OVC................................................................................ Orphans and Vulnerable Children PDP.................................................................. Programme Development and Production POC................................................................................. Provincial Outreach Coordinator QAA...........................................................................Quality Assurance and Accreditation QUESTT..............................................Quality Education Services Through Technology SEO-ODL.................................Senior Education Officer-Open and Distance Learning TED ...................................................................................Teacher Education Department ZIP................................................................................................. Zonal In-service Provider Grade 3 Evaluation Report, 2006 ii EXECUTIVE SUMMARY Since the inception of IRI in 2000, the effectiveness of IRI has been evaluated by investigating whether or not children listen to Learning at Taonga Market programs have mastered the intended learning goals, and/or perform as well as children at the same grade level in GRZ schools. There has been four evaluations at Grade 1, (2000 and 2001, 2003, 2005). The first three evaluations sought to investigate whether there was demand for IRI and regular attendance at the IRI learning centers, and how much learning was actually taking place. In addition to the end-of-year evaluation of children in IRI centers, the 2005 evaluation documented performance of children in GRZ schools which were piloting IRI, compared to GRZ schools which were not using IRI. For the first time in 2006, QUESTT conducted evaluations at Grades 2 and 3. Grade 3 data in this evaluation report were collected from in October/November in five of nine provinces. 47 schools were selected out of 625 IRI schools in the province, with 10 non-IRI schools being sampled as a control group. The total sample size was 742 of 10 111 learners, 6.2 percent of the population. Findings respond to evaluation questions on demand for IRI as manifested in the enrollment and daily attendance, learner and mentor characteristics, and student mastery of English Language, Mathematics, Science and Social Studies using a curriculum-based achievement tests. With all 72 districts in Zambia using it, IRI is in high demand in Zambia. Programmes are been used by an increasing number of learners in IRI centres, community schools, and recently in GRZ schools. The increase in demand for IRI is established first from the pattern of establishing IRI centres (from 893 in 2005 to 1022 in 2006), the number of students enrolled in IRI centers and community who participate in IRI (from 56 233 in 2005 to approximately 81 324 in 2006), and in that IRI programmes had been developed and were being broadcast for all primary grades. The demand for establishing IRI persists, even when the programs can be accessed through GRZ schools. Enrolment at the lower primary grades is about equal for boys and girls. However, the 2006 enrolments indicate that girls drop out of school at a higher rate than boys at the upper grades (44.2 percent of girls compared to 55.8 percent of girls in Grade 7). Other characteristics of IRI learners are a high proportion of above-age learners, which reflects the difficulty that children have experienced in gaining access to education; hence the IRI program does provide a second chance for such learners. The proportion of Grade 3 learners that were older than 9 years in 2006 is 76.0 percent, compared to X percent in GRZ schools. The evaluation also highlights the fact that at 35.0 percent, the proportion of orphans is much higher for IRI learners compared to 20.0 percent GRZ schools. The proportion of orphans in IRI community schools and GRZ schools was much higher in some provinces. Attendance and actual participation in IRI lessons depends partly on whether centres and schools receive daily broadcasts. Mentors reported that they had good radio reception was acceptable in most cases (more than 90 percent in IRI centres and community schools. It was also reported that radios were in good working condition throughout the school year. Data showed that 80.0 percent of the learners attended at least 60 percent of the lessons (90 or more lesson out of 150 lessons). Orphans attended less frequently than non-orphaned children. Overall learners in IRI schools performed better than learners in non-IRI control schools (48.9 percent compared to 42.9 percent). IRI learners performed better than the control group in Maths and English and Social Studies; performed at par in Science; all learners performed poorly Grade 3 Evaluation Report, 2006 iii Grade 3 Evaluation Report, 2006 iv in Math and English (Table 14). The Western Province registered the highest overall performance (69.9 percent). In Central province control groups performed significantly better than IRI learners in all subtests (Table15). Boys performed better than girls in Math, English and Social Studies, while boys and girls performed at par in Science. Disparities were wider in maths, both for IRI and non-IRI control schools (Table 16). The differences were significant only in maths. Older learners performed better in both IRI and non-IRI school. (Table 17). Performance was better for IRI learners, both orphans and non-orphans, with children who have the mother as the one living parent performing best, followed by double orphans. Other factors that affected performance had to do with teacher characteristics. Learners who had a teacher with 3 or more years of teaching had an overall test score of 46.7 percent; while those with teachers with less teaching experience got 39.6 percent. For new teachers of 1 or 2 years teaching experience, learners performed better in IRI centers, while for those with 3 or more years of teacher experience, they performed better in non-IRI centers. This indicates that IRI is a good solution, but not a substitute for formal teacher training qualification. The results of the evaluation indicate that mentors and communities remain motivated and committed to IRI activities that take place in the learning centres. This is due in part, to a successful outreach strategy which includes, among others, providing education and assistance to mentors who experience debilitating health problems. While many suggestions for improvement were made in the report, recommendations for discussion and follow-up action are as follows: 1. Work aggressively with other organisations to mitigate of attrition among girls at the middle basic level. QUESTT should pay special attention to gender dynamics at the middle school level, first by accessing the body of knowledge that already exists, and supporting already existing programmes for retaining girls in schools. 2. Monitor the impact grants programmes in IRI centres and community schools. Income generating activities would provide IRI centres and community schools with the income needed to compensate teachers adequately so that they remain at the school and teach classes regularly. The income should also be used to buy school supplies, such as chalk, pencils and exercise books. 3. Underage children should be assessed before being promoted to the next grade level. QUESTT should work with MOE to develop policy guidelines for assessed progression for under-aged learners. Younger children would benefit more from repeating a grade proceeding on to the next grade. 4. QUESTT should use the information on attendance and performance of orphans to strengthen programming for the OVC programs. In particular, emphasis should be put on training in gender roles in the care of orphans. Information should also be shared with partner organisations with OVC programs. 5. Teachers should NOT teach more than two classes. While it is sometimes difficult for IRI centres and Community Schools to recruit enough teachers to meet this guideline, administrators and school committees should be aware that having a teacher lead more than two classes will be detrimental to their children’s performance. 1.0 INTRODUCTION 1.1 Background Learning at Taonga Market (LTM) is a series of educational radio programs that deliver the basic education curriculum, using the Interactive Radio Instruction (IRI) approach. LTM lessons are written and recorded by Educational Broadcasting Services (EBS), under the auspices of the USAID-funded Quality Education Services Through Technology (QUESTT) Project. Each lesson consists of a 30-minute broadcast, along with activities that the class completes before and after the broadcast. The activities for each lesson and the program are described in a mentor’s guide. The programme follows the national curriculum and the MOE’s calendar of three terms. There are 150 lessons at each grade level, plus five teacher training broadcasts at the beginning of each term. QUESTT is managed by the Education Development Center (EDC). In 2006 Learning at Taonga Market was broadcast to Grades 1 through 6. Grade 7 programs will be offered for the first time in 2007. Table 1 shows the number of learners using IRI in 2006. Table 1: Number of IRI learners in 2006, by province and gender Province Female Male Total Central 6662 6620 13,282 Copperbelt 1765 1869 3,634 Eastern 8589 8388 16,977 Luapula 1362 1492 2,854 Lusaka 7199 7056 14,255 North-Western 4737 3480 7,062 Northern 3582 5172 9,909 Southern 4458 4452 8,910 Western 2096 2310 4,406 Total 40,485 40,839 81,289 LTM is designed to give learners in community schools and IRI centres the opportunity to complete seven years of education through radio-based learning. It is also used in GRZ schools as a supplementary learning resource. At the end of the primary cycle, LTM learners in IRI centers may register for the Primary School Leaving Certificate Examination (PSLE) in community schools. Learners who pass the PSLE have a choice to attend upper basic grades in the government schools, or enrol in the DODE Alternative Upper Basic Education Programme at distance learning centres. As shown in Table 2, the majority of learners using IRI are currently enrolled in IRI centres, since this was the original target for the programme. Table 2: Number of learners in IRI Centers and Community Schools, 2006 Type of School Number of Learners IRI Centres 38,258 Community Schools 43,066 Total 81.324 Several evaluations of IRI based-learning have been conducted, the latest being Grade 1 in 2005. The major findings of the 2005 evaluation were that: x Demand in 2005 was greater than in the past with a rise in enrolment at Grades 1 to 5 from 38,513 in 2004 to 56,233 in 2005. x The programme meets the needs of girls, as the overall enrolment ratio was 50.5 percent girls and 49.5 percent boys. Grade 3 Evaluation Report, 2006 1 x Grade 1 learners made satisfactory gains in Mathematics and English language. x The majority of Grade 1 learners could not read at the expected level. in a Zambian Language. In response to the last finding, Zambian language readers are being distributed to community schools. Also, IRI methodologies have been incorporated into the pre-service teacher education curriculum. An evaluation of LTM at Grade 1 was also performed in GRZ schools during 2005. LTM is being used in government schools to supplement other methodologies, such as NBTL, SITE, ROC and MARK. In 2005, LTM was piloted at Grade 1 in 36 government schools. Learners in the 36 pilot schools and 14 control schools were tested at the beginning and the end of the year. The pilot results showed: x Learners using LTM had greater gains in English and Mathematics x Girls and boys using LTM had equal learning gains x Enrolment increased in LTM classes x LTM motivated learners and promoted good time and class management Given the effectiveness of LTM and the need to improve access to quality education, the Ministry of Education approved of the roll-out of LTM to community and GRZ schools throughout Zambia for January 2007. With the roll-out of IRI to GRZ schools, the number of GRZ learners using IRI promises to grow dramatically. The current evaluation is the first evaluation of LTM at Grade 3. It asks the same questions as earlier evaluations; whether there is demand for LTM, who the learners are, whether they attend radio lessons and whether they are learning. A sample of learners was tested at the end of the year in community schools and IRI centres. The performance of learners who were using IRI was compared with the performance of a control group drawn from non-IRI community schools. IRI Teachers were interviewed to learn about factors that impact the effectiveness of LTM at Grade 3. This report describes demand for IRI, the overall profile of learners, the results of the testing and the teachers interviews. 1.2 Purpose of the Evaluation The overall goal of this evaluation is to document the effectiveness of IRI at Grade 3 in IRI centres and community schools with a view to make adjustments to the programme and/or its implementation. The evaluation questions are as follows: 1. What is the level of demand for IRI in Zambia? 2. What are the characteristics of the children who participate in IRI? 3. How frequently do learners attend daily broadcasts? 4. Are learners achieving English Language, and Mathematics, Science and Social Studies as expected at Grade 3 level? The evaluation strategy stipulated the use of population parameters to describe the profile of learners in the centers, in particular their sex, age, who their guardians are, and whether they have any living parents (orphan status), and sample statistics for the achievement scores. Data on enrollment, background characteristics of the learners, and attendance was collected at using sample data. Grade 3 Evaluation Report, 2006 2 2.0 EVALUATION METHODOLOGY The design of the evaluation was guided by a MOE steering committee for the evaluation of IRI, composed of members of the Examinations Council of Zambia (ECZ), the Directorate of Open and Distance Education (DODE), Teacher Education and Specialized Services (TESS), Curriculum and Standards and the QUESTT Project. The Steering Committee produced an evaluation plan that described the purpose of the evaluation, and information that should be collected, along with procedures for collecting data, analyzing data and producing the report. The Steering Committee also reviewed and revised the sampling design, created a schedule of evaluation activities and allocated resources to ensure that activities would be completed as scheduled. This section describes the sampling design and the procedures used to develop assessment instruments. 2.1 Sampling Design A posttest only model was used in this evaluation. IRI community schools and IRI centres and non-IRI control schools were sampled. To select the schools, a three-level sampling procedure was used. First, five of nine provinces were selected, two urban provinces (Lusaka and Copperbelt), and two rural provinces (Eastern and Western) and one province along the line of rail (Southern province). Table 3 below indicates IRI schools and control non-IRI Schools in the Grade 3 sample. Out of 599 IRI community schools that use IRI in the five provinces, 47 schools were sampled. An additional 10 control schools, which do not use IRI were included in the sample. The number of IRI schools in each province determined proportionally, with efforts being made to test at IRI schools that were part of the sample in the pervious year. Table 3: Total number of schools in the sample, by province and school type Province Total IRI Schools Sampled IRI Schools Control Schools Central 162 10 4 Copperbelt 58 6 2 Eastern 155 12 4 Lusaka 140 14 0 Western 84 5 0 Total 599 47 10 At the third stage of sampling, IRI learners were sampled proportionally from each province using a simple random sampling of 6.2 percent of the total IRI learners (10 111) in Grade 3 registered in 2006. For each class tested, an equal number of boys and girls were selected, even though the actual number that was tested is 51.4 percent boys to 48.5 percent girls. Table 4 below shows the number of learners assessed, by province and sex. Table 4: Total number of learners in the sample, by province and sex Province Sex of learner Male Female Total Central 107 102 209 Copperbelt 48 39 87 Eastern 95 105 200 Lusaka 109 92 201 Western 23 22 45 Total 382 360 742 Grade 3 Evaluation Report, 2006 3 Test administrators were instructed to administer the test at IRI schools that had good radio reception throughout the year and to select learners who had high or medium attendance. A total of 150 radio lessons are broadcast during the year, hence attendance was rated high if, of a possible 150, learners attended 120 or more radio lessons, medium if learners attended between 90 to 119 lessons, or low for less than 90 lessons. 2.2 Assessment of Learning Achievement The Steering Committee was supported by a Technical Committee, which included an evaluation specialist from the University of Botswana and members of ECZ, DODE, Curriculum and Standards, and the QUESTT Project. The function of the Technical Committee was to produce the Grade 3 assessments and revise them with the assistance of pilot test data. This section describes the procedures used to develop the assessments. 2.2.1 Test Planning and Development Test planning took place at a Test Development Workshop in October 2005. The Technical Committee reviewed the Zambian lower primary curriculum and determined the content for Grade 3 English Language, and Mathematics, Science and Social Studies. A distinction was made between terminal objectives that should have been mastered by the Grade 3 level and developmental objectives that would be mastered at later grades. Once terminal objectives had been selected for assessment, a test plan was developed with tasks designed to assess each objective. The Committee then divided into small groups to write items for each section of the assessment. These items were compiled and then reviewed at a one-day workshop. 2.2.2 Piloting and Reviewing the Tests The general purpose of piloting and reviewing the tests was to determine if the items measured the intended skills and if they were at the expected levels of difficulty. More specifically, the objectives of the pilot were to ensure that: 1. The tests could be administered to the desired number of pupils in a day 2. The test items yielded the intended information 3. The test items were at the right level of difficulty 4. The items discriminated well among high, medium and low level learners Draft versions of the Grade 3 tests were piloted in February 2006 among pupils during the first few weeks of Grade 4, a few weeks after completing the Grade 3 curriculum. Twelve people formed two teams of test administrators. The test administrators came from DODE, CDC, QUESTT and UNZA. Data from the pilot testing was compiled in SPSS and used to calculate two statistics: percentage correct scores and correlation coefficients. The percentage correct was calculated by dividing the total points that learners earned by the total possible points. An item with 90 percent correct would be a very easy item because the learners would have earned 90 percent of the total possible points for that item. On the other hand, an item with 10 percent correct would be very difficult because learners would have earned only 10 percent of the possible points for that item. The percentage correct was used to determine whether an item was as easy or as difficult as expected. Members of the Technical Committee rated each test item as Easy, Medium or Hard in terms of the objective that was being tested. Items were examined carefully for revision when administrators encountered problems administering the item during the pilot or when an item was supposed to test a relatively easy or hard objective but the percentage correct did not correspond with its rating. Grade 3 Evaluation Report, 2006 4 The correlation coefficients for each item were determined by correlating learners’ scores for individual items with their scores on the entire test section. This correlation helped committee members see how well the learners’ scores on an individual item corresponded with their scores on that section of the test. When learners who performed well on an item also performed well on the section and when those who performed poorly on the item also performed poorly on the section, then the item had a high correlation. Items with a correlation below 0.40 were looked at carefully to see if they were measuring what they should be measuring. Items were revised when getting the correct answer correct depended on skills other than those being tested. The pilot test succeeded in achieving each of the four objectives. 1. It was determined that the Grade 3 test could be administered to group of learners 2. Pilot data and experiences from the administration were used to revise the items so that they would yield the intended information. 3. Items that were too easy or difficult were revised or removed from the test. 4. Each section of the test was determined to have a good balance of easy, medium and difficult items. Once the Technical Committee had revised the Grade 3 tests in conjunction with the pilot data, the tests were reproduced for administration. The final version of the Grade 3 assessment is presented in Appendix A. 2.2.3 Live Test Administration Test administrators included POCs, DRCCs and Senior Education Officers from DODE, Standards and TESS as well as a lecturer from the University of Zambia. Test administrators reviewed guidelines for sampling learners, administering the tests and interviewing teachers. A training workshop for test administrators from each of the five provinces was held. Administrators went through the actual tests, and the testing guidelines. They were also given time to practise administering the test to each other. Live testing was conducted in October and November, 2006. Five teams of four members each were involved in the testing at IRI Centres and Community Schools. Each team had a team leader who was in charge of ensuring quality control, compiling results and submitting reports. Reports contained information about where testing was done and any problems with test administration. Each team submitted all test results and teacher questionnaires. Test administration proceeded smoothly with no problems that would invalidate the results of any centre. The main problem that test administrators encountered was the inability to test at certain centres because the schools had closed for the end of the year. Team leaders also provided lists of the strengths and weaknesses of each centre. Grade 3 Evaluation Report, 2006 5 3.0 FINDINGS AND DISCUSSION 3.1 Demand for IRI Over the past seven years LTM has enabled many out-of-school children access to basic education in all the nine provinces of Zambia. From an initial 21 IRI centres with 1,254 Grade 1 learners in 2000, in 2006 IRI programs were being broadcast in 1022 IRI centers and community schools, to 81 324 learners enrolled in Grades 1 to 6. The demand for IRI can be traced first from the pattern of establishing IRI centres as shown in Table 5. Table 5: Number of IRI Centres and Community School, and total enrollment Provinces Total IRI centres 2004 Closed IRI centres 2004/05 New IRI centres 2005 New IRI centres 2006 Centres submitted data 2006 Number of learners 2006 Central 135 18 23 49 162 13 282 Copperbelt 85 4 41 11 58 3 634 Eastern 94 8 97 74 197 16 977 Luapula 71 9 14 6 55 2 854 Lusaka 86 7 90 40 140 14 225 Northern 77 4 44 26 112 9 909 North Western 27 1 7 36 82 7 062 Southern 51 4 54 14 132 8 890 Western 44 2 14 24 84 4 406 Others 35 Total 670 57 384 280 1022 81 324 In addition to the number of new centers being opened in each year, the demand for IRI has been manifested in the number of community schools adopting the use of IRI as a teaching method in their school, and the number of centers that have been able to stay open over the years. For instance, 280 more schools started using IRI in all provinces compared to 384 new schools that adopted IRI in 2005. Eastern, Central and Lusaka province had higher number of new schools that used IRI 2006. The demand for IRI is high as shown with learners enrollment figures in Table 6. Table 6: Enrolment of IRI learners from 2000 to 2006, by sex Learners 2000 Grade 1 2001 Grade 1 and 2 2002 Grade 1 to 4 2003 Grade 1 to 5 2004 Grade 1 to 5 2005 Grade 1 to 5 2006 Grade 1 to 6 Male - 3 994 7 104 11 561 19 412 27 819 40 860 Females - 3 788 6 989 11 202 19 101 28 414 40 464 Total 1,254 7 782 14 093 22 763 38 513 56 233 81 324 Percent growth - 521 81 62 69 46 45 In any given year, the majority of learners that enroll in schools are Grade 1 learners, indicating that for many communities, QUESTT responds to the need for access and providing children with the opportunity to learn. Progression from one year to the next also shows that there is demand for IRI beyond Grade 1. For example, Table 7 tracks Grade 3 learners in 2003 and in the subsequent years - the size of Grade 3 learners has grown from 7 176 in 2003, to 12 032 in 2006. Grade 3 Evaluation Report, 2006 6 Table 7: Grade 3 learners in 2003 – 2006, by sex Item 2000 2001 2002 2003 2004 2005 2006 Male - - - 3 707 4 808 4 991 5 894 Female - - - 3 469 4 391 5 118 6 138 Total enrolment - - - 7 176 9 199 10 109 12 032 Growth - - - - 2 023 910 1 923 Percentage growth Pilot Baseline - - 28 10 18 Percent growth of the Grade 3 cohorts shows growth rates of 28 percent in 2004, 10 percent in 2005, and 18 percent in 2006. This means either that the Grade 1 cohort has been larger in progressive years or that IRI is picking centers and community schools are picking up additional learners from transfers, or both. 3.2 Profile of learners in IRI schools 3.2.1 Sex and age of learners Girls are systematically excluded from learning opportunities in most developing countries, hence the need to disaggregate enrolment by gender in order to determine whether or not there is equal participation of boys and girls. The table shows that 49.8 percent of the learners in IRI centers and community schools were girls, while 50.2 percent were boys in 2006. Table 8: Number of learners in IRLCs and Community Schools, by grade level Grade Male percent Female percent Total Grade 1 17440 49.0 18169 51.0 35609 Grade 2 9107 50.1 9076 49.9 18183 Grade 3 6138 51.0 5894 49.0 12032 Grade 4 3,996 51.5 3,763 48.5 7759 Grade 5 2,554 52.8 2,280 47.2 4834 Grade 6 1,604 55.8 1,268 44.2 2872 Total 40839 50.2 40450 49.8 81289 In contrast to the formal school system where dropout rates for girls tend to be higher than those for boys, IRI has traditionally had a good track record of providing equal access and participation opportunities. During the first few years of IRI when LTM was at the lower grades, the overall enrolment ratio between boys and girls was within one percent point, an indication then that IRI provided equal opportunity and access to education to girls and boys. These data indicate that the pattern persists only up to about Grade 4. For Grades 5 and 6, the proportions of girls to boys has widened considerably, with far fewer girls participating than boys at Grade 6. While the pattern of participation is generally similar in GRZ schools, the participation of girls in IRI centres and community schools is 2 percent points lower at Grade 6. Given that there is a higher proportion of orphans and vulnerable children in IRI centres, it is possible that as they grow older, girls take on more roles of caring and/or providing for their siblings and foster families, thus denying them a chance to continue attending school. More investigation into this systematic attrition of girls should be made through the QUESTT OVC program and other girls empowerment programs run by QUESTT partners. IRI learners are also typically older children who either missed an opportunity to attend school, or dropped out. The average age for the learners in the sample was 10 years 3 months for learners in IRI centres and community schools, (compared to 9 years 3 months for learners in Grade 3 Evaluation Report, 2006 7 GRZ schools). Table 9 presents age by grade level in the IRI population. While it varies by grade level, it shows that only 23.8 percent of all learners are at the recommended age for the grade, while 60.1 percent of the learners were above the age recommended for their grade level. The proportion of learners that were younger was 13.3 percent. This pattern persists through all the grade levels. Table 9: 2006 enrolments, by age recommended for each grade Below recommended age At recommended age Above recommended age Grade Level and age Total N percent N percent N percent G1 (7 yrs) 35 633 6770 19.0 11759 33.0 16391 46.0 G2 (8 yrs) 18 184 2000 11.0 4000 22.0 11638 64.0 G3 (9 yrs) 12 036 843 7.0 1685 14.0 9147 76.0 G4 (10 yrs) 7 760 621 8.0 1086 14.0 5665 73.0 G5 (10 yrs) 4 834 387 8.0 532 11.0 3771 78.0 G6 (11 yrs) 2 870 201 7.0 287 10.0 2239 78.0 Total 81 317 10822 13.3 19349 23.8 48851 60.1 (From DODE and QUESTT, 2005:32) The high proportion of above-age learners reflects the difficulty that children have experienced in gaining access to education; hence the IRI program provides a second chance for such learners. Being a system that makes special considerations for disadvantaged learners, some of whom are responsible for the upkeep of their younger siblings, Learning at Taonga Market has allowed a higher degree of flexibility to learners. Rather than miss the opportunity to attend, learners are generally allowed to bring their younger brothers and sister along to the IRI centres. It could also be that parents are using IRI centres to fill a need for early childhood care facilities (crèches, pre-school or pre-primary). While having under-age learners cannot be discouraged without taking away the flexibility for the learners who bring their younger siblings along, the fact that the content of IRI programmes is not well suited for their level was mentioned in previous evaluations. In order to derive any benefit from the IRI programme, under-age learners do require specialized teaching approaches that are not part of the preparation and training of mentors, hence the IRI system cannot afford to be burdened with younger learners. As will be reported in the subsequent sections, the performance of under-age learners tends to be lower than that of age-appropriate and older learners. Hence, overloading the system with under-age learners might temper with the effectiveness of IRI as a learning tool. It would be desirable at this stage in IRI programming for QUESTT to experiment with pre￾primary programs while EBS continues with revisions that would align Learning at Taonga Market with the new curriculum. These could be useful in at least three contexts. First, as the opportunity presents itself, community schools (and GRZ schools) will be able to use the programs for the pre-primary grade. Second, some kind of standardization would be achieved when private providers are exposed to the same materials. Third, parents who work from home will be able to listen in with their younger children, a highly desirable practice in terms of preparing children for school and cultivating a culture of being part of children’s learning activities. 3.2.2 Orphan Status Since 2003 the proportion of orphans in IRI schools has been higher when compared to orphans in GRZ basic schools. Overall, 33.7 percent of the IRI learners were orphans in 2006, compared Grade 3 Evaluation Report, 2006 8 to 20 percent in GRZ basic schools as indicated in the Table 10 below. In 2006, 23 percent of the IRI learners were single orphans, while 10 percent were double orphans. Table 10: Percentage of orphans in IRI centres, by year Overall IRI 2004 IRI 2005 IRI 2006 Single orphans 8 605 12 636 18 993 Double orphans 4 583 6 252 8 488 Total orphans 13 188 18 888 27 481 Total enrolled 38 513 56 233 81 324 Percent orphans (IRI) 34.2 33.5 33.8 Percent orphans (GRZ) 20 20 20 Twenty-three percent (23) of the IRI learners were single orphans, while 12 percent were double orphans. Differences between provinces in the number of orphans varied between 29 percent in Eastern and Southern Provinces to as high as 53 per cent in Western Province. However, with the exception of Eastern, Northern, and Western provinces, the trend in 2006 was that of decrease in proportion of learners who are orphans as indicated in Table 11. Table 11: Percentage of orphans in IRI centres and GRZ schools, by province and year Provinces IRI Centres GRZ Schools 2005 2006 Change 2005 2006 Change Central 36 34 -2 20 20 0 Copperbelt 45 34 -11 20 18 -2 Eastern 24 29 5 20 20 0 Luapula 47 40 -7 19 19 0 Lusaka 46 44 -2 22 22 0 Northern 24 26 2 16 17 1 North Western 46 40 -6 17 18 1 Southern 30 29 -1 20 21 1 Western 44 53 9 23 24 1 All Provinces 36 35 -1 20 20 0 As expected, the proportion of orphans is higher in IRI centres where there are more vulnerable children than in GRZ schools. A comparison between provinces shows more interesting trends. First, the proportion of orphans in 6 of 9 provinces decreased, with the Copperbelt registering the highest decrease at 11 percent. Second, Western and Eastern provinces registered an increase of orphans in the IRI centre learners. At 53 percent, the highest proportion of orphans was found in Western province, with Lusaka being second highest at 44 percent. With the high prevalence rate of HIV/AIDS in Zambia it is assumed that many of the orphans lost their parents to HIV/AIDS and related illnesses. A response to this problem has been the introduction of the OVC component that addresses issues relating to HIV/AIDS, orphanhood and other issues for equally vulnerable children. 3.3 Attendance For the purposes of this evaluation, high and medium attendance levels are considered acceptable, while low attendance is considered to be unacceptable. The next table shows the Grade 3 Evaluation Report, 2006 9 number of learners who had either high or medium attendance—that is, attendance of 60 percent or more as shown in Table 12. 16.2 percent of the learners had no attendance data. Of those remaining the results indicate that 58.3 percent of the learners had high attendance and 25.7 percent had medium attendance, while 16.0 percent had low attendance. Table 12: Percent attendance for Grade 3 learners, by province Province Attendance Rating (percent) High (120 and above) Medium (90 to 119 ) Low (90 or less) Total Central 112 (58.0) 51 (26.4) 30 (15.5) 193 Copperbelt 18 (56.3) 11 (34.4) 3 (09.4) 32 Eastern 87 (54.0) 60 (37.3) 14 (08.7) 161 Lusaka 110 (57.3) 37 (19.3) 45 (23.4) 192 Western 29 (65.9) 5 (11.4) 10 (22.7) 44 All provinces 356 (58.3) 164 (25.7) 102 (16.0) 622 Test administrators were instructed to select learners with low attendance only where there were not enough learners with medium or high attendance. The fact that learners with low attendance were tested indicates that there were not enough learners with high or medium attendance, which means that learners in Lusaka and Western provinces had poorest attendance. It should be noted, also, that at 44.0 percent and 53.0 percent respectively, Lusaka and Western were the two provinces higher proportion of orphans. Further analysis indicates that orphans attended IRI lessons less frequently than children who are not orphans and shown in Table 13. Table 13: Percent attendance for Grade 3 learners, by orphan status Orphan status of learner Attendance Frequency Percent Valid Percent High or Medium 290 76.1 83.6 Both parents alive Low 57 15.0 16.4 High or Medium 29 64.4 82.9 Only father alive Low 6 13.3 17.1 High or Medium 140 82.4 90.9 Only mother alive Low 14 8.2 9.1 High or Medium 60 65.9 70.6 No parents alive Low 25 27.5 29.4 Having one or both parents alive translated to having children attend school more. 70.6 percent of learners who have no living parents fell in the high/medium category attendance, compared to 83.6 percent who have both parents. A more interesting result was that of children who had a mother as their only parent they registered a 90.9 percent high/medium category attendance. This is a significant outcome in terms of the gender aspects of guiding younger children. It suggests that children are more likely to attend school if they have one parent, the mother, than if they have both parents. It may also suggest that orphaned children do not receive the necessary guidance and support when it comes to attending school, either from caregivers or because they are heads of households. . Grade 3 Evaluation Report, 2006 10 3.4 Performance in Maths, English, Science and Social Studies The section looks at Grade 3 learners’ performance in the areas of Maths, English, Science and Social Studies. The section will discuss the results of all learners tested in IRI centres and IRI Community schools, as well as in non-IRI control schools. The other part of the discussion will describe the overall performance of learners by type of school and location. The description of the results will also look at the performance of the learners in relation to the learning environment. 3.4.1 Overall and subtest performance Grade 3 assessment consisted of four subtests: Mathematics was worth 27 points, English 35, Science 27 and Social Studies 20 points. Table 14 presents the mean scores for the overall test and for the subtests for all learners—those using IRI as well as those in control schools. Overall, learners in IRI schools performed better than learners in non-IRI control schools (48.9 percent compared to 42.9 percent). Learners in IRI schools performed better than learners in control schools in Mathematics, English, and Social studies. In Science., the performance of learners IRI and non-IRI control was at par (a mean of 65.6 percent for each group). The differences between the means were significant at a = .05 (see Appendix B for t-tests). Table 14: Mean Scores for all learners by school type and subtest School Type Maths 27 points English 35 points Science 27points S.Studies 20 points Overall 109 points IRI learners Mean 10.5 11.5 17.7 13.6 53,3 Percent 38.9 32.9 65.6 68.0 48.9 Control Mean 7.6 8.9 17.7 12.5 46.8 learners Percent 28.1 25.4 65.6 62.5 42.9 While IRI learners posted better performance in Math and English, it is a source of disappointment that they showed mastery of less than 40 percent of what they should know at the end of the grade. Of the learners tests in each province, the Western province did not have a control group. Table 15 shows that overall, IRI learners performed better than the control learners in Copperbelt, Eastern and Lusaka provinces. In Central province control groups performed significantly better than IRI learners in all subtests. Table 15: Percent mean scores for all learners, by province, school type, and subtest Province School Type Math English Science Social Studies Overall Test Central IRI learners 33.3 28.9 62.2 74.5 46.5 Control learners 37.8 35.7 70.4 80.0 52.8 Copperbelt IRI learners 33.3 27.7 66.3 54.0 43.5 Control learners 17.8 8.3 78.5* 36.5 33.2 Eastern IRI learners 39.6 26.0 57.8 74.5 46.1 Control learners 27.8 16.0 53.3 67.0 37.5 Lusaka IRI learners 36.7 40.3 70.0 55.5 49.5 Control learners 20.4 32.0 64.4 47.5 40.0 Western IRI learners 70.7 48.9 84.8 85.5 69.9 In Western province where only IRI learners were tested, the overall mean was exceptionally higher than all provinces. However, this could be due to the fact that the sample of learners tested was smaller compared to other IRI learners tested in other provinces. Grade 3 Evaluation Report, 2006 11 3.4.2 Mean scores by sex The Grade 3 overall test results show a difference in performance between girls and boys. In both IRI and control groups, boys perform better than girls in all but the science subtest, where they are at par with girls as shown in the table below. Except for math in the case of IRI learners, the differences in means significant differences (see Table 2 in Appendix B). Table 16: Percent mean scores by subtest and sex Type of school Sex Maths English Science Social Studies Male 41.5 34.0 65.6 69.0 IRI learners Female 36.7 31.7 65.6 67.0 Male 30.7 26.0 65.6 63.0 Control learners Female 25.6 25.1 65.9 62.0 While we expected boys in control groups to perform better than girls, IRI has typically been an equaliser in performance among boys and girls in past evaluations that were conducted at lower grades (2001, 2004). 3.4.3 Mean scores by age More than half of the learners sampled are 11 years or older. 76.0 percent of Grade 3 learners are older than 9 years, the expected age at Grade 3 (see Table 9). As can be expected, age had a positive relationship with the performance of the learners. The mean scores (percent) increased with their age, with learners who were below the recommended age for Grade 3 (8 years and below) performing the lowest in the overall test (29.7 percent), and those who were oldest (14 years and above) posting the highest performance (57.9 percent) on the overall test. The differences between the means are significant.1 Table 17: Percent mean scores, by type of school and age category Age Category School type Mathematics English Science Social Studies 8 years and IRI learners 31.5 16.4 24.5 34.7 below (21) Non-IRI learners 21.4 5.9 54.8 55.8 9 to 10 years IRI learners 29.5 21.5 55.8 57.3 (199) Non-IRI learners 25.5 17.1 61.5 63.8 11 to 13 years IRI learners 37.1 29.1 64.9 65.8 (412) Non-IRI learners 26.3 24.7 64.4 55.4 14 years and IRI learners 46.4 36.3 72.5 76.4 above (110) Non-IRI learners 29.3 28.8 66.9 65.2 While the pattern of performance with IRI learners was clearly that of increasing scores with increasing age, older learners in control schools did not necessarily perform better in Social Studies as shown in Table 17. It is not clear why age was not an advantage in Social Studies. Also, when disaggregated by age, the youngest learners in control school learners performed better that IRI learners in Science and Social Studies. 1 Mathematics: F =8.34, p =.00; English: F=18.97, p = .01, Science: F=20.93, p = .00, Social Studies: F=8.67, p = .00 Grade 3 Evaluation Report, 2006 12 3.4.4 Percent mean scores by school type and orphan status Orphan status was shown earlier to be related to attendance, with children who have no living parents attending school the least (70.6 percent in high and medium attendance category). Also, children who have one parent, the mother, registered the highest attendance (90.6 percent in high and medium attendance category). Table 18 presents performance by orphan status. Table 18: Percent mean scores, by type of school, orphans status and subtest School Type Orphan Status Maths English Science Social Studies Total Score Both parents alive (266) 30.9 25.8 62.0 63.6 42.2 Only father alive (39) 38.5 28.5 64.3 59.3 44.8 Only mother alive (122) 42.3 30.7 69.7 72.3 50.0 IRI Schools No parents alive (60) 39.2 31.4 69.0 67.3 48.5 Both parents alive (115) 25.3 0.2 62.6 60.5 37.5 Only father alive (6) 17.8 31.3 58.9 38.1 35.7 Only mother alive (48) 30.2 23.8 62.9 62.3 41.4 Community Control Schools No parents alive (31) 24.9 34.4 66.6 57.3 43.7 Comparison by school type shows that children who have both parents and orphans in IRI schools performed better than those in non-IRI control schools on the overall test. By orphan status, children who have the one parent being the mother not only attend school more; they also perform better than children with both parents, in IRI schools and in non-IRI control schools. The data further shows that even though they registered the lowest attendance (see Table 13), double orphans perform better than children who have both parents, or who have their fathers as the single parent. Since the data shows that mothers do better at nurturing attendance and performance, a possible explanation who be that primary caregivers are women, and that mothers figures in the lives of double orphans provide the same kind of nurturing. The differences between the means are significant.2 3.5 Other characteristics affecting performance at Grade 3 A number of other factors influenced learning and performance. These included teacher characteristics and factors in the immediate learning environment. 3.5.1 Teacher experience The gender, age and attendance of learners had an impact on their performance. In addition, several other characteristics had an impact on learning achievement. Table 19 shows that the learners achieved better results with more experienced teachers. Learner taught by teachers and mentors with 1-2 years of teaching experience had an aggregate score of 39.6 percent, compared to 46.7 percent for learners with teachers who had 3-4 years teaching experience. Table 19: Percent mean score by type of school and teacher experience School type 1 - 2 years 3 - 4 years IRI Community Schools Mean 48.5 55.0 Percent 44.0 50.0 Control Community Schools Mean 43.4 61.4 Percent 40.0 56.0 Aggregate Percent 39.6 46.7 2Mathematics: F =8.56, p =.00; English: F=5.50, p = .01, Science: F=15.41, p = .00, Social Studies: F=4.83, p = .01 Grade 3 Evaluation Report, 2006 13 Teachers with more experience had learners performing better both IRI schools and non-IRI control schools. These results confirm the notion that experienced teachers are better able to help children learn and indicate that it is important to retain teachers so that their learners can benefit from that experience, both in IRI and non-IRI schools. However, in the case of new teachers of 1 or 2 years, children performed better in IRI centers, a support for the claim that the IRI provides the much needed guidance for new teachers. Inversely, children performed better in non-IRI control schools in the case of more experienced teachers. This could be because community schools do have some trained teachers – teachers who are more adaptable in the long run. The result also indicates that while IRI with mentors with no formal teachers training qualification can be a good solution, it does not substitute the need for formal training. 3.5.2 Number of classes taught in the school year Teachers in community schools and IRI centres typically teach more than one class. Table 20 shows that overall test score for learners declined with increased number of classes the teacher taught in 2006. This shows that a teacher can comfortably manage to teach 1 to 2 classes and still be efficient. This result was more significant in IRI community schools than in IRI centre. Table 20: Mean score for overall test by number of classes the teacher taught in 2006 1 or 2 classes 3 classes or more IRI centres Mean 46.8 44.4 Percent 43 41 IRI community Mean 57.8 49.1 school Percent 53 45 All IRI schools Mean 53 47.1 Percent 49 43 3.5.2 Location of the school Learners in urban areas tended to perform better than learners in rural areas in both the IRI schools and the control schools as shown in Table 21.The difference in scores of urban and rural learners in IRI schools at 5.1 percent, while the difference in control community schools is 6.9 percentage points. The disparity in performance was slightly wider in control community schools than in IRI schools. This for IRI is not a good outcome; IRI has in past evaluations been able to mitigates the effect of attending school in a rural area. Table 21: Percent mean scores by locality and type of school Location School Mathematics English Science S. Studies Total Score IRI N 319 319 319 319 319 Percent 34.1 23.2 60.6 69.0 42.7 N 142 142 142 142 142 Rural Non-IRI Control Percent 25.9 15.5 63.0 61.2 37.3 IRI N 222 222 222 222 222 Percent 39.7 35.0 66.7 58.7 47.8 N 59 59 59 59 59 Urban Non-IRI Control Percent 26.9 37.0 64.1 55.6 44.2 Data further shows that the greatest disparities between learners in urban and in rural schools, for both IRI (11.8 percent) and non-IRI learners (21.5 percent) was in English. This is understandable in that urban learners are more exposed to the use of English than rural learners. Grade 3 Evaluation Report, 2006 14 Apart from the issues around exposure to the English language, learners in rural areas often have other obstacles to learning that are not faced by learners in urban areas. Rural areas also tend to have fewer materials. The IRI program mitigates the effects of learning in a rural area. Each mentor’s guide offers teachers 150 prepared lesson plans for the year, making it easier for teachers to prepare their lessons. The lessons attempts to provide children with curriculum-based activities that use locally available materials. Grade 3 Evaluation Report, 2006 15 4.0 DISCUSSION AND RECOMMENDATIONS 4.1 Summary 4.1.1 Demand for IRI The demand for IRI continues to increase as a number of new IRI centres are opened, and more and more community schools use LTM in their classrooms. The number of IRI centres and community schools increased from a total of 857 in 2005 to 1,022 in 2006, while the number of learners increased from 56,233 in 2005 to 81,324 in 2006. While a relatively small amount of this growth is due to the expansion of IRI to Grade 6 in 2006, the majority of the growth is attributed to the increasing number of IRI centres and community schools that open to provide children with access to education. This means that the public continues to turn to IRI to provide education for children who would not otherwise receive education. The results also show that a small number of centres (about 9 percent) closed in 2004/05. This does not indicate a lack of demand in those communities. The success of IRI centres depends on effective centre committees that can rally communities to pool their resources to support volunteer teachers, build shelters or classrooms that can be used as teaching space, and engage in other activities that sustain the school. When such support is not forthcoming, centres close down only to reopen again a year or so later in most cases. 4.1.2 Characteristics of the IRI Population Three features of the IRI population that have been observed in past evaluations were again observed in 2006: 1. The IRI population continues to be almost equally divided between boys and girls. 2. IRI learners tend to be older than the recommended age for their grade level. 3. Orphans make up a high proportion of IRI learners The overall 2006 ratio of 50.2 percent boys and 49.8 percent girls indicates that IRI continues to provide equal access to learning for girls and boys. However, 2006 figures indicate that the ratio of girls to boys follows a trend similar to what is seen in GRZ schools: lower percentages of girls are enrolled at higher grade levels, which means that attrition for girls are higher than for boys (Table 8). QUESTT should flag this result to several partner organisations that work in programs that promote girls education at the middle basic level. They may find the information useful as evidence for pressing for more girls’ empowerment efforts. IRI learners are typically older than the recommended age in that IRI centres admit children and young adults of all ages. The data shows on average that 23.8 percent learners are at the recommended age for their grade level; about 60.1 percent are above the recommended age for their grade while 13.3 percent are younger. Most of the younger learners find themselves in school by default when their older siblings who are caregivers bring them along to IRI centres during the time of the broadcast. The number of orphans enrolled in IRI centres increased from 20,244 in 2005 to 27,481 in 2006. Given the overall increase in the IRI population, orphans continued to represent about the same proportion of learners in 2006 (about 34 percent) as they did in 2005. And as in 2005, the proportion of orphans in IRI centres was greater than the proportion of orphans in GRZ schools, which report to have 20 percent orphans in 2006. Twenty-three percent (23) of the IRI learners were single orphans, while 12 percent were double orphans in 2006..At 53 percent, Grade 3 Evaluation Report, 2006 16 Western province had the highest number of orphans, followed by Lusaka province. It may also be desirable to isolate Western province for more monitoring in the remaining years of the program to investigate the observed increase in the number of orphans and how the effects may be mitigated. In summary, IRI centres and community schools provide girls and boys with equal access to education; they also provide orphans with greater access to education than the government schools. In addition, older learners are able to attend school when the opportunity is availed to them. These characteristics of the IRI population indicate that IRI continues to serve vulnerable children well. 4.1.3 Attendance of IRI Learners Attendance data was obtained for 83.8 percent of the learners in the sample (16.2 percent of learners had no attendance data). 80.0 percent of the learners in the sample had high or medium attendance, which means that they attended more than 60 percent of the time. This level of participation in IRI broadcasts is a vote of confidence for the program in IRI centres and community schools do attend With double orphans registering the least frequent attendance (70.6 percent in high/medium category attendance), followed by children coming from homes with the father as the living parent. Children who had a mother as their only living parent had the best attendance (90.9 percent in high/medium category attendance). Low attendance rates in comparison to other children suggests that orphaned children do not receive the necessary guidance and support when it comes to attending school, either from caregivers or because they are heads of households. There are often speculations that double orphans do more than their fare share of chores in their foster homes, some of which keep them from attending schools. The finding that children are more likely to attend school if they have one parent, the mother, than if they have a father only or both parents is a significant outcome in terms of the gender aspects of guiding younger children. QUESTT may want to put more emphasis on finding out why children who have a father as their assisting fathers with their parenting roles through their OVC programs. Also, with the preponderance and level of funding of OVC programs in Zambia, these information should be shared with other organization for systematic study. 4.1.4 Achievement Learners in IRI centres performed better than control learners in community schools in the areas of maths, English and social studies. IRI learners at community schools obtained similar scores to their counterparts in the control group in science. While there was a clear advantage in being in an IRI class for other subjects, science content is generally more difficult and needs specialist knowledge that mentors may not have. Science teaching also requires specialised resources which are not be available in IRI schools. While it has always been intention of the QUESTT project to put additional emphasis on the teaching of science, more in-service training should be provided. In particular, mentors should be shown how to prepare for Science lessons using materials in their immediate environments. Learners in urban schools performed better than learners in rural schools; however, IRI mitigated the effect of attending a rural school. The gap in performance between urban and rural learners was lower among IRI learners (4.2 percentage points) than among learners in control schools (17.7 percentage points). In addition, IRI learners in rural schools had a higher overall Grade 3 Evaluation Report, 2006 17 test score (48.2 percent) than control learners in rural schools (43.8 percent). IRI mitigates the effects of learning in a rural area by providing teachers with a mentor’s guide of prepared lessons and by providing children with curriculum-based activities that use locally available materials. Characteristics that had an impact on performance were age, attendance, and orphan status. Boys performed better than girls, although the gender gap was lowest (2.1 percentage points) for children who were at the recommended age for Grade 3 (9 years old). Generally, older children performed better than younger children, with underage children performing far below those at the recommended age for Grade 3 in some subjects. In fact, the result on performance of under￾aged children have been consist enough to influence policy on the progression of children from one grade level to the next. The Ministry of Education should consider introducing a policy of assessed progression for under-aged learners. This would mean that performance of underage children is carefully assessed to see if they are ready to be promoted to the next grade, and retain them if necessary. This should be done by schools and the district education offices, with direct input from parents. The policy should also set a ceiling on the proportion of children that can be retained ii a class. A number of teaching characteristics also had an impact on learning performance. Teachers with more experience had learners performing better. Longer teaching experience seemed to be even more of an asset for community school teachers, a few of which have a formal teaching qualification, while newer teachers realised better learner performance. This result indicates that it is important to retain teachers. It also illustrates the effectiveness of IRI – the use of the radio teacher to provide guidance in case of less experienced classroom teachers, and to help the teacher lead the class in learner-centred activities. Over time IRI teachers are able to model the correct learner-centred behaviours. Plans are underway to provide training in IRI to all new teachers through a distance teacher training course to improve the quality of instruction. The test scores were also analyzed according to the number of classes taught by the teacher. IRI learners who had a teacher that taught one or two classes performed better than learners with a teacher who led three classes or more. This seems to suggest that teachers should not be required to teach more than two classes. While it is sometimes difficult for IRI learning centres and Community Schools to recruit enough teachers to meet this guideline, administrators and school committees should be aware that having a teacher lead more than two classes will be detrimental to their children’s performance, and provide more support to teachers who teach more than two classes. 4.2 Recommendations A number of suggestions for improvement were made in the discussion of findings. Some of these are recommended for further action. 1. Work aggressively with other organisations to mitigate of attrition among girls at the middle basic level. Now that it is clear that attrition rates for girls are high even for IRI schools, QUESTT should pay special attention to gender dynamics at the middle school level, first by accessing the body of knowledge that already exists, and supporting already existing programmes for retaining girls in schools. 2. Monitor the impact grants programmes in IRI centres and community schools. Two of the factors that are suspected to affect performance in rural schools were a lack of learning materials and the number of weeks that the learners were not taught. Income generating activities would provide IRI centres and community schools with the income needed to Grade 3 Evaluation Report, 2006 18 compensate teachers adequately so that they remain at the school and teach classes regularly. The income should also be used to buy school supplies, such as chalk, pencils and exercise books. 3. Underage children should be assessed before being promoted to the next grade level. QUESTT should work with MOE to develop policy guidelines for assessed progression for under-aged learners. Younger children would benefit more from repeating a grade proceeding on to the next grade. This approach would work if teachers are clear of expectations at each grade level, and have the adequate skills in assessment and using assessment information to make decisions. 4. QUESTT should use the information on attendance and performance of orphans to strengthen programming for the OVC programs. In particular, emphasis should be put on training in gender roles in the care of orphans. Information should also be shared with partner organisations with OVC programs. 5. Teachers should be NOT teach more than two classes. While it is sometimes difficult for IRI centres and Community Schools to recruit enough teachers to meet this guideline, administrators and school committees should be aware that having a teacher lead more than two classes will be detrimental to their children’s performance. Grade 3 Evaluation Report, 2006 19 Grade 3 Evaluation Report, 2006 20 REFERENCES DODE & QUESTT. 2005. 2005 IRI Enrolment Bulletin. Lusaka: Directorate of Open and Distance Education, QUESTT Project and USAID. [Cited as DODE & QUESTT, 2005, 2005 IRI Enrolment] DODE & QUESTT. 2006. 2005 IRI Enrolment Bulletin. Lusaka: Directorate of Open and Distance Education, QUESTT Project and USAID. [Cited as DODE & QUESTT, 2006, 2005 IRI Enrolment] DODE & QUESTT. 2005. Learning at Taonga Market in Government Schools: An Evaluation of the 2005 Pilot of Interactive Radio Instruction in Grade One. Lusaka: Directorate of Open and Distance Education, QUESTT Project and USAID. [Cited as DODE & QUESTT, 2005, LTM in GRZ Schools] DODE & QUESTT. 2005. Learning at Taonga Market: IRI Learning Centres and Community Schools: An Evaluation of Interactive Radio Instruction at Grade One in 2005. Lusaka: Directorate of Open and Distance Education, QUESTT Project and USAID. [Cited as DODE & QUESTT, 2005, LTM in IRLCs] Kariuki, S., Letshabo, K., Laflin, M., Mutale, K., Phiri, M. and Sakala, G. 2000. October 2000 Evaluation of the Interactive Radio Pilot for Out-of-School Audiences. Lusaka: USAID-Zambia. Letshabo, K. 2003. Interactive Radio Instruction for Out-of-School Children and Youth in Zambia: An Evaluation Report for Grade One Learners, 2003. Lusaka: Educational Broadcasting Services, Ministry of Education. Letshabo, K. and Kariuki, S. 2001. Interactive Radio Instruction for Out-of-School Children and Youth in Zambia: An Evaluation Report for Grade One Learners,2001. Lusaka: Educational Broadcasting Services, Ministry of Education. Mackay, D., Thompson, B. and Schaub, P. 1970. Breakthrough To Literacy. London: Longman for the Schools Council. Republic of Zambia Ministry of Education. 2006. Educational Management Information System. Lusaka: Ministry of Education. Republic of Zambia Ministry of Education. 1996. Educating Our Future. Lusaka: Ministry of Education. Siaciwena, R., Thomas, M., Luangala, J. and Haambokoma, C. 2002. Report on the Rapid Appraisal of the Ministry of Education’s Interactive Radio Instruction Basic Education Delivery System. Lusaka: USAID-Zambia. Grade 3 Test 21 APPENDIX A: GRADE 3 TEST • Read this information before the test and seek clarification where necessary. • Ask the class teacher rate each learner’s Zambian language ability with the guidance of the “Descriptors for Language Mastery”. Select 7 children from each category— High, Medium and Low—to be tested. • Record the names of the pupils in the same order on each of the scoring grids. Record the Zambian language rating of Low, Medium or High on the English Language scoring grid. • Everything that appears in italics in the test is for the information or direction of th /test administrator. Please do not read such words to the learner. Words that are supposed to be read to the learners are NOT in italics. • Each test question may be presented to the learner a maximum of two times. • Administer the test in the following order: 1. Mathematics: Administer all questions as a group. Read each question to the group. Go to the next question when everyone has had time to answer. Translation into the local Zambian language is allowed on all mathematics questions. 2. English Language: Administer all questions as a group. Read each question to the group. Go to the next question when everyone has had time to answer. All questions and instructions must be given in English. 3. Science: Administer all questions to one learner at a time, away from other learners. Translation into the local Zambian language is allowed on all science questions. 4. Social Studies: Administer all questions to one learner at a time, away from other learners. Translation into the local Zambian language is allowed on all social studies questions. • To keep the testing time as short as possible, three administrators should deliver the test—one for each section. After the Mathematics and Literacy Tasks 1 and 2 are completed, have the learners go from the literacy administrator to the English language administrator. • Use the scoring grid to record scores. For Mathematics and Literacy Tasks 1 and 2, record the scores after collecting the learners’ writing. For questions given individually, record a score as the learner gives a response. Score ranges for each question are shown at the top of each column on the grid. Record NR for No Response. • Sampling among boys and girls at the centers should be proportional. • Before starting the test, test administrators should write the names and information about the sampled learners on the answer sheet for Literacy Tasks 1 and 2 and the scoring grids. Grade 3 Test 22 MATHEMATICS SCORING RUBRIC, GRADE 3 x Administer all Mathematics tasks to a group of children. x Ensure that the children write their names properly on the mathematics answer sheets. x Guide the students through each question until they have had enough time to give an answer. x Translation into the local Zambian language is allowed on all questions. x Begin by greeting and introducing yourself. x Encourage children to write calculations on the page. My name is ____________________________________ I am a boy / girl . I am _______ years old. 1. 4 2 1 5 1 point if some figures + 3 1 1 are correct. 4 5 2 6 2 points for the correct answer. 2. 1 7 7 9 1 point if some figures - 1 4 3 3 are correct. 3 4 6 2 points for the correct answer. NR 0 1 2 NR 0 1 2 3. 8 9 7 6 1 if carried correctly + 6 4 8 only once. 9 6 2 4 2 if carried correctly only twice. 3 points for correct answer. 4. 4 2 5 1 if the learner 3 7 6 borrowed correctly __4 9 only once 2 if borrowed correctly only twice. 3 points for correct. answer. NR 0 1 2 3 NR 0 1 2 3 5. 300 300 x 3 300 900 300 1 if 9 appears in a wrong answer. 2 for the correct answer using repeated addition. 3 for the correct answer using multiplication. 6. 2 4 6 246 x 4 246 9 8 4 (3 points) 246 246 1 if one is multiplied correctly. 2 for the repeated addition or for two correct numbers in the answer. 3 for the correct answer. NR 0 1 2 3 NR 0 1 2 3 82 1 if one figure is divided 7. 4 328 correctly. 2 for the correct answer. 8. Fill in the missing numbers: 6, 12,18, 24, 30, 36, 42 1 for one correct number. 2 for two correct numbers. 3 for correct numbers. NR 0 1 2 NR 0 1 2 3 Grade 3 Test 23 9. Please complete this number by using any two factors. X = 12 2 X 6 3 X 4 1 if one factor is correct. 2 for two correct factors. 10. You have K1000 and you buy onions for K450. How much is your change? I will bring K 550 change. NR 0 1 2 NR 0 1 11. You begin walking from home at 11:15 hours and you reach the centre At the time shown in the clock face. At what time did you arrive at the center? _11: 30 How many minutes did you walk? 15 Minutes 1 12 12. Read the sentence below. a. Monday ∉ {Sunday, Monday, Tuesday} b. Monday ∈ {Sunday, Monday, Tuesday} Sentence _B_ is correct. 1 for the correct answer. NR 0 1 2 NR 0 1 13. Write the fraction of the shaded part of the square below. 1/4 1 for a proper fraction other than 1/4 2 if the answer is correct. NR 0 2 Grade 3 Test 24 MATHEMATICS, GRADE 3 (Final Version) My name is ____________________________________ I am a boy / girl . I am _______ years old. 1. 4 2 1 5 + 3 1 1 _______ 2. 1 7 7 9 - 1 4 3 3 _______ NR 0 1 2 NR 0 1 2 3. 8 9 7 6 + 6 4 8 _______ 4. 4 2 5 - 3 7 6 ______ NR 0 1 2 3 NR 0 1 2 3 5. 300 x 3 ______ 6. 2 4 6 x 4 _______ NR 0 1 2 3 NR 0 1 2 3 7. 4 328 8. Fill in the missing numbers: 6, 12, 18, ___, 30, ___, ___ NR 0 1 2 NR 0 1 2 3 Grade 3 Test 25 9. You have K1000 and you buy onions for K450. How much is your change? I will bring K______ change. 10. You begin walking from home at 11:15 hours and you reach school at the time shown on the clock. a. What time did you arrive at school? I arrived at __________ hours. b. How many minutes did you walk? I walked for ______ minutes. NR 0 1 NR 0 1 2 11. Read the sentences below. A. Monday ∉ {Sunday, Monday, Tuesday} B. Monday ∈ {Sunday, Monday, Tuesday} Which is correct: A or B? Sentence _____ is correct. 12. Write the fraction of the shaded part of the square below. _____ of the square is shaded. NR 0 1 NR 0 1 2 Well done! Thank you for answering the questions. Grade 3 Test 26 Grade 3 English Language Administration Guidelines and Scoring Rubrics x Administer all English Language tasks to a group of children. x Pass out the answer sheets to the English tasks. Ensure that the children write their names properly on the first page. x Guide the students through each question until they have had enough time to give an answer. x All instructions must be given in English. No translation is allowed. x Begin by greeting and introducing yourself. Hello, my name is __________. I am very happy to see you today. We are going to listen to a story. Then I will ask you some questions about the story. x Read the listening questions on the first page with the children. x Ask the children to turn their answer sheets over and listen to the story. x Read the story slowly and with good intonation. Task 1: Listening Comprehension Mr. Zulu lives in a house with a grass roof. The roof needs new grass. Mr. Zulu has a broken leg. He asks his friend Mr. Sitali to put new grass on his roof. Mr. Zulu says, “Mr. Sitali my friend. You can see that I have a broken leg, but my roof needs new grass. Can you help me, my friend? Can you put new grass on the roof for me?” Mr. Sitali says, “Yes, I will help you. I will put new grass on the roof for you.” The next day Mr. Sitali comes to Mr. Zulu’s house to start work. He puts the ladder next to the house. He climbs the ladder and starts putting the grass on the roof. Suddenly, a cow comes running into the yard. The cow is running fast. It hits the ladder. The ladder falls over. Down falls the ladder! Down falls Mr. Sitali! Mr. Zulu asks, “Are you alright, my friend? Mr.Sitali says, “Yes, but look at your ladder. It’s all in pieces!” Mr. Zulu says, “Be careful my friend or you will end up like me!” x Guide the students through each question until they have had enough time to give an answer. x When everyone has finished, tell the children to turn to the next page and read the story. Grade 3 Test 27 Listening, Grade 3 My name is ____________________________________ Instructions: Choose the best answer. Write A, B or C. Example: Who is Mr Sitali? A. Mr. Zulu’s doctor B. Mr. Zulu’s brother C. Mr. Zulu’s friend Answer: C 13) What is wrong with Mr. Zulu? A. He is lazy. B. He needs money. C. He has a broken leg. Answer: C 14) What does Mr. Zulu ask Mr. Sitali to do? A. Buy grass for the roof B. Put grass on the roof C. Cut grass in the bush Answer: B 15) What happens when the ladder falls over? A. The ladder breaks. B. Mr. Sitali hurts his leg. C. Mr. Zulu falls down. Answer: A 16) What is the story about? A.Mr. Sitali helps Mr. Zulu. B. Mr. Sitali builds a new house. C. Mr. Sitali finds his cow. Answer: A Grade 3 Test 28 Task 2: Reading Comprehension x Ask the children to read the story and give them time to read the passage on their own. DO NOT READ THE PASSAGE ALOUD FOR THE CHILDREN. THEY MUST READ INDIVIDUALLY. x Guide the students through each question until they have had enough time to give an answer. Students may refer back to the reading passage while answering questions. x When everyone has finished, tell the children to turn to the next page and give the children instructions for the next task. Instructions: Read the story. A boy is sitting in school. His teacher is Mrs. Chanda. She wants him to write his name. Mabvuto says to himself, “Mabvuto means “problems”. But I am not a problem. I do not have problems. I need a new name.” Mabvuto sits and thinks. Then he thinks of a new name. He writes his new name on the paper and then he smiles. The word on the paper is “Gift”. Instructions: Choose the best answer. Write A, B or C. 17) Where is the boy? A. At home B. At school C. At church Answer: B 18) What does the name “Mabvuto” mean? A. Gift B. Happy C. Problems Answer: C 19) What is the boy’s new name? A. Gift B. Mabvuto C. Problems Answer: A 20) How does the boy feel at the end of the story? A. sad B. angry C. happy Answer: C Scoring: One point for each correctly selected answer. Grade 3 Test 29 Writing Tasks x Guide the students through questions 21, 22 and 23 until they have had enough time to give an answer. x All instructions must be given in English. No translation is allowed. 21. Writing about 2 Things Instructions: Write the words in the blank spaces like the example. Example: 1 hat 2 ___hats___ 1 boy 2 __________ 1 shoe 2 __________ 1 baby 2 __________ 1 knife 2 __________ Scoring: One point for each correctly written answer. 22. Punctuation Instructions: Rewrite the sentences using correct punctuation. Use capital letters, question marks (?) and full stops (.). 1. what are you doing_ 2. he is writing to his friend bupe_ 3. she went to ndola in may_ Scoring: One point for each required capital letter and punctuation mark. Marks will not be awarded for capital letters if the learner uses capital letters for the whole sentence. The lowest score for each sentence is 0. The highest scores are: a) 2 points, b) 3 points and c) 4 points, for a total of 9 possible points. Grade 3 Test 30 23. Dictation Walk among the group to show everyone the picture of the goat before saying the each dictation sentence aloud. x Repeat each dictation sentence three times: one time clearly at regular speed and two times slowly as the students write and check their writing. Instructions: Look at the picture and listen to the sentence. Write the sentence on your answer sheet in English. I will say each sentence three times. a) Look at the goat. b) It is standing on two legs. c) It is eating leaves. Scoring: One point for each correctly spelled word, for 14 possible points. Do not score punctuation. x Collect the answer sheets as the children complete their work. As the children finish, they may proceed to the science test. Grade 3 Test 31 Listening, Grade 3 My name is ____________________________________ Instructions: Choose the best answer. Write A, B or C. Example: Who is Mr Sitali? D. Mr. Zulu’s doctor E. Mr. Zulu’s brother F. Mr. Zulu’s friend Answer ____ 13) What is wrong with Mr. Zulu? A. He is lazy. B. He needs money. C. He has a broken leg. Answer ____ 14) What does Mr. Zulu ask Mr. Sitali to do? A. Buy grass for the roof B. Put grass on the roof C. Cut grass in the bush Answer ____ 15) What happens when the ladder falls over? A. The ladder breaks. B. Mr. Sitali hurts his leg. C. Mr. Zulu falls down. Answer ____ 16) What is the story about? A. Mr. Sitali helps Mr. Zulu. B. Mr. Sitali builds a new house. C. Mr. Sitali finds his cow. Answer ____ Grade 3 Test 32 Reading, Grade 3 Instructions: Read the story. A boy is sitting in school. His teacher is Mrs. Chanda. She wants the boy to write his name. Mabvuto says to himself, “Mabvuto means ‘problems’. But I am not a problem. I do not have problems. I need a new name.” Mabvuto sits and thinks. Then he thinks of a new name. He writes his new name on the paper and then he smiles. The word on the paper is “Gift”. Instructions: Choose the best answer. Write A, B or C. 17) Where is the boy? A. At home B. At school C. At church Answer ____ 18) What does the name “Mabvuto” mean? A. Gift B. Happy C. Problems Answer ____ 19) What is the boy’s new name? A. Gift B. Mabvuto C. Problems Answer ____ 20) How does the boy feel at the end of the story? A. Sad B. Angry C. Happy Answer ____ Grade 3 Test 33 21. Writing about 2 Things Instructions: Write the following words in plural form. Example: 1 hat 2 hats . 1 boy 2 __________ 1 shoe 2 __________ 1 baby 2 __________ 1 knife 2 __________                                   22. Punctuation Instructions: Rewrite the sentences using correct punctuation. Use capital letters, question marks (?) and full stops (.). a. what are you doing_ __________________________________________________ b. he is writing to his friend bupe_ __________________________________________________ c. she went to ndola in may_ __________________________________________________ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ 23. Dictation Instructions: Write the sentences. a. __________________________________________________ b. __________________________________________________ c. __________________________________________________ Grade 3 Test 34 GRADE 3 – SCIENCE TEST x Administer all science questions to one learner at a time, away from other learners. x Questions can be given in English or translated into the local language of play. x Begin by greeting and introducing yourself. 24. I will mention three things. Please tell me which ones can dissolve in water. a. Sand b. Sugar c. Salt If a child mentions one thing and stops, ask for another thing that dissolves in water. One point for each correct answer: Sand does not dissolve. Sugar dissolves. Salt dissolves. 25. If you eat this most of the time, is it a balanced diet? a. nshima, meat, kapenta b. nshima, meat, vegetables c. meat, nshima, rice One point for each correct answer: a. No b. Yes c. No NR 0 1 2 3 NR 0 1 2 3 26. I am going to mention body parts and their uses. Tell me the pairs that go together. For example, “skin” and “touch” go together. a. Tongue and taste: Do they go together? b. Nose and see: Do they go together? c. Eye and smell: Do they go together? d. Ear and hear: Do they go together? One point for each correct answer: a. Yes b. No c. No d. Yes 27. Germs can be transmitted through: a. Water b. Talking on the telephone with a sick person. c. Breathing air in a room where someone is coughing One point for each correct answer: a. Yes b. No c. Yes NR 0 1 2 3 4 NR 0 1 2 3 28. Mention two ways through which people can get HIV/AIDS: 1. Unprotected sex./Not using a condom. 2. Razor blades, needles. 3. Blood transfusion. [If a child mentions variations of unprotected sex, ask for another way of getting HIV.] Accept up to two different correct answers. 29. Give two reasons why drinking a lot of beer is bad: 1. Leave family. 2. Waste money. 3. Bad behaviour. 4. Destroy the body. 5. Destroying property [If a child mentions variations of the same reason, ask for a different reason.] Accept up to two different correct answers. NR 0 1 2 NR 0 1 2 30. Name three objects that can allow heat to pass through. 1. Pot 6. Pan 2. Spoon 7. Kettle 3. Iron bar or wire 8. Axe 4. Nail 9. Hoe 5. Knife 10. Slasher Accept up to three different correct answers. 31. What animals do people keep at home? How are they helpful to people? 1. Dogs for security. 2. Cows for ploughing / meat/milk/selling. 3. Chickens for selling / food/ relish. 4. Cats for killing rats / snakes. 5. Goats for milk / meat. Give one point for each way in which animals are helpful for up to three points. A learner can give three uses for one animal or the same use for three different animals. NR 0 1 2 3 NR 0 1 2 3 32. Give two reasons why it is important to take a child to the under-five clinic for immunizations. 1. Discover that the child is sick. 2. Protect the child against disease. 3. Progress in growth of the child. Accept a partially correct answer. 33. Mention two ways that you can make water safe for drinking. 1. Boiling 2. Adding chlorine Accept a partially correct answer. NR 0 1 2 NR 0 1 2 Grade 3 Test 35 GRADE 3 – SOCIAL AND DEVELOPMENT STUDIES x Administer all social studies questions to one learner at a time, away from other learners. x Questions can be given in English or translated into the local language of play. x Begin by greeting and introducing yourself. 34. Mention one person who works at a clinic. Nurse, doctor, health worker Accept one correct answer. 35. Which one of these diseases has no cure? a. HIV/AIDS b. Malaria c. Measles NR 0 1 NR 0 1 36. Why do you always wash your hands before eating your food? 1. To clean hands 2. To avoid diseases Accept a partially correct answer. 37. Name three types of transport used in Zambia. 1. Bicycle, boats, Aero-plane, Train 2. Types: Road, Water, Air and Rail Accept if a learner gives “types” of transport. Award up to three marks. NR 0 1 2 NR 0 1 2 3 38. Give two types of work done by a leader in your community, such as a Chief, a Headman or a Councilor. 1. To settle disputes. 2. To rule/ administer. 3. To allocate plots. 4. To provide security Accept a partially correct answer. 39. Name three ways of sending a message to someone who lives far from your community. 1. Letter 4. Drumming 2. Phone 5. T.V. 3. Radio 6. Word of mouth If a child mentions one thing and stops, ask for another way of sending a message. One point for each correct answer up to 3 marks. NR 0 1 2 NR 0 1 2 3 40. Give two reasons why is it important to follow rules in school or in the home? 1. One can be a well- behaved person. 2. To grow up as a responsible person. 3. To finish school. 4. To live longer. 5. To live well in the future. 6. To avoid trouble Award up to two marks. 41. Give two reasons why it is important to look after plants and trees. 1. Fruits 2. Poles for shelter. 3. Good air. 4. Wood for furniture. Award up to two marks. NR 0 1 2 NR 0 1 2 41. Give two reason why it is important to look after rivers and lakes. 1. Water for drinking. 2. Water for cooking. 3. Water for washing body. 4. Water for washing clothes. 5. Fish Award up to two marks. 43. Mention two ways that people help each other in the community. 1 Giving. 2 Provide foods. 3. Provide transport. 4. Helping the sick. 5. Visitation. 6. Lead the blind. Award up to two marks. NR 0 1 2 NR 0 1 2 Well done, and thank you for answering the questions. Grade 3 Mathematics [27 points] School Name: ___________________ District: _________________ Province: ________________ How often does someone at home help you with maths? 1 Add 2 Sub￾tract 3 Add 4 Sub￾tract 5 Mult￾iply 6 Mult￾iply 7 Divide 8 Pattern 9 Money 10 Time 11 Sets 12 Name Frac-tions A g e S e x 1=Never 2=Only when stuck 3=Sometimes 4=Often 0 - 2 0 - 2 0 - 3 0 - 3 0 - 3 0 - 3 0 - 2 0 - 3 0 - 1 0 - 2 0-1 0 - 2 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. Grade 3 Test 36 English Language [35 points] School Name: ___________________ District: _________________ Province: ________________ Zambian Language Ability [Get from teacher.] How often does someone at home help you with reading in a Zambian language? Task 1 13 Listen Comp. Task 1 14 Listen Comp. Task 1 15 Listen Comp. Task 1 16 Listen Comp. Task 2 17 Read Comp. Task 2 18 Read Comp. Task 2 19 Read Comp. Task 2 20 Read Comp. Task 3 21 Plurals Task 4 22 Punc￾tuation Task 5 23 Dictation Name H = High M= Medium L = Low 1=Never 2=Only when stuck 3=Sometimes 4=Often 0 - 1 0 - 1 0 - 1 0 - 1 0 - 1 0 - 1 0 - 1 0 - 1 0 - 4 0 - 9 0 - 14 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. Grade 3 Test 37 Grade 3 Science [27 points] School Name: ___________________ District: _________________ Province: ________________ 24 Dissolve 25 Diet 26 Senses 27 Germs 28 HIV/AIDS 29 Beer 30 Heat 31 Animals 32 Immunise 33 Water Name 0 - 3 0 - 3 0 - 4 0 - 3 0 - 2 0 - 2 0 - 3 0 - 3 0 - 2 0 - 2 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. Grade 3 Test 38 Grade 3 Test 39 Grade 3 Social Studies [20 points] School Name: ___________________ District: _________________ Province: ________________ 34 Clinic 35 Disease 36 Wash Hands 37 Transport 38 Leader 39 Message 40 Rules 41 Plants 42 Rivers 43 Comm. Help Name 0 - 1 0 - 1 0 - 2 0 - 3 0 - 2 0 - 3 0 - 2 0 - 2 0 - 2 0 - 2 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. Grade 3 Test 40 APPENDIX B: T-TEST TABLES Table 1: t-test for Equality of Means for Table 14 Subtest t-value df Significance (2-tailed) Mean Difference Std. Error Difference English 3.88 369.4 .00 2.5 .66 Mathematics 5.39 388.0 .00 2.9 .55 Science -0.12 394.1 .90 -.05 .42 Social Studies 2.38 360.2 .01 1.1 .46 Table 2: t-test for Equality of Means for Table 16 School Type Subtest t-value df Significance (2-tailed) Mean Difference Std. Error Difference Mathematics 2.06 537.0 .04 1.2 .60 English 1.16 538.8 .25 0.8 .71 Science -0.01 536.5 .99 -0.1 .47 IRI Social Studies 0.92 536.7 .36 0.5 .48 Mathematics 1.58 198.0 .12 1.4 .91 English 0.23 199.0 .82 0.3 1.12 Science -0.21 198.6 .84 -0.1 .70 Non-IRI Control Social Studies 0.21 198.9 .84 0.2 .79