Republic of Zambia Ministry of Education _________________________________________________ Learning at Taonga Market: Community and GRZ Schools Grade 2 Evaluation of Interactive Radio Instruction __________________________________________________________ ii TABLE OF CONTENTS H1.0 INTRODUCTION................................................................................................................... 1H H1.1 BACKGROUND....................................................................................................................... 1H H1.2 PURPOSE OF THE EVALUATION.............................................................................................. 2H H2.0 EVALUATION METHODOLOGY...................................................................................... 3H H2.1 SAMPLING DESIGN ................................................................................................................ 3H H2.2 ASSESSMENT OF LEARNING ACHIEVEMENT .......................................................................... 4H H3.0 FINDINGS AND DISCUSSION ............................................................................................ 7H H3.1 DEMAND FOR IRI .................................................................................................................. 7H H3.2 PROFILE OF IRI LEARNERS.................................................................................................... 8H H3.3 ATTENDANCE...................................................................................................................... 10H 3.4 H PERFORMANCE IN MATHS, ZAMBIAN LANGUAGE, LIFE SKILLS & ENGLISH....................... 11H 3.5 H PERFORMANCE IN GRZ SCHOOLS ....................................................................................... 13H 3.6 H PERFORMANCE IN IRLCS AND COMMUNITY SCHOOLS ....................................................... 15H 3.7 H TEACHERS’ PREDICTIONS OF ZAMBIAN LANGUAGE PERFORMANCE ................................... 19H 4.0 H CONCLUSION AND RECOMMENDATIONS ................................................................ 21H 4.1 H DEMAND FOR IRI ................................................................................................................ 21H 4.2 H CHARACTERISTICS OF THE IRI POPULATION ....................................................................... 21H 4.3 H ATTENDANCE OF IRI LEARNERS ......................................................................................... 22H 4.4 H ACHIEVEMENT OF GRADE 2 GRZ LEARNERS...................................................................... 22H 4.5 H ACHIEVEMENT OF GRADE 2 LEARNERS AT IRLCS AND COMMUNITY SCHOOLS ................. 23H 4.6 H TEACHER PREDICTIONS OF ZAMBIAN LANGUAGE PERFORMANCE....................................... 24H 4.7 H RECOMMENDATIONS ........................................................................................................... 25H HREFERENCES.................................................................................................................................... 27H HAPPENDICES ..................................................................................................................................... 28H HAPPENDIX A: 2006 GRADE 2 ASSESSMENT....................................................................................... 28H HAPPENDIX B: QUESTIONNAIRE FOR GRZ TEACHERS ........................................................................ 48H iii LIST OF ACRONYMS DEBS .....................................................................District Education Board Secretary DODE...................................................... Directorate of Open and Distance Education DRCC...................................................................District Resource Centre Coordinator EBS ..........................................................................Educational Broadcasting Services EDC................................................................................Education Development Centre GRZ.................................................................. Government of the Republic of Zambia IRI ......................................................................................Interactive Radio Instruction LTM..................................................................................... Learning at Taonga Market MOE..............................................................................................Ministry of Education OVC.......................................................................... Orphans and Vulnerable Children PDP............................................................... Programme Development and Production POC............................................................................. Provincial Outreach Coordinator QAA......................................................................Quality Assurance and Accreditation QUESTT............................................Quality Education Services Through Technology SEO-ODL.................................Senior Education Officer-Open and Distance Learning TED ...............................................................................Teacher Education Department ZIP.......................................................................................... Zonal In-service Provider Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 1 1.0 INTRODUCTION 1.1 Background Learning at Taonga Market is a complementary delivery system for the basic education curriculum, using interactive radio instruction methodologies. Learning at Taonga Market lessons are written and recorded by Educational Broadcasting Services (EBS), under the auspices of the USAID-funded Quality Education Service Through Technology (QUESTT) Project. Each lesson consists of a 30-minute broadcast, along with activities that the class completes before and after the broadcast. The activities for each lesson are described in a mentor’s guide. The programme follows the national curriculum and the MOE’s calendar of three terms. There are 150 lessons at each grade level, plus five teacher training broadcasts at the beginning of each term. QUESTT is managed by the Education Development Center (EDC). In 2006 Learning at Taonga Market was broadcast to Grades 1 through 6. Table 1 shows the number of learners using IRI in 2006. Table 1: Number of IRI learners in 2006, by province and gender Province Female Male Total Central 6649 6594 13243 Copperbelt 1793 1891 3684 Eastern 8687 8501 17188 Luapula 1418 1513 2931 Lusaka 7226 7220 14446 North￾Western 3648 3528 7176 Northern 4145 4457 8602 Southern 4215 4156 8371 Western 3016 1899 4915 Total 40797 39759 80556 Grade 7 lessons will be offered for the first time beginning in 2007. In addition to its use in GRZ schools, the programme is designed to give learners in community schools and IRI centres the opportunity to complete seven years of education through the radio lessons and take the Primary School Leaving Certificate Examination. Learners who pass the grade seven examinations can attend upper basic grades in the government schools or enrol in the DODE Alternative Upper Basic Education Programme at distance learning centres. Several evaluation of Learning at Taonga Market have been conducted, the latest being Grade 1 in 2005. The major findings of the 2005 evaluation were that: x Demand in 2005 was greater than in the past with a rise in enrolment at Grades 1 to 5 from 38,513 in 2004 to 56,233 in 2005. x The programme meets the needs of girls, as the overall enrolment ratio was 50.5 percent girls and 49.5 percent boys. x Grade 1 learners made satisfactory gains in numeracy and English. x The majority of Grade 1 learners could not read at the expected level. In response to the last finding, Zambian language readers are being distributed to community schools. Also, IRI methodologies have been incorporated into the pre-service teacher education Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 2 curriculum, while additional NBTL training is being provided to mentors. An evaluation of LTM at Grade 1 was also performed in GRZ schools during 2005. LTM is being used in government schools to supplement other methodologies, such as NBTL, SITE, ROC and MARK. In 2005, LTM was piloted at Grade 1 in 36 government schools. Learners in the 36 pilot schools and 14 control schools were tested at the beginning and the end of the year. The pilot results showed: x Learners using LTM had greater gains in English and numeracy x Girls and boys using LTM had equal learning gains x Enrolment increased in LTM classes x LTM motivated learners and promoted good time and class management Given the effectiveness of LTM and the need to improve access to quality education, in 2006 the Ministry of Education approved of the roll-out of LTM to community and GRZ schools throughout Zambia. The roll-out is currently being carried out in 2007. As shown in Table 2, the majority of learners using IRI are currently enrolled in IRI centres, since this was the original target for the programme. Table 2: Number of learners by type of school, 2006 Type of School Number of Learners IRI Centres 38,258 Community Schools 43,066 GRAND TOTAL 81.324 With the roll-out of IRI to GRZ schools in 2007, the number of GRZ learners using IRI promises to grow dramatically. This is the first evaluation of LTM at Grade 2. It asks the same questions as an earlier evaluation of Grade 1: whether there is demand for LTM, who the learners are, whether they attend radio lessons and whether they are learning. A sample of learners was tested at the end of the year in GRZ schools, community schools and IRI centres. The performance of learners who were using IRI was compared with the performance of control groups, consisting of learners from GRZ schools and community schools that were not using IRI. Teachers were interviewed to learn about environmental factors other than IRI that may have had an impact on learning. This report describes the results of the interviews and the testing. 1.2 Purpose of the Evaluation The overall goal of this evaluation is to document the effectiveness of IRI at Grade 2 in GRZ schools, IRI centres and community schools with a view of making adjustments to the programme and/or its implementation. The evaluation questions are as follows: 1. What is the level of demand for IRI in Zambia? 2. What are the characteristics of the children who participate in IRI? 3. How frequently do learners attend daily broadcasts? 4. Are learners achieving functional literacy skills in a Zambian language? 5. Are learners achieving English language, and mathematics skills as expected at Grade 2 level? Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 3 2.0 EVALUATION METHODOLOGY The design of the evaluation was guided by a Steering Committee for the Evaluation of IRI composed of members of the Examinations Council of Zambia (ECZ), the Directorate of Open and Distance Education (DODE), Teacher Education and Specialized Services (TESS), Curriculum and Standards and the QUESTT Project. The Steering Committee produced an evaluation plan that described the purpose of the evaluation, and information that should be collected, along with procedures for collecting data, analyzing data and producing the report. The Steering Committee also reviewed and revised the sampling design, created a schedule of evaluation activities and allocated resources to ensure that activities would be completed as scheduled. This section describes the sampling design and the procedures used to develop assessment instruments. 2.1 Sampling Design The 2006 IRI evaluation used a comparison model that included GRZ schools, Community schools and IRI centres. Fifty percent of the GRZ control schools and non-IRI Community schools were added to the sample, while all 36 IRI pilot GRZ schools were included in the original number of sample schools in the evaluation. Taking the above into consideration, a three-level sampling procedure was used to select the Grade 2 learners to be tested in the 2006 IRI Evaluation. At the first stage, five of nine provinces were selected, two urban provinces (Lusaka and Copperbelt), and two rural provinces (Eastern and Western) and one province along the line of rail (Southern province). Out of 625 IRI schools in the five provinces, 39 schools were sampled. An additional 26 non-IRI Community schools were included in the sample. The number of IRI schools in each province was determined proportionally, with efforts being made to test at IRI schools that were part of the 2005 sample. The table below indicates IRI schools and control non IRI Schools in 2006 sample. Table 3: Total number of schools in the sample, by province and school type Province Total number of schools Sampled Schools Control schools IRI GRZ IRI GRZ Non-IRI Community schools GRZ Central 160 10 10 5 8 6 Copperbelt 57 10 4 2 9 4 Eastern 200 0 10 5 0 0 Lusaka 125 11 10 5 4 3 North Western 83 5 5 3 5 2 Total 625 36 39 26 26 15 Only 26 of the 36 GRZ schools which were included in the IRI pilot were tested. Ten (10) GRZ pilot schools were not tested mostly because they did not conduct IRI classes regularly. At the third stage of sampling, IRI learners were sampled proportionally from each province using a simple random sampling of 5.2 percent of the total IRI learners (11,534) in Grade 2 registered in 2006. For each class tested, an equal number of boys and girls were selected. However the number of boys and girls were selected randomly from the register using an appropriate interval until the required number of boys and girls had been selected. The Test administrators were instructed to administer the test at IRI schools that had good radio reception throughout the year and to select learners who had high or medium attendance. A total of 150 radio lessons are broadcast during the year, Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 4 hence attendance was rated high if, of a possible 150, learners attended 120 or more radio lessons, medium if learners attended between 90 to 119 lessons, or low for less than 90 lessons. Table 4 below shows the number of the tested learners, by province and sex. Table 4: Total number of learners in the sample, by province and sex Province Sex of learner Male Female Total Central 209 220 429 Copperbelt 139 152 291 Eastern 131 136 267 Lusaka 166 160 326 Western 35 30 65 North Western 46 46 92 Total 726 744 1470 2.2 Assessment of Learning Achievement The Steering Committee was supported by a Technical Committee, which included an evaluation specialist from the University of Botswana and members of ECZ, DODE, Curriculum and Standards, and the QUESTT Project. The function of the Technical Committee was to produce the Grade 2 assessments and revise them with the assistance of pilot test data. This section describes the procedures used to develop the assessments. 2.2.1 Test Planning and Development Test planning took place at a Test Development Workshop from 11th to 13th October 2005. The Technical Committee reviewed the Zambian lower primary curriculum and determined the content for testing for Zambian Language Literacy and Life Skills, English Language and Mathematics for Grade 2. A distinction was made between terminal objectives that should have been mastered by the Grade 2 level and developmental objectives that would be mastered at later grades. Once terminal objectives had been selected for assessment, a test plan was developed with tasks designed to assess each objective. The Committee then divided into small groups to write items for each section of the assessment. These items were compiled and then reviewed at a one-day workshop on 19th October 2005. 2.2.2 Piloting and Reviewing the Tests The general purpose of piloting and reviewing the tests was to determine if the items measured the intended skills and if they were at the expected levels of difficulty. More specifically, the objectives of the pilot were to ensure that: 1. The tests could be administered to the desired number of pupils in a day 2. The test items yielded the intended information 3. The test items were at the right level of difficulty 4. The items discriminated well among high, medium and low level learners Draft versions of the Grade 2 tests were piloted in February 2006 to collect information about the tests. Grade 2 tests were piloted among pupils during the first few weeks of Grade 3, a few weeks after completing the Grade 2 curriculum. Twelve people formed two teams of test administrators. The test administrators came from DODE, CDC, QUESTT and UNZA. Data from the pilot testing was compiled in SPSS and used to calculate two statistics: percentage correct scores and correlation coefficients. The percentage correct was calculated by dividing the Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 5 total points that learners earned by the total possible points. An item with 90 percent correct would be a very easy item because the learners would have earned 90 percent of the total possible points for that item. On the other hand, an item with 10 percent correct would be very difficult because learners would have earned only 10 percent of the possible points for that item. The percentage correct was used to determine whether an item was as easy or as difficult as expected. Members of the Technical Committee rated each test item as Easy, Medium or Hard in terms of the objective that was being tested. Items were examined carefully for revision when administrators encountered problems administering the item during the pilot or when an item was supposed to test a relatively easy or hard objective but the percentage correct did not correspond with its rating. The correlation coefficients for each item were determined by correlating learners’ scores for individual item with their scores on the entire test section. This correlation helped committee members see how well the learners’ scores on an individual item corresponded with their scores on that section of the test. When learners who performed well on an item also performed well on the section and when those who performed poorly on the item also performed poorly on the section, then the item had a high correlation. Items with a correlation below 0.40 were looked at carefully to see if they were measuring what they should be measuring. Items were revised when answering the item correctly required skills other than those being tested. The pilot test succeeded in achieving each of the four objectives. 1. It was determined that the Grade 2 test could be administered more than the required numbers of pupils per day. 2. Pilot data and experiences from the administration were used to revise the items so that they would yield the intended information. 3. Items that were too easy or difficult were revised or removed from the test. 4. Each section of the test was determined to have a good balance of easy, medium and difficult items. Once the Technical Committee had revised the Grade 2 tests in conjunction with the pilot data, the tests were reproduced for administration. Table 5 below shows the skills covered in each subtest and the points allocated, while the final version of the Grade 2 assessment is presented in Appendix A. Table 5: Subtest skill areas and point values Mathematics [13 points] Zambian Language Literacy [36 points] x Addition x Subtraction x Multiplication x Division x Money x Length x Size x Time 4 3 2 1 1 1 1 1 x Writing Words x Writing a sentence from words x Writing a sentence from dictation x Free writing x Reading aloud x Reading comprehension 10 3 3 3 12 5 Life Skills [8 points] English Language [29 points] x Problem solving x Gender roles x Learner’s roles 2 3 3 x Naming jobs x Describing activities x Reading words 5 6 4 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 6 x Reading sentences x Writing words 4 10 2.2.3 Live Test Administration Training workshop for test administrators Educators from each of the five provinces acted as test administrators, including POCs, DRCCs and Senior Education Officers from DODE, Standards and TESS as well as a lecturer from the University of Zambia. Test administrators reviewed guidelines for sampling learners, administering the tests and interviewing teachers. After reading each section of the test, administrators were given time to practise administering the test to each other. All translations for the Zambian language literacy section were provided in printed form to ensure that the items were presented to learners in large, neat print. Translations were provided in Cinyanja, Icibemba, Kiikaonde, Silozi and Tonga to cater for learners in the tested areas. Five teams of four members each were involved in the testing at IRI Centres and Community Schools, while four teams administered tests at GRZ schools in October/November, 2006. Each team had a team leader who was in charge of ensuring quality control, compiling results and submitting reports. Reports contained information about where testing was done and any problems with test administration. Each team submitted all test results and teacher questionnaires. Test administration proceeded smoothly with no problems that would invalidate the results of any centre. The main problem that test administrators encountered was the inability to test at certain centres because the schools had closed for the end of the year. Team leaders also provided lists of the strengths and weaknesses of each centre. To provide feedback from evaluation information, each centre will be given a description of its strengths and weaknesses along with the executive summary of the evaluation report. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 7 3.0 FINDINGS AND DISCUSSION 3.1 Demand for IRI Over the past seven years the Learning at Taonga Market radio program has enabled many out-of￾school children access basic education in all the nine provinces of Zambia. From an initial 21 IRI centres with 1,254 Grade 1 learners in 2000, in 2006 IRI programmes were being broadcast in 1009 IRI centers and community schools, to 80,556 learners enrolled in Grades 1 to 6. The demand for IRI can be traced first from the pattern of establishing Interactive Radio Learning Centers (IRI centres) as shown in Table 6. Table 6: Number of IRI Centres and Community School, and total enrollment Provinces Total IRI centres 2003 Total IRI centres 2004 Total New IRI centres 2005 Total New IRI centres 2006 Total IRI centres submitting data 2006 Number of learners in IRI Centres in 2006 Central 96 135 23 49 160 13,243 Copperbelt 63 85 41 11 60 3,884 Eastern 61 94 97 74 206 17,188 Luapula 54 71 14 6 55 2,931 Lusaka 46 86 90 40 136 14,446 Northern 82 77 44 26 104 8,602 North Western 53 27 7 36 81 7,176 Southern 26 51 54 14 120 8,371 Western 35 44 14 24 87 4,915 Total 516 670 384 280 1009 80,556 In addition to the number of new centers being opened in each year, the demand for IRI has been manifested in the number of community schools adopting the use of IRI as a teaching method in their school, and the number of centers that have been able to stay open over the years. For instance, 280 more schools started using IRI in all provinces compared to 384 new schools that adopted IRI in 2005. Eastern, Central and Lusaka province had higher number of new schools that used IRI 2006. Table 7: Number of IRI learners from 2000 to 2006, by sex 2000 Grade 1 2001 Grade 1 and 2 2002 Grade 1 to 4 2003 Grade 1 to 5 2004 Grade 1 to 5 2005 Grade 1 to 5 2006 Grade 1 and 6 Male 3994 7104 11561 19412 27819 39759 Female 3788 6989 11202 19101 28414 40797 Total enrolment 1254 7782 14093 22763 38513 56233 80556 Absolute growth 6528 6311 8670 15750 17720 24323 Percent growth pilot baseline 81.1 61.5 69.2 46.1 43.3 The demand for IRI continues in 2007 with more GRZ and community schools adopting the use of IRI after IRI was shown to be successful in the 36 GRZ pilot schools. The results showed that GRZ schools using IRI increased its enrolment. The QUESTT project has continued to operate with the philosophy to meet the educational needs of the vulnerable groups such as the out-of-school Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 8 youths and orphans. Also, fewer centres were reported closed in 2006 (23 centres) as compared to 57 that closed in 2005. The main reasons reported for centres closing is lack of support of centres from the communities. However, 57 IRI centres re-opened in 2006 in North-Western province, Southern province, Luapula and Western province. 3.2 Profile of IRI Learners 3.2.1 Sex and age of learners Girls are systematically excluded from learning opportunities in most developing countries, hence the need to disaggregate enrolment by gender in order to determine whether or not there is equal participation of boys and girls. The table shows that 50.6 percent of the learners in IRI centers and community schools were girls, while 49.4 percent were boys in 2006. Table 8: Number of learners in enrolled in IRI centres and Community schools, by grade and sex Grade Male percent Female percent Total Grade 1 16994 48.9 17735 51.1 34729 Grade 2 8872 46.6 10149 53.4 19021 Grade 3 6202 51.3 5893 48.7 12095 Grade 4 3755 51.0 3605 49.0 7360 Grade 5 2430 52.8 2169 47.2 4599 Grade 6 1506 54.7 1246 45.3 2752 Total 39759 49.4 40797 50.6 80556 Since the inception of IRI, the overall enrolment ratio between boys and girls has been within one percent point, an indication that IRI provides equal opportunity and access to education to girls and boys. A similar ratio of male and female learners is maintained within each Grade level, which suggests also, that attrition rate for boys and girls was about equal. Zambia IRI, and indeed IRI in general, has a good track record of providing equal access and participation opportunities, unlike in the formal school system where dropout rates for girls tend to be higher than those for boys. This result is in direct contrast with data from GRZ schools, where there were consistently more boys participating overall, and fewer girls at the higher primary grades. The table below shows that the percentage of girls in Zambian government schools declines steadily with increasing grade levels. Table 9: Number of learners in enrolled in GRZ schools, by grade and sex Grade Male percent Female percent Total Grade 1 147010 49.5 149966 50.5 296976 Grade 2 151614 49.9 152428 50.1 304042 Grade 3 147623 50.1 147121 49.9 294744 Grade 4 147608 50.7 144114 49.4 291722 Grade 5 143972 51.3 136471 48.7 280443 Grade 6 134290 51.8 124850 48.2 259140 Grade 7 129954 54.5 108492 45.5 238446 The proportions of girls to boys are similar in GRZ schools and IRI schools from Grade 1 to 4. The percentage of girls in GRZ schools continues to decline at Grades 5 to Grade 6 and a similar situation is observed in IRI centres. IRI learners are also typically older children who either missed an opportunity to attend school, or dropped out. The average age for the learners in the Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 9 sample was 10 years 3 months for learners in IRI centres and community schools, and 9 years 3 months for learners in GRZ schools. Table 10 below presents age by grade level in the IRI population. It shows that the mean age for Grade 1 learners was 8 yrs old, with 23 percent of the first graders being at the recommended age of 7 years old, while the majority of IRI learners (61 percent) were above the age recommended for their grade level. Table 10: Recommended age enrolment by grade in 2006 Below recommended age At recommended age Above recommended age Grade Level and age Total N percent N percent N percent G1 (7 yrs) 35 633 6770 19 11759 33 16391 46 G2 (8 yrs) 18 184 2000 11 4000 22 11638 64 G3 (9 yrs) 12 036 843 7 1685 14 9147 76 G4 (10 yrs) 7 760 621 8 1086 14 5665 73 G5 (10 yrs) 4 834 387 8 532 11 3771 78 G6 (11 yrs) 2 870 201 7 287 10 2239 78 Total 81 317 10821 19 19350 33 48850 46 (From DODE and QUESTT, 2005:32) The high proportion of above-age learners reflects the difficulty that children have experienced in gaining access to education; hence the IRI program provides a second chance for such learners. Overall, 13 percent of the learners were younger than the recommended age. The proportion of Grade 1 learners that were 6 years or younger in 2005 was considerably higher at 16.4 percent, compared to 10 percent in 2003. Being a system that makes special considerations for disadvantaged learners, some of whom are responsible for the upkeep of their younger siblings, Learning at Taonga Market has allowed a higher degree of flexibility to learners. Rather than miss the opportunity to attend, learners are generally allowed to bring their younger brothers and sisters along to the IRI centres. It could also be that parents are using IRI centres to fill a need for early childhood care facilities (crèches, pre-school or pre-primary). While having under-age learners cannot be discouraged without taking away the flexibility for the learners who tend for their younger siblings as they attend, the fact that the content of IRI programmes is not well suited for their level was mentioned in previous evaluations. In order to derive any benefit from the IRI programme, under-age learners do require specialized teaching approaches that are not part of the preparation and training of mentors, hence the IRI system should cannot afford to be burdened with younger learners. As will be reported in the subsequent sections, the performance of under-age learners tends to be lower than that of age-appropriate and older learners, hence overloading the system with under-age learners may render IRI less effective as a learning tool. It may be possible at this stage in IRI programming to develop a programme for the pre-school or pre-primary level to be used in at least two contexts. First, as the opportunity presents itself, community schools (and GRZ schools) will be able to use the programmes for the pre-primary grade. Second, parents who work from home will be able to listen in with their younger children, a highly desirable practice in terms of preparing children for school and cultivating a culture of being part of children’s learning activities. 3.2.2 Orphan Status Since 2003 the proportion of orphans in IRI schools has been higher when compared to orphans in GRZ basic schools. Overall, 35 percent percent of the IRI learners were orphans in 2006, compared to 20 percent in GRZ basic schools as indicated in the table below. In 2006, 23 percent of the IRI Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 10 learners were single orphans, while 12 percent were double orphans. Differences between provinces in the number of orphans varied between 29 percent in Eastern and Southern Provinces to as high as 53 per cent in Western Province. However, with the exception of Eastern, Northern, and Western provinces, the trend in 2006 was that of decrease in proportion of learners who are orphans. Table 11: Percentage of orphans in IRI centres and GRZ schools, by province and year Provinces IRI Centres GRZ Schools 2005 2006 Change 2005 2006 Change Central 36 34 -2 20 20 0 Copperbelt 45 34 -11 20 18 -2 Eastern 24 29 5 20 20 0 Luapula 47 40 -7 19 19 0 Lusaka 46 44 -2 22 22 0 Northern 24 26 2 16 17 1 North Western 46 40 -6 17 18 1 Southern 30 29 -1 20 21 1 Western 44 53 9 23 24 1 All Provinces 36 35 -1 20 20 0 As expected, the proportion of orphans in IRI centres where there are more vulnerable children is higher than in GRZ schools. A comparison between provinces shows more interesting trends. First, there was no change in 4 of the 9 provinces, an expected result when looking at change only a year later. Second, the proportion of orphans in the Copperbelt has decreased, both for IRI centers and GRZ schools. This is a welcome development which will be investigated further. Third, there was a 1-point increase in the other 4 of 9 provinces, including Western and Northern provinces which registered an increase of orphans in the IRI centre learners. With the high prevalence rate of HIV/AIDS in Zambia it is assumed that many of the orphans lost their parents to HIV/AIDS and related illnesses. A response to this problem would be for Learning at Taonga Market to strengthen curricular activities that address issues relating to orphanhood. It may also be desirable to conduct a case study for Western province to investigate the observed increase in the number of orphans and how the effects may be mitigated. 3.3 Attendance For the purposes of this evaluation, high and medium attendance are considered acceptable, while low attendance is considered to be unacceptable. The next table shows the number of learners who had either High or Medium attendance—that is, attendance of 60 percent or more. The results indicate that 45.2 percent of the learners had high attendance and 31.6 percent had medium attendance, while 14.1 percent had low attendance. Table 12: Percent attendance for Grade 2 learners, by province Province Attendance Rating High (80 and above) Medium (60 to 79) Low (59 or less) Not reporting Total Central 261 (60.8) 136 (31.7) 32 (07.5) 0 (00.0) 429 Copperbelt 108 (37.1) 106 (36.4) 54 (18.6) 23 (07.9) 291 Eastern 115 (43.1) 92 (34.5) 12 (04.5) 48 (18.0) 267 Lusaka 136 (41.7) 72 (22.1) 55 (16.9) 63 (19.3) 326 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 11 Western 17 (26.2) 34 (52.3) 14 (21.5) 0 (00.0) 65 North Western 27 (29.3) 24 (26.1) 40 (43.5) 1 (01.1) 92 All provinces 664 (45.2) 464 (31.6) 207 (14.1) 135 (9.2) 1470 Test administrators were instructed to select learners with low attendance only when there were not enough learners with medium or high attendance. The fact that learners with low attendance were tested indicates that there were not enough learners with high or medium attendance available at the sampled schools. Among the sampled learners, boys had slightly better attendance than girls. A total of 77.3 percent of the boys and 76.2 percent of the girls had had 60 percent attendance or more. Central Province had the best attendance with 92.5 percent of the learners having acceptable attendance, while North-Western Province had the worst attendance at 55.4 percent. With 76.2 percent of the girls and 77.3 percent of the boys having high or medium attendance, girls and boys tended to have similar levels of attendance. IRI has a reputation of making school enjoyable, with the result that more learners want to attend school. Grade 2 attendance figures tended to support that conclusion as shown in Table 13. Table 13: Percent attendance for Grade 2 learners, by type of school Province Attendance Rating High (80 and above) Medium (60 to 79) Low (59 or less) Not reporting IRI Centre 39.5 32.5 11.7 16.3 IRI Community school 75.7 19.4 4.9 0.0 Control Community school 44.7 27.5 15.6 12.3 GRZ schools using IRI 43.6 37.4 14.7 4.3 Control GRZ schools 48.1 29.5 21.9 0.5 Desired and acceptable attendance was in the high and medium category. The proportion of low attendees was highest in control schools. This means that all Grade 2 classes using IRI, both in the IRI centres, community schools and GRZ school had higher attendance than control schools. In terms of boosting attendance, IRI had the highest success in community schools. 3.4 Performance in Maths, Zambian Language, Life Skills & English This section examines Grade 2 learners’ performance in the areas of Mathematics, Zambian Language Literacy, Life Skills and English. The section is divided into three parts. The first part looks at the results for all learners. The latter sections examine the impact of environmental factors on learner performance, looking first at the GRZ results and then the results in IRI centres and Community Schools. The Grade 2 assessment consisted of four subtests. The complete assessment, comprised of 37 items and scoring rubrics, as presented in Appendix A. The Mathematics subtest was worth 13 points, Zambian Language Literacy was worth 36 points, Life Skills was worth 8 points and English Language was worth 29 points for a total of 86 points. Table 14 presents the mean scores for the overall test and for the subtest for all learners—those using IRI as well as those in control schools. Table 14: Overall mean scores, by province and subtests Province Maths Zambian Language Life Skills English Language Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 12 [13 points] [36 points] [8 points] [29 points] Central Mean 7.6 7.1 3.6 11.1 Percent 58.6 19.7 45.0 38.3 Copperbelt Mean 6.6 3.6 2.5 7.8 Percent 50.8 10.0 31.3 26.9 Eastern Mean 6.8 9.0 4.8 6.6 Percent 52.3 25.0 60.0 22.8 Lusaka Mean 5.6 4.3 5.1 7.1 Percent 43.1 11.9 63.8 24.5 Western Mean 8.5 10.1 6.2 6.7 Percent 65.4 28.1 77.5 23.1 North Western Mean 7.3 7.9 3.8 13.0 Percent 56.2 21.9 47.5 44.8 Total Mean 6.8 6.3 4.0 8.6 Percent 52.3 17.5 50.0 29.7 Overall performance was at about 50 percent for math and lifeskills, and far below Zambian Language and English (17.5 percent and 29.7 percent, respectively). Comparison between provinces shows Western province as performing higher than other provinces in 3 of 4 learning areas, but not performing as well in the English Language Test. Table 15 presents the mean scores for each test section and for the overall test by the type of school. The results for control schools are highlighted. Table 15: Overall mean scores, by school type and subtest Type of school Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] IRI Mean 7.1 7.6 3.5 10.2 28.5 GRZ School Percent 54.6 21.1 43.8 35.2 33.1 Mean 5.6 8.0 3.6 8.1 25.4 Percent 43.1 22.2 45.0 27.9 29.5 Control GRZ School IRI Centre Mean 6.7 5.9 4.5 7.4 24.6 Percent 51.5 16.4 56.3 25.5 28.6 Mean 6.3 3.8 3.8 8.4 22.2 Percent 48.5 10.6 47.5 29.0 25.8 IRI Community School Mean 6.5 7.1 4.2 8.6 26.3 Percent 50.0 19.7 52.5 29.7 30.6 Control Community School Performance in GRZ schools followed the same pattern found in the Grade 1 evaluation of IRI in GRZ schools in 2005 (DODE & QUESTT, 2005, LTM in GRZ Schools). Learners who used IRI performed better than the control group in Maths and English, while IRI learners and the control group had similar levels of performance in Zambian Language Literacy. IRI and control learners also had similar performance in the area of Life Skills. Performance of IRI learners in IRI centres and in community schools differed from the trend found in GRZ schools. Learners in IRI centres performed better than control learners in community schools in the areas of Maths and Life Skills; however, the IRI learners did not perform as well as the control learners in Zambian Language Literacy or English. In general, IRI learners in community schools did not perform as well as learners at IRI centres. IRI learners at community schools obtained similar scores to their counterparts in the control group in Maths and English, Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 13 but IRI learners performed below the level of the control group in Zambian Language Literacy and Life Skills. 3.5 Performance in GRZ Schools This section looks at the environmental factors that may have had an impact on the performance of learners at GRZ schools. First, the impact of several learner characteristics (gender, age and attendance) will be examined. Second, we investigate how learners’ performance was affected by the teacher’s years of experience, time spent on pre-broadcast activities and assistance with school work at home. Finally, we examine whether learners who were not being taught Zambian Language Literacy in their language of play performed as well as learners whose language of play was the same as the language of instruction. The Grade 2 results show a difference in performance between girls and boys using IRI with boys performing better in all subject areas. On the other hand, in the control schools, girls in performed slightly better in Zambian Language and English Language, while boys performed slightly better in maths and Life Skills. The differences in performance in control schools cancelled each other out, yielding a similar overall test scores for girls and boys Table 16: Mean scores for IRI learners in GRZ schools, by sex and subtest Sex of learner Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] Male Mean score 7.5 8.4 4.1 10.6 30.6 Percent 57.7 23.3 51.3 36.6 35.6 Mean score 6.8 6.9 3.0 9.9 26.6 GRZ learners using IRI Female Percent 52.3 19.2 37.5 34.1 30.9 Male Mean score 6.0 7.9 3.8 7.9 25.6 Percent 46.2 21.9 47.5 27.2 29.8 Mean score 5.2 8.2 3.5 8.4 25.2 Control group of GRZ learners Female Percent 40.0 22.8 43.8 29.0 29.3 These results contrast with those obtained in the evaluation of IRI at Grade 1 in GRZ schools where girls and boys using IRI had similar learning gains while boys in the control schools tended to make greater gains than girls (DODE & QUESTT, 2005, LTM in GRZ Schools). In earlier IRI evaluations in IRI centres, boys and girls have always performed at par. A comparison of the results between GRZ schools using IRI and the control group further suggests that the IRI is systematically favouring boys. This result needs to be investigated further to find out why IRI is not yielding similar results in GRZ schools. Table 17 shows a further breakdown of the results for the 374 girls and boys who used IRI according to their age. Table 17: Mean scores for IRI learners in GRZ schools, age category Age Category Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] 7 years Mean score 5.0 1.5 2.5 5.5 14.5 and below Percent 38.5 4.2 31.3 19.0 16.9 8 to 9 Mean score 6.5 6.2 3.2 9.6 25.4 years Percent 50.0 17.2 40.0 33.1 29.5 10 to 11 Mean score 7.9 10.1 3.7 11.1 32.8 years Percent 60.8 28.1 46.3 38.3 38.1 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 14 12 years Mean score 9.2 10.4 5.1 12.4 37.1 and above Percent 70.8 28.9 68.3 42.8 43.1 Table 17 shows that age had an impact on the performance of the learners. The performance of learners increased with their age with learners who were below the recommended age for Grade 2 (Below 8 years) performing the lowest (16.9 percent) and those who were oldest (12 years and above) performing the best (43.1 percent). In all age categories, boys performed better than girls, with the difference being widest for the learners in the 7 years and below category. Another factor that had an impact on the results was learner attendance. As shown in Table 17, good attendance had a positive impact on test performance. Scores in all subject areas decreased with declines in attendance. That is, learners with High attendance tended to perform better than those with Medium attendance, who in turn performed better than those with Low attendance with scores of 39.9 percent, 30.2 percent and 24.1 percent respectively. Those learners with better attendance benefited more from the programme. This finding highlights the importance of promoting good attendance. The gender, age and attendance of learners had an impact on their performance. In addition, several teacher characteristics had an impact on learning achievement. Table 18: Mean scores for IRI learners in GRZ schools, by attendance Attendance rate Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] High Mean 7.9 10.8 3.8 11.8 34.3 (80 and above) Percent 60.8 30.0 47.5 40.7 39.9 Medium Mean 6.8 6.1 3.3 9.8 26.0 (60 to 79) Percent 50.3 16.9 41.3 33.8 30.2 Low Mean 5.9 2.8 3.2 8.7 20.7 (59 or less) Percent 45.4 7.8 40.0 30.0 24.1 One additional factor that had an impact on results in the Zambian Language Literacy subtest was the language used for instruction. According to NBTL principles, children are supposed to be taught literacy skills in the same language that they speak when playing. The idea is that children should learn to read and write a language that they can already speak and understand; they should not be subjected to the more difficult task of trying to understand the meaning of unfamiliar vocabulary at the same time that they are learning to read and write. There are seven languages of education used in Zambia. Teachers choose the language of instruction by selecting the one that is predominantly used as the language of play in the area or the language that is most similar to the language of play. Individual learners may not use the language of instruction when they play because they are part of a migrant or minority community or because their language of play does not happen to be one of the languages of instruction. Table 19 shows the scores on the Zambian Language Literacy subtest according to whether or not the language of play and instruction are the same. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 15 Table 19: Mean Scores for the Zambian Language Literacy by language Language of Play & Instruction are Same Language of Play & Instruction are Different N 219 155 Mean 8.3 6.6 IRI GRZ School Percent 23.1 18.3 N 118 76 Mean 7.8 7.5 Control GRZ School Percent 21.6 20.8 Of the 584 learners from GRZ schools who were sampled, 231 (39.6 percent) were not receiving Zambian Language instruction in the language of play. The percentage of learners who were not receiving instruction in the language of play was greater at IRI GRZ Schools (41.4 percent) than at the Control GRZ Schools (36.2 percent). The results for learners in IRI GRZ Schools who were not receiving instruction in the language of play (18.3 percent) were lower than for those who did receive instruction in the language of play (23.1 percent) with a 4.8 percent difference in performance. Learners in Control GRZ Schools who did not receive instruction in the language of play (20.8 percent) performed about the same as learners who received instruction in the language of play. The results of the IRI learners indicate that IRI instruction has a greater impact on the achievement of literacy skills when the language of instruction is the same as the language of play. 3.6 Performance in IRLCs and Community Schools The 2005 evaluation of IRI at Grade 1 revealed that girls and boys obtained similar levels of learning achievement in the areas of English Language and numeracy (DODE & QUESTT, 2005, LTM in IRLCs). Zambian Language Literacy was tested for the first time in IRI centres and community schools in the 2005 evaluation of IRI, where it was found that boys made significantly greater learning gains than girls. A possible explanation given for this difference in learning was that boys had better attendance than girls with 78.9 percent of the boys attending 60 percent or more of the lessons compared to 61.1 percent of the girls. Table 20 provides results for Grade 2 boys and girls who were studying at IRI centres, IRI community schools and control community schools. Table 20: Mean scores for IRI learners in IRLCs and Community schools, by sex and subtest Sex of learner Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] Male Mean score Mean 7.1 6.8 4.4 8.3 Percent Percent 54.6 18.9 55.0 28.6 Mean score Mean 6.3 4.2 4.3 6.9 IRLC and Community school learners using IRI Female Percent Percent 48.5 11.7 53.8 23.8 Male Mean score Mean 6.8 8.8 4.6 9.5 Percent Percent 52.3 24.4 57.5 32.8 Mean score Mean 6.1 5.8 4.0 7.5 Control group of community school learners Female Percent Percent 46.9 16.1 50.0 25.9 In both the control schools and the IRI schools, boys performed better than girls in all subjects. The only area of similar performance was in Life Skills where the girls in IRI schools scored an average of 4.3 points (53.8 percent) and boys scored 4.4 points (55.0 percent). The gender gap is greater in Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 16 the control schools, where there is a 7.3 percent difference in the Overall Test score, as compared to the IRI schools, where there is a 6.3 percent difference. Two factors that might explain differential performance of boys and girls have already been isolated; boys attend more regularly than girls; boys at IRI centres are older than girls by one year, and therefore more mature. In the context of IRI this result suggests that the Zambia program and its implementation should be examined for other factors that disadvantage girls, and those should be dealt with before they become institutionalised. Table 21 provides a further breakdown of results according to age groups. Table 21: Mean scores for IRI learners in IRLCs and Community schools, age category Age Category Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] 7 years Mean score 4.5 2.0 3.8 5.3 15.5 and below Percent 34.6 5.6 47.5 18.3 18.0 8 to 9 Mean score 6.3 4.4 4.0 7.4 22.1 years Percent 48.5 12.2 50.0 25.5 25.7 10 to 11 Mean score 6.6 6.1 4.3 7.7 24.7 years Percent 50.8 16.9 53.8 26.6 28.7 12 years Mean score 7.8 9.5 4.9 9.4 31.7 and above Percent 60.0 26.4 61.3 32.4 36.9 Without exception, the Grade 2 mean scores for each test section as well as the Overall Test score became higher with increases with age. This is consistent with findings of past evaluations at Grade 1 (DODE & QUESTT, 2005, LTM in GRZ Schools; DODE & QUESTT, 2005, LTM in IRLCs). Differences in performance between girls and boys exist for all age groups with boys consistently performing better than girls. The difference in performance is lowest for the Grade 2 target group, which is eight to nine years of age (2.9 percent). The difference in performance became greater for children below eight years (4.1 percent) and children who were 10 to 11 years old (4.8 percent). The greatest gap in performance (11.6 percent) occurred among the children who were 12 years of age and older. While girls had slightly lower attendance compared to boys, this small difference in attendance would not warrant the observed differences in performance. Other reasons need to be explored for the difference in performance. Table 22 presents the mean scores for all IRI and community school learners according to their attendance. Table 22: Mean scores for IRI learners in IRLCs and Community schools, by attendance Attendance rate Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] High Mean 6.7 6.3 4.3 8.5 25.9 (80 and above) Percent 51.5 17.5 53.8 29.3 30.1 Medium Mean 7.1 7.0 4.5 8.0 26.6 (60 to 79) Percent 54.6 19.4 56.3 27.5 30.9 Low Mean 6.9 4.9 4.2 8.0 23.9 (59 or less) Percent 53.1 13.6 52.5 27.5 27.8 Higher rates of attendance led to higher performance. While there was little difference in performance between the learners with Medium and High attendance, those learners with Medium and High attendance tended to perform better than those with Low attendance. Of the Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 17 886 learners who were sampled, 400 (45.1 percent) had High attendance and 262 (29.6 percent) had Medium attendance, giving a combined total of 662 (74.7 percent) learners who attended 60 percent or more of the lessons. This indicates that there was generally good attendance at IRI centres and community schools. The next table provides the results of learners according to the type of school and the location of the school. Table 23: Mean Scores for Learners by type of school and location of school Type Of school Location of school Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] Mean Score 7.7 5.2 4.7 8.6 26.2 Urban Percent 59.2 14.4 58.8 29.7 30.5 Mean Score 6.1 5.8 4.2 7.1 23.2 IRI Centres and IRI Community Schools Rural Percent 46.9 16.1 52.5 24.5 27.0 Mean Score 7.3 8.4 4.9 12.0 32.7 Urban Percent 56.2 23.3 61.3 41.4 38.0 Mean Score 5.9 6.0 3.6 6.1 21.6 Control Community Schools Rural Percent 45.4 16.7 45.0 21.0 25.1 Mean Score 7.6 6.2 4.8 9.7 28.2 Urban Percent 58.5 72.2 60.0 33.4 32.8 Mean Score 6.1 5.9 4.1 6.8 22.8 Total Rural Percent 46.9 16.4 51.3 23.4 26.5 Learners in urban areas tended to perform better than learners in rural areas in both the IRI schools and the control schools. The disparity in performance was much greater in Control Community Schools than it was in IRI Schools: the difference in scores of urban and rural learners in Control Community Schools was 12.9 percentage points while the difference in IRI Schools was 3.5 percent. This shows that the use of IRI had the effect of reducing the impact of attending school in a rural area. Learners in rural areas often have obstacles to learning that are not faced by learners in urban areas. Certified teachers often prefer to work in urban areas where they have easier access to amenities, such as electricity and healthcare, creating staff shortages in rural schools. Schools in rural areas also tend to have fewer materials. The IRI program mitigates the effects of learning in a rural area by providing a mentor’s guide that offers teachers 150 prepared lesson plans for the year, making it easier for teachers to prepare their lessons. Secondly, the lessons provide children with curriculum-based activities that use locally available materials. However, even with the mentor’s guides, significant gains in performance are made when mentors and teachers receive training in IRI training in IRI methodologies as reflected in Table 24 below. Table 24: Mean Scores for the overall test by IRI training No IRI Training Had IRI Training N 265 274 Mean 24.3 24.9 IRLC Percent 28.2 28.9 N 82 21 Mean 19.5 33.1 IRI Community School Percent 22.7 38.5 Total N 347 295 Mean 21.9 29.0 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 18 Percent 25.5 33.7 Table 24 shows that learners who had a teacher with IRI training earned a overall test score of 33.7 percent, while those with teachers who had no IRI training had a mean score of 25.5 percent. Scores followed a similar trend for learners at both IRI centres and IRI Community Schools. IRI training seemed to benefit teachers in community schools more, some of whom are trained teachers. When looking at the value added by IRI training it is a concern that the proportion of teachers who have not received IRI trained remains as high as it is (54.0 percent). However, this picture may have changed with the roll-out of training in 2007. Another factor that had an impact on learner performance was the amount of experience the teacher had. Table 25 shows the learners’ mean test scores according to their teachers’ years of experience. Table 25: Mean scores on the overall test by school type and years of teaching experience Type of school Years of teaching experience N Mean Percent IRI Centre 2 years or less 213 23.2 26.9 3 to 5 Years 246 25.4 29.5 6 years or more 61 31.1 36.2 IRI Community School 2 years or less 79 21.3 24.7 3 to 5 Years 24 25.7 29.8 Control Community School 2 years or less 93 19.6 22.8 3 to 5 Years 77 23.0 26.7 6 years or more 74 38.2 44.5 IRI GRZ School 2 years or less 199 26.4 30.7 3 to 5 Years 103 33.1 38.5 6 years or more 72 27.5 32.0 Control GRZ School 2 years or less 106 23.8 27.7 3 to 5 Years 24 28.8 33.5 6 years or more 80 26.5 30.8 Data from IRI centres, IRI community schools and control community schools showed a consistent trend: the more experience teachers had the better their results in terms of learner performance. These results confirm the notion that experienced teachers are better able to help children learn and indicate that it is important to retain teachers so that their learners can benefit from that experience. GRZ schools showed a mixed result, both in IRI and non IRI classes. Learners who had teachers with 3 to 5 years teaching experience performed better than learners who had teachers with 2 years experience or less. However, after being on the job for more than 5 years, 6 years and over, teachers in GRZ schools tended to produce inferior results. A number of factors are responsible for this, one of the most common being that after five years or so at the same teaching job, teachers become demotivated. In such cases many of them spend time on activities such as furthering their own studies by enrolling in distance education courses, and often use class time for their own assignments. Another factor that was investigated was assistance with school work that caregivers provided to children at home and how that affects achievement of learning. Test administrators were instructed to ask each learner how often someone at home helped them with each subject. Learners were given the choices of responding “never”, “only when stuck”, “sometimes” or “often”. Table 26 shows the GRZ learners’ responses. The question was asked during English language testing, and the responses were reflected on the overall scores (all subtests). Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 19 Table 26: Mean scores on the overall test by school type and help on received at home Type of school Help in English language at home N Mean Percent IRI Centre Never 315 22.6 26.2 Only When Stuck 80 24.6 28.6 Sometimes 101 28.1 32.6 Often 38 33.6 39.1 IRI Community School Never 87 19.5 22.6 Only When Stuck 3 34.3 39.9 Sometimes 12 36.0 41.9 Often 1 66.0 76.7 Control Community School Never 157 22.9 26.7 Only When Stuck 17 32.5 37.8 Sometimes 58 33.5 39.0 Often 12 27.0 31.4 IRI GRZ School Never 175 25.3 29.4 Only When Stuck 48 28.5 33.2 Sometimes 66 38.1 44.3 Often 17 26.4 30.6 Control GRZ School Never 104 20.9 24.3 Only When Stuck 31 38.0 44.2 Sometimes 44 31.3 36.4 Often 14 30.6 35.5 Of the 1380 learners who responded to the question on assistance with homework, 60.7 percent reported that they never receive assistance, 13.0 percent receive assistance whenever they need it, 20.4 percent receive assistance only some of the time, while 6 percent receive it regularly. In all types of schools the majority of learners who never received assistance performed lowest, suggesting that learners were in need of such assistance. Other than that, the results show that assistance with homework makes a difference with performance. In IRI centres and community schools learners perform better than their counterparts who receive assistance only sometimes or when they need it. In the case of GRZ schools, learners who receive assistance often times do not perform as well. It is not clear what this results suggest – but the quality of such assistance could be an issue. 3.7 Teachers’ predictions of Zambian Language Performance One finding of the 2005 report on LTM in ILRCs and community schools was that Grade 1 teachers at those institutions were unable to rank their learners according to NBTL milestones. Teachers and mentors were given more assistance in assessing learner performance throughout the year, and were requested as part of this evaluation, to rate performance of their learners in Zambian Language Literacy. The expectation was that learners who were classified by their teachers as high performers would score better than those reading at the medium level, who in turn would perform better than those classified as low readers. Table 27 shows the average scores of the learners for each Zambian Language skill by the teachers’ ratings. Table 27: Predicted Reading Ability in Zambian Language Literacy by School Type and Ability Grouping Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 20 Type of School Teacher’s Rating N Writing Words [10points] Writing Sentences [9 points] Reading Aloud [12 points] Reading Comprehe [5 points] Zambian Language Literacy [36 points] High 64 2.0 5.3 7.2 3.6 20.8 Medium 101 1.2 2.3 2.9 2.4 8.3 IRI GRZ Schools Low 192 0.3 1.2 0.7 0.7 2.6 High 28 4.4 7.8 10.8 4.4 31.0 Medium 44 2.0 2.6 3.6 2.7 10.9 GRZ Control Schools Low 122 0.1 1.2 0.8 0.5 2.3 High 70 0.6 1.7 1.3 4.2 8.8 Medium 76 0.7 2.7 2.0 3.2 10.4 IRI Learning Centres Low 252 0.2 1.1 0.5 0.6 2.3 High 7 1.4 2.1 1.6 4.3 10.0 Medium 10 2.4 3.2 1.8 1.5 9.4 IRI Community Schools Low 86 0.2 1.3 0.4 0.3 2.1 High 48 0.7 2.3 2.5 4.4 11.7 Medium 31 1.5 4.7 5.1 3.0 17.0 Community Control Schools Low 120 0.3 1.7 0.7 0.2 2.7 GRZ teachers tended to make accurate ratings of learner performance in Zambian Language Literacy, both in case of learners in IRI classes and in non-IRI classes. The scores of learners rated as high, medium and low tend to decline as predicted. The results were mixed for teachers at community schools, with those in IRI classes making better predictions that those in control community schools. The worst groups in predicting learner performance were teachers in IRI centres and community control schools. One distinguishing factor between teachers that could have prepared them for the task of predicting learner performance is that GRZ teachers are trained teachers while teachers in community schools and IRI centres are untrained. All else being equal, trained teachers are more equipped with skills of assessing learner performance than untrained teachers. Another teaching characteristic that had an impact on learning achievement was the amount of time spent on pre-broadcast activities. Table 28 shows subtest and overall scores for learners according to the amount of time they usually spent on activities before the broadcast. Table 28: Amount of time learners spent on pre-broadcast activities Time Learners Spent on Activities Maths [13 points] Zambian Language [36 points] Life Skills [8 points] English Language [29 points] Overall Test [86 points] More than 30 minutes 7.2 8.3 3.1 10.4 28.9 11-30 minutes 7.4 7.3 3.9 10.6 29.2 5-10 minutes 5.2 4.5 3.4 4.7 17.8 Those learners who usually spent five to ten minutes on pre-broadcast activities had lower scores on all subtests than learners who usually spent 11 to 30 minutes and those who spent 30 minutes or more on pre-broadcast activities. Pre-broadcast activities as described in the mentor’s guide include warm-up activities that allow learners to review what they have done and become acquainted with the topics that they are about to cover. Monitoring at GRZ schools revealed that teachers often used time before the broadcast to do more extensive post-broadcast activities from the previous lesson because they were better able to fit those activities into their schedule before the next broadcast. In essence, when teachers reported that their classes usually spent 11 to 30 minutes or more than 30 minutes on pre-broadcast activities, the classes were likely spending some time on follow-up activities from the prior lesson as well as introductory activities for the next lesson. Spending more than ten minutes on those activities each day had a positive impact on learner achievement. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 21 4.0 CONCLUSION AND RECOMMENDATIONS This evaluation, like Grade 1 evaluations before it, set out to answer a number of questions about the effectiveness of IRI at Grade 2 in GRZ schools, IRI centres and community schools with a view of making adjustments to the programme and/or its implementation. Five evaluation questions addressed whether or not there is demand for IRI in GRZ schools and demand to set up IRI learning centres; the characteristics of out-of-school children who participate in IRI; whether learners attend daily broadcasts as frequently as expected; and whether learners are achieving functional literacy skills in a Zambian language, English language and mathematics skills as expected at Grade 2 level. The evaluation findings are summarised below. 4.1 Demand for IRI The demand for IRI continues to increase as evidenced by the number of IRI centres that are established and continue to function, and the growing numbers of schools using the methodology. The number of IRI centres and community schools increased from a total of 893 in 2005 to 1,045 in 2006, while the number of learners increased from 56,233 in 2005 to 80,556 in 2006. A relatively small amount of this growth is due to the expansion of IRI to Grade 6 in 2006. Another source of growth has been the number of GRZ schools that are using IRI. The public continues to turn to IRI to provide education for children who would not otherwise receive education. 4.2 Characteristics of the IRI Population Three features of the IRI population that have been observed in past evaluations were again observed in 2006: 1. The IRI population continues to be almost equally divided between boys and girls. 2. IRI learners tend to be older than the recommended age for their grade level. 3. There is a higher proportion of orphans in IRI centres than in GRZ schools. The overall 2006 ratio of 49.4 percent boys and 50.6 percent girls indicates that IRI continues to provide equal access to learning for girls and boys. However, 2006 figures indicate that the ratio of girls to boys follows a similar trend to what is seen in GRZ schools: as the grade level increases, lower percentages of girls are enrolled. This indicates that there is higher attrition among girls than boys. This trend should continue to be monitored and programmes for retaining girls at the middle basic level should be designed and implemented. Also, IRI learners tend to be older for the grade level than their school going counterparts. Within the group, boys tend to be older than girls by 1 year or so. Given the overall increase in the IRI population, orphans continued to represent about the same proportion of learners in 2006 (35 percent) as they did in 2005 (36 percent). As in 2005, the proportion of orphans in IRI centres (35 percent) was greater than the proportion of orphans in GRZ schools (20 percent) in 2006. In summary, IRI centres and community schools provide girls and boys with equal access to education and they provide orphans with greater access to education than the government schools. In addition, older learners are able to attend school despite their inability to do so in the past. These characteristics of the IRI population show that IRI continues to serve vulnerable children well. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 22 4.3 Attendance of IRI Learners There is generally good attendance at IRI centres and community schools. 76.6 percent of the learners had high and medium attendance while 14.1 percent had low attendance. Girls and boys exhibit have similar levels of attendance. As anticipated, learners in IRI classes were attending considerably more than learners in control classes, lending support to the conclusion that children enjoy the IRI learning environment more. 4.4 Achievement of Grade 2 GRZ Learners The Grade 2 GRZ learners performed much the same as the Grade 1 learners from the 2005 evaluation report. Learners using IRI performed better than learners in control schools in the areas of mathematics and English. IRI and control learners performed similarly in the areas of Zambian Language Literacy and Life Skills. Both IRI and Control GRZ Schools had similar levels of materials, so the difference in performance cannot be attributed to a lack of materials in control schools. Three learner characteristics had an impact on results: attendance, age and gender. In general, children with higher attendance rates performed better than those with lower attendance, older learners performed better than younger ones and boys performed better than girls. Although low attendance was found to have a negative impact on achievement, learners generally had good attendance. The attraction of listening to the radio and participating in learner-centred activities makes IRI lessons attractive to learners, promoting good attendance and achievement. The finding that girls using IRI did not perform as well as boys contrasts with results from earlier studies of Grade 1. The difference in performance between Grade 2 girls and boys was lowest for those who were at the recommended ages of eight to nine years old. This highlights the importance of having learners, especially girls, start school at an appropriate age. Learners who start school below the recommended age of seven should be allowed to repeat Grade 1 before being promoted to Grade 2. Teaching characteristics that had an impact on achievement include whether or not teachers received IRI training, time spent on learning activities before the broadcast, and the match between the language of play and the language of instruction. First, learners whose teachers received IRI training had better results. This highlights the need to reach all IRI teachers with the necessary training. Second, those learners who spent eleven minutes or more on learning activities before the broadcast performed better than those who spent only five to ten minutes on activities. While activities during the radio broadcasts promote learning, children need the opportunity to reinforce those skills by completing activities outside of the broadcast time. Teachers should be monitored regularly to ensure that they plan and include appropriate learning activities in their schedules. Third, learners who received assistance with schoolwork at home achieved better results than those who did not receive assistance. Moreover, in the subjects of mathematics and English language, IRI mitigated the impact of not receiving assistance at home. This was most evident in the area of mathematics, where IRI learners who received no assistance at home performed on par with control learners who received assistance sometimes and when they were stuck. It is likely that the learner-centred activities conducted during and outside of broadcast time help compensate for the lack of individual assistance at home, assistance that may be more difficult for orphaned children to find. Fourth, learners who received instruction in their language of play Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 23 obtained better results than those who did not receive instruction in the language of play. Teachers at GRZ schools should make every effort to provide literacy instruction in the language of play. Head teachers should ensure that the lower basic teachers who are responsible for teaching Zambian Language Literacy are proficient in the language and able to model correct reading and writing in the language of play. 4.5 Achievement of Grade 2 Learners at IRLCs and Community Schools The performance of IRI learners in IRI centres and in community schools differed from the trend found in GRZ schools. Learners in IRI centres performed better than control learners in community schools in the areas of Maths and Life Skills; however, the IRI learners did not perform as well as the control learners in Zambian Language Literacy or English. In general, IRI learners in community schools did not perform as well as learners in IRI centres. IRI learners at community schools obtained similar scores to their counterparts in the control group in Maths and English, but IRI learners performed below the level of the control group in Zambian Language Literacy and Life Skills. These results can be attributed to the lack of materials in the IRI centres and Community Schools, both of which were poorly equipped with chalk, pencils and exercise books. Community Schools had the added burden of being poorly equipped with mentor’s guides. Learners in urban schools performed better than learners in rural schools; however, IRI mitigated the effect of attending a rural school. The gap in performance between urban and rural learners was lower among IRI learners (3.5 percentage points) than among learners in control schools (12.9 percentage points). In addition, IRI learners in rural schools had a higher overall test score (27.0 percent) than control learners in rural schools (25.1 percent). IRI mitigates the effects of learning in a rural area by providing teachers with a mentor’s guide of prepared lessons and by providing children with curriculum-based activities that use locally available materials. Four learner characteristics had an impact on performance: gender, age, attendance and the number of weeks not taught. Boys performed better than girls, although the gender gap was lowest (2.9 percentage points) for children who were at the recommended age for Grade 2 (eight to nine years old). As with the GRZ results, this shows the importance of having girls and boys enter Grade 1 at the recommended age of seven. Older children performed better than younger children, with underage children performing far below those at the recommended age for Grade 2. The performance of underage children should be carefully assessed by the teacher to see if they are ready to be promoted to Grade 2. Higher rates of attendance led to better results. Learners who attended 60 percent of the lessons or more performed better than learners who attended less than 60 percent of the lessons. Of the 886 learners who were sampled, 662 (74.7 percent) attended 60 percent or more of the lessons, indicating that there was generally good attendance at IRI centres and community schools. A similar finding was that the number of weeks not taught had a negative impact on performance. Children who were not taught for four to five weeks had lower scores than those who were not taught for zero to three weeks. Monitoring has revealed that teachers at IRI centres and Community Schools have trouble getting financial support for the work they do. This leads to teacher absenteeism, which has a negative impact on learning. IRI centres and community schools need to find the means to provide adequate and consistent financial support for teachers so that they remain at the school and teach classes regularly. Teacher training on IRI methodology had a great impact on learner performance. While IRI was designed to require little teacher training, the training does provide the benefits of giving teachers Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 24 practise at following the instructions of the radio teacher and understanding the daily routine. The training also helps teachers become familiar with the mentor’s guide, which provides guidance in lesson planning, creating teaching aids and performing continuous assessment. Providing teachers with three to five days of training in IRI allows learners to gain the most benefit from the program. The second teaching characteristic that had an impact on learner performance was experience at teaching. Teachers with more experience had learners with better performance. This indicates that it is important to retain teachers. However, the data also indicated that IRI mitigated the effects of having a teacher with two years of experience or less. IRI centre teachers who had two years of experience or less yielded results that were similar to control learners who had teachers with three to five years of experience. This can be explained by the fact that the mentor’s guide provides 150 lessons for each grade level, making it easy for new teachers to prepare lessons. In addition, the radio teacher provides guidance to the classroom teacher, helping the teacher lead the class in learner-centred activities that cover the curriculum in the subjects of Mathematics, Zambian Language Literacy, English Language, Social Studies, Science, and Life Skills, as well as Spiritual and Moral Education. It would be useful to provide training in IRI to all new teachers during pre￾service education to improve the quality of instruction. Lastly, the test scores were analyzed in terms of the amount of help that learners received at home. Analyses of results for the Zambian Language, English Language and Mathematics subtests each revealed that those learners who received help at home with schoolwork outperformed those who did not receive help. The implication is that learners would benefit from additional assistance with schoolwork outside of the class. A practical solution would be to implement an after-school programme where learners who need help with their schoolwork have the opportunity to get assistance from other learners in their class or older learners from other classes. 4.6 Teacher predictions of Zambian Language Performance GRZ teachers tended to make accurate ratings of learner performance in Zambian Language Literacy. On the other hand, the teachers at IRI centres and Community Schools were able to distinguish a learner with Low performance from a learner with Medium or High performance, but they generally could not distinguish between a learner with Medium and a learner with High performance. These results are similar to the findings of the 2005 Grade 1 report (DODE & QUESTT, 2005, LTM in IRLCs) in which teachers in IRI centres and community schools were unable to accurately rate the performance of learners in Zambian Language Literacy. As with the Grade 1 IRI centre and community school teachers of the 2005 report, the Grade 2 IRI centre and community school teachers need further instruction in NBTL methodologies so that they can assess their pupils’ progress and provide appropriate support to help individual learners achieve literacy skills. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 25 4.7 Recommendations A number of suggestions for improvement were made in the discussion of findings. Some of these are recommended for further action. 1. Monitor the trend of attrition among girls at the middle basic level. Now that IRI has been rolled out to Grades 5 to 7, attrition rates for girls and boys should be monitored closely. If the attrition rates for girls continue to outpace the rate for boys, then programmes for retaining girls at the middle basic level should be designed and implemented. 2. Sensitize school managers to the need of providing instruction in Zambian Language Literacy in the language of play. Head teachers should ensure that the lower basic teachers who are responsible for teaching Zambian Language Literacy are proficient in the language and able to model correct reading and writing in the language of play. As with the Grade 1 IRI centres and community school teachers of the 2005 report, the Grade 2 teachers in IRI centres and community school need coaching in critical aspects of the NBTL methodology so that they can be able to better assess their pupils’ progress and provide appropriate support to help individual learners achieve literacy skills. Such critical aspects should be packaged as part of the IRI training. 3. Sensitize parents to the need of enrolling children in Grade 1 at the age of seven. As learners grow older, they become more self-conscious of the difference between their age and the age of their classmates, making it harder for them to participate freely in the classroom and benefit from their education. Apparently, this is more of a problem for girls than boys, as the learning gap between girls and boys increased with age. Children, especially girls, should be enrolled in Grade 1 at the age of seven to gain the most benefit from their education. 5. Assess underage learners before promoting them to Grade 2. Grade 2 children who were younger than eight had the lowest performance in this study. They probably would have benefited more from repeating Grade 1 than entering Grade 2. Teachers need to be trained to have children who cannot perform basic literacy skills at Grade 1 repeat that grade. 6. Provide pre-service education in IRI to all new teachers. Learners who had a teacher who was trained in IRI benefited more from the programme. The longest IRI workshops last five days. It would be useful to provide training in IRI to all new teachers during pre￾service education to improve the quality of instruction. 7. Design a community-based after-school programme where learners can get help with school work. A practical solution would be to implement an after-school programme where learners who need help with their schoolwork have the opportunity to get assistance from other learners in their class or older learners from other classes. 8. Provide mentors with training on NBTL methodologies by distance learning. Develop a training program on delivering NBTL through IRI. This could be played on air as part of the Radio Mentor Training programme. It could also be packaged in CDs and tapes. This would allow IRI mentors to study while they work and attend face-to-face sessions with lecturers during school breaks. The training program should emphasise the basic tenets of NBTL. It should also teach mentors how to lead reading activities, how to care for readers, Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 26 and how to use assessment strategies to gauge if children are achieving the literacy goals as set by the curriculum. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 27 REFERENCES DODE & QUESTT. 2005. 2005 IRI Enrolment Bulletin. Lusaka: Directorate of Open and Distance Education, QUESTT Project and USAID. [Cited as DODE & QUESTT, 2005, 2005 IRI Enrolment] DODE & QUESTT. 2005. Learning at Taonga Market in Government Schools: An Evaluation of the 2005 Pilot of Interactive Radio Instruction in Grade One. Lusaka: Directorate of Open and Distance Education, QUESTT Project and USAID. [Cited as DODE & QUESTT, 2005, LTM in GRZ Schools] DODE & QUESTT. 2005. Learning at Taonga Market: IRI Learning Centres and Community Schools: An Evaluation of Interactive Radio Instruction at Grade One in 2005. Lusaka: Directorate of Open and Distance Education, QUESTT Project and USAID. [Cited as DODE & QUESTT, 2005, LTM in IRLCs] Kariuki, S., Letshabo, K., Laflin, M., Mutale, K., Phiri, M. and Sakala, G. 2000. October 2000 Evaluation of the Interactive Radio Pilot for Out-of-School Audiences. Lusaka: USAID￾Zambia. Letshabo, K. 2003. Interactive Radio Instruction for Out-of-School Children and Youth in Zambia: An Evaluation Report for Grade One Learners, 2003. Lusaka: Educational Broadcasting Services, Ministry of Education. Letshabo, K. and Kariuki, S. 2001. Interactive Radio Instruction for Out-of-School Children and Youth in Zambia: An Evaluation Report for Grade One Learners,2001. Lusaka: Educational Broadcasting Services, Ministry of Education. Mackay, D., Thompson, B. and Schaub, P. 1970. Breakthrough To Literacy. London: Longman for the Schools Council. Republic of Zambia Ministry of Education. 2004. Educational Management Information System. Lusaka: Ministry of Education. Republic of Zambia Ministry of Education. 1996. Educating Our Future. Lusaka: Ministry of Education. Siaciwena, R., Thomas, M., Luangala, J. and Haambokoma, C. 2002. Report on the Rapid Appraisal of the Ministry of Education’s Interactive Radio Instruction Basic Education Delivery System. Lusaka: USAID-Zambia Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 28 APPENDICES Appendix A: Grade 2 Assessment Instruments, 2006 2006 GRADE 2 ASSESSMENT MATHEMATICS, LITERACY AND ENGLISH LANGUAGE SKILLS Achievement Tests IRI Centres & Community Schools 25th October to 10th November 2006 Regular Government Schools 13th to 24th November 2006 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 29 Instructions Before the test: • Read this information before the test and seek clarification where necessary. • Ask the class teacher to rate each learner’s Zambian language ability with the guidance of the “Descriptors for Language Mastery”. • Record the names of the pupils in the same order on each of the scoring grids. Record the Zambian language rating of Low, Medium or High on the Zambian Language Literacy scoring grid. • The sample should include 50 percent girls and 50 percent boys. During the test: • Everything that appears in italics in the test is for the information or direction of the test administrator. Please do not read such words to the learner. Words that are supposed to be read to the learners are NOT in italics. • Each test question may be presented to the learner a maximum of two times. • Administer the test in the following order: 1. Mathematics: Administer all questions as a group. Read each question to the group. Go to the next question when everyone has had time to answer. Translation into the local Zambian language is allowed on all mathematics questions. 2. Literacy: Administer Tasks 1 and 2 as a group. Then administer the other questions to one learner at a time, away from other learners. All questions must be given in the local Zambian language. 3. English Language: Administer all questions to one learner at a time, away from other learners. All questions must be given in English. Translation is not allowed. • To keep the testing time as short as possible, three administrators should deliver the test—one for each section. After the Mathematics and Literacy Tasks 1 and 2 are completed, have the learners go from the literacy administrator to the English language administrator. • Use the scoring grid to record scores. For Mathematics and Literacy Tasks 1 and 2, record the scores after collecting the learners’ writing. For questions given individually, record a score as the learner gives a response. Score ranges for each question are shown at the top of each column on the grid. Record NR for No Response. • Before starting the test, test administrators should write the names and information about the sampled learners on the answer sheet for Literacy Tasks 1 and 2 and the scoring grids. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 30 MATHEMATICS, GRADE 2 My name is ____________________________________ I am a girl / boy . I am _______ years old. 1. 2 1 5 + 3 1 1 _____ 2. 4 7 6 + 2 1 8 _____ NR 0 1 2 NR 0 1 2 3. 8 3 2 - 4 6 1 _____ 4. is the same as 4 x NR 0 1 2 3 NR 0 1 5. 6 birds have 2 legs each. The number of legs altogether is . . . 6 x 2 = legs 5. 6 birds have 2 legs each. The number of legs altogether is . . . 6 x 2 = legs 6. If 4 children share 20 mangoes equally, each child will get . . . 20 ÷ 4 = mangoes NR 0 1 NR 0 1 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 31 7. Betty has K500 and buys one pencil at K100 and one book at K300. How much change does she get? change 8. Which line is longer? Circle A or B. NR 0 1 NR 0 1 9. Which pumpkin can go in the box? Circle the pumpkin that will fit. 10. What time in the morning is the clock showing? hours NR 0 1 NR 0 1 Well done! Thank you for answering the questions. A B Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 32 GRADE 2 – MATHEMATICS TEST SCORING RUBRIC x Administer all Mathematics tasks to a group of children. x Ensure that the children write their names properly on the mathematics answer sheets. x Guide the students through each question until they have had enough time to give an answer. x Translation into the local Zambian language is allowed on all questions. x Begin by greeting and introducing yourself, and then allow learners to write their name in the space that has been provided My name is ____________________________________ I am a girl / boy . I am _______ years old. 1. 2 1 5 1 point if some figures + 3 1 1 are correct. _5 2 6 2 points for the correct answer. 2. 4 7 6 1 point if some figures + 2 1 8 are correct. _ 6 9 4_ 2 points for the Correct answer. NR 0 1 2 NR 0 1 2 3. 8 3 2 1 if carried correctly - 4 6 1 only once. 3 7 1 2 points for correct answer. 4. is the same as 4 x NR 0 1 2 NR 0 1 5. 6 Birds have 2 legs each. The number of Legs altogether is … 6 x 2 = 6. If 4 children share 20 mangoes equally, each child will get… 20 ÷ 4 = mangoes NR 0 1 NR 0 1 7. Betty has K500 and buys one pencil at K100 and one book at K300. How much change doe she get? Answer: K100 change 8. 3 12 5 A 3 12 5 A B Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 33 Which line is longer? Answer: B NR 0 1 NR 0 1 B Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 34 9. Which pumpkin can go in the box? Circle the pumpkin that will fit. One point if the bottom pumpkin is circled and the top and middle pumpkins are not circled. 10. What time in the morning is the clock showing? Ans… 08:00 hours NR 0 1 NR 0 1 Well done thank you for answering the questions Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 35 Literacy (Zambian Language) x Administer Tasks 1 and 2 as a group. x All discussion and questions must be in the local Zambian language. x Begin by greeting, introducing yourself and passing out the answer sheet for Tasks 1 and 2. Hello, my name is __________. I am very happy to see you today. Today we are going to be doing some tests to find out what you have learned in Grade 2. I will ask you some questions. Please write the answers on this paper. [Pass out the answer sheet for Tasks 1 and 2.] Now, I am to going read out words for you to write. Task 1: Word dictation and using words to form sentences Question Stem Instructions Items 11 to 15: Write the following words on your paper: For Items 11 to 15: Dictate the following words: 11. ball 12. child 13. clinic 14. washing 15. bicycle Item 16: Now here are some words. Write them on your paper. Use all the words to form a sentence. For Item 16: Let the learners see the following words – firewood, mother, is carrying – ask them to make a sentence. Rubric for Task 1 - Word dictation and using words to form sentences Items 11 to 15: [5 dictated words worth 2 points each for a total of 10 points] Writes nothing at all, or writing not legible/readable 0 Writes the dictated word but with a spelling error 1 Writes the dictated word legibly, with no spelling errors 2 Item 16: Mother is carrying firewood. Writes nothing at all, or writing not legible/readable, or the sentence is not related to the dictation 0 Writes the sentence legibly using one or two dictated words, but not the full sentence 1 Writes the sentence legibly using all dictated words, but with only one spelling error 2 Writes the sentence legibly using all dictated words, with no errors 3 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 36 Task 2: Picture Story Writing Question Stem Instructions Item 17: Write this sentence on your paper: ‘Mother is cooking nshima’. Item 18: Write a sentence that describes what is taking place in picture 2. Show the learners two pictures that are arranged in sequence on a poster. Tell the learners that the pictures make a story, and show them how the sequence progresses. Discuss the first picture with the learners to help them understand the story. Ask the learners to tell you what mother is doing in the picture. You will be listening for the following, or similar sentence: Mother is cooking nshima. Ask the learners to tell you what mother and father are doing in the second picture. Discuss the picture with the children until it clear that children understand the picture. Ask the learners to write this sentence on the paper: Mother is cooking nshima. Give the learners a maximum of 5 minutes to write the sentence. After the learners have had five minutes to write the first sentence, ask the learners to write down a sentence about Picture 2. Give the learners a maximum of 5 minutes to write. Task 2: Picture Story Scoring Rubric Item 17 (Dictation): Mother is cooking nshima Writes nothing at all, or writing not legible/readable, or the sentence is not related to the picture 0 Writes one or two words of the dictated sentence legibly, but not the full sentence 1 Writes the dictated sentence fully, correctly and legibly, but with one or more spelling errors 2 Writes the dictated sentence fully, correctly and legibly, with no spelling errors 3 Item 18 (Free writing): Mother and father are eating nshima [or something similar] Writes nothing at all, or writing not legible/readable, or the sentence is not related to the picture 0 Writes one or two words legibly, but not a full sentence. 1 Creates a full legible sentence about the picture, but with one or more spelling errors. 2 Creates a full legible sentence about the picture, with no spelling errors 3 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 37 Answer Sheet for Zambian Language Literacy Tasks 1 & 2 Grade 2, 2006 Student’s Name: _______________________________________________ Name of School: _______________________________________________ District: ______________________________________________________ Province: ____________________________________________________ Date: _____________________ 11 . ________________________________ ____ 14 . ________________________________ ____ 12 . ________________________________ ____ 15 . ________________________________ ____ 13 . ________________________________ ____ 16. _____________________________________________________________________________ _____________________________________________________________________________ 17. _____________________________________________________________________________ _____________________________________________________________________________ 18. _____________________________________________________________________________ _____________________________________________________________________________ _____________________________________________________________________________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 38 _____________________________________________________________________________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 39 • Administer Tasks 3 to one learner at a time, away from other learners. • All questions must be in the local Zambian language. • Begin by greeting, introducing yourself and checking the information about the learner on the scoring grid. Now, I would like you to read a story for me. After you read the story, I will ask you some questions. Please read the story aloud. Task 3: Reading Item Instructions Reading Text: Musa is cooking nshima. A chicken comes and eats the mealie-meal. He chases the chicken away. The chicken topples the mealie-meal. Musa is crying. Present the reading text to the individual learner. The story should be written legibly on a separate sheet of paper, without the item numbers. The sentences should appear on one sheet of paper with each sentence on a separate line. Using the rubric and the scoring grid, score each item (sentence) as the learner reads aloud. Note: Do not help the learner read the sentences. Rubric for Task 3 Item 19: Musa is cooking nshima. Item 20: A chicken comes and eats the mealie￾meal. Item 21: He chases the chicken away. The chicken topples the mealie-meal. Musa is crying. Use the rubric below to rate the learner on sentences 19, 20 and 21 as they read. The maximum possible for each sentence is 4 points per sentence. Not able to read any words at all, or mumbling incomprehensibly 0 Reads sounds or syllables, but cannot read a complete word 1 Reads one or more complete words, but cannot read the complete sentence 2 Reads all of the words of the sentence, but with hesitancy or has to repeat certain words 3 Reads all words of the sentence fluently 4 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 40 Task 4: Reading Comprehension Item 22: What was Musa cooking? Item 23: What did the chicken do? Item 24: How do you imagine Musa felt at the end of the story? Why? Get the text from the learner and ask the questions. Using the rubric and the scoring grid, score each item as the learner gives answers. Rubric for Task 4 Item 22: What was Musa cooking? Incoherent or mumbling response, or wrong answer 0 Correct answer: nshima 1 Item 23: What did the chicken do? Incoherent or mumbling response, or wrong answer 0 A partially correct answer. (For example: It ate. OR It toppled.) 1 A correct answer. (For example: It ate the mealie-meal. OR It toppled the mealie-meal.) 2 Item 24: How do you imagine Musa felt at the end of the story? Why? Incoherent or mumbling response, or wrong answer (e.g. Musa fell.) 0 A correct description of Musa’s feelings. (For example: He felt bad. He was disappointed.) 1 A correct description of Musa’s feelings and a correct explanation for his feelings. (For example: He felt bad. Because the chicken toppled the mealie-meal. OR He was disappointed. He did not have any food to eat. 2 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 41 Task 5: Life Skills Item 25: If Musa is going to cook nshima again, what should he do to prevent the chicken from toppling the mealie meal? Item 26: Why is it okay for Musa, a boy, to cook instead of doing a boy’s task, such as looking after goats? Item 27: What chores do you do at home? For a girl: Can you also cut firewood? For a boy: Can you also wash the dishes? The purpose of these questions is to test several life skills objectives: x Gender Roles Objective: To encourage the learners to recognize that boys and girls can do the same things at home and at school (Mentor’s guide, p81) x Family Roles Objective: Appreciate the importance of their role in their own family (Mentor’s guide, p.46) If the learner was unable to read the story, the story should be read aloud for the learner before asking the life skills questions.Using the rubric and the scoring grid, score each item as the learner gives answers. Rubric for Task 5 Item 25 [Objective: Problem-solving]: If Musa is going to cook chicken again, what should he do to prevent the chicken from toppling the mealie meal? Provides a realistic response with a good solution. (For example: Scare the chicken away. Cover the mealie-meal. Put the mealie-meal in a high place.) 2 Provides a realistic response, but not a good solution (Example: Kill the chicken. Poison the chicken.) 1 No response, incoherent or mumbling response, or an answer that does not provide a solution to the problem. 0 Item 26 [Objective: Gender roles]: Why is it good for Musa, a boy, to cook instead of doing a boy’s task, such as looking after goats? Provides a response; accepts that roles should be interchangeable, and cites one reason that shows maturity in thinking, such as self-empowerment, or empowerment of the sexes; self sufficiency. (For example: It is good for him to cook because boys can do the same things as girls. OR He should know how to cook so he can take care of himself.) 3 Provides a response; accepts that roles can be interchangeable, and gives practical reasons. (For example: It is good for him to cook when his mother or sister are not at home.) 2 Provides a response; insists on keeping the roles separate. (For example: He should not cook. OR He should let his mother cook because she knows how to do it better than him.) 1 No response, incoherent or mumbling response, or an answer unrelated to the question. (For example: I don’t know.) 0 Item 27 [Objective: Learners’ roles in their own families]: What chores do you do at home? For girls: Can you also chop firewood? For boys: Can you also wash the dishes? Provides a response; accepts that roles should be interchangeable, and cites one reason that shows maturity in thinking, such as self-empowerment, or empowerment of the sexes; self sufficiency. (For example: Yes, I can do that because I need to help my brother/sister/mother/father. OR Yes, I can do that. I can do the same things that boys/girls do.) 3 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 42 Provides a response; accepts that roles can be interchangeable, and gives practical reasons. (For example: Yes, I can do that when my brother/sister are not at home.) 2 Provides a response; insists on keeping the roles separate. (For example: No, I cannot do that kind of work. OR No, my brother/sister does that.) 1 No response, incoherent or mumbling response, or an answer unrelated to the question. (For example: I don’t know.) 0 Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 43 Grade 2 English Language • Administer all questions to one learner at a time, away from other learners. • All questions must be given in English. Do not translate. • If a child responds in a local language, ask the child to give an answer in English. Now, I am going to ask you questions about some pictures. The pictures are like the ones you talk about here at the learning centre, OK? Now, here are the pictures: Task 1: Speaking—Naming Occupations Question Stem Instructions 28. What is this person’s job? Show five pictures one at a time to each individual learner. The pictures show people from the following occupations: doctor/health worker, police officer, teacher, farmer/gardener, driver. Hold up a picture and ask the learner, “What is this person’s job? The learner earns one mark for each job identified correctly. After asking about each of the five pictures, give the learner a second chance if necessary. Scoring: One point for each correct word for a total of 5 possible points. Task 2: Describing a Common Activity Now, I am going to ask you a question about yourself. Please tell me everything that you can. Question Stem Instructions 29. At school you learn. What do you do after school? 30. Last Friday you went to school. What did you do last Saturday? Ask the learner the question. Rate each response using the rubric below. Rubric for Task 2 Scor e Description NR No response is given. 0 Says nothing at all, or speaking not comprehensible, or the response is not related to the question, or the response is not in English. 1 Says one or two comprehensible words that are related to the question, but not a full sentence. For example: Play. Eat. Church. 2 Says a comprehensible phrase or sentence related to the question, but with grammatical or structural errors. For example: 29. I played football. Do homework. 30. I go to market. Play football. 3 Says a comprehensible sentence related to the question with no grammatical or structural errors. For example: 29. I play football. I wash dishes. I do my homework. I play with friends. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 44 30. I went to the market. I played football. I went to church. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 45 Task 3: Reading—Matching Words with Pictures Show four pictures and six words written on flash cards to each individual learner. The six words are: cup, ball, window, flower, sister, clock. Point to a picture and ask the learner what it is, then ask the learner to point to the word for that picture. The learner earns one mark for each word identified correctly. After asking about each of the four pictures, give the learner a second chance if necessary. Now, I am going to ask you questions about some pictures. Here are the pictures: 31a. What is this? Now point to the word “cup”. 31b. What is this? Now point to the word “ball”. 31c. What is this? Now point to the word “window”. 31d. What is this? Now point to the word “flower”. Scoring: One point for each correct word for a total of four possible points. Task 4: Reading—Matching Sentences with Pictures [Show four pictures and four sentences written on a sheet of paper to each individual learner. Point to a picture and ask the learner a question about the picture. The purpose of the question is to help the learner understand the idea of the sentence: the answer to the question is NOT scored. Then ask the learner to point to the correct sentence. The learner earns one mark for each word identified correctly. After asking about each of the four pictures, give the learner a second chance if necessary.] Now, I am going to ask you questions about some more pictures. Here are the pictures: 32a. Is this cat small? (Learner: No, it is big.) Now point to the sentence “The cat is big.” 32b. What is the man carrying? (Learner: He is carrying a bag.) Now point to the sentence “The man is carrying a bag.” 32c. Where is the book? (Learner: On a table.) Now point to the sentence “The book is on a table.” 32d. What is the girl doing? (Learner: Kicking a ball.) Now point to the sentence “The girl is kicking a ball.” Scoring: One point for pointing to a correct sentence, for a total of four possible points. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 46 Task 5: Writing Words from Dictation [Give the learner an answer sheet. Tell the learner that you will read a sentence and then he or she will need to write a word. Say each word two times after reading the sentence.] Now, I am going to ask you write some words. First, I will read a sentence and then I will ask you to write one word from the sentence. 33. We write in an exercise book. 34. The tree is tall 35. The bus is on the road. 36. The dress is yellow. 37. Mother is sweeping. Scoring rubric for Items 33 to 37: 5 dictated words worth 2 points each Scor e Description NR No response is written. 0 The word is spelt completely wrong. That is, not a single letter is written correctly. 1 At least one letter from the word is written correctly. 2 The word is spelt correctly. Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 47 Answer Sheet for English Task 5: Dictation Grade 2, 2006 Student’s Name: ___________________________________________________ 33. _______________________________ 34. _______________________________ 35. _______________________________ 36. _______________________________ 37. _______________________________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 48 Appendix B: Questionnaire for GRZ Teachers Instructions: This questionnaire collects information about Grade 2 classes in Zambian government schools that are using Taonga Market radio broadcasts. Please answer all questions. If no information was available or obtained, write “NA” in the space available for answering the questions. The respondent for this questionnaire is a Grade 2 teacher who is currently using Taonga Market educational broadcasts in a government school or a Grade 2 teacher in a control school. Record the answers by circling the answer or writing down a response. Section A: General Information A1. Date: _______________________ A2. Province: _____________________________ A3. District:_______________________ A4. Village/Compound: _________________________ A5. Name of the school: ___________________________________________________________ A6. Is the school using IRI or is it a control school? (Circle one.) 1. Using IRI 2. Control school A7. Where is the school located? 1. In an urban area 2. In a rural area A8. Name & Position of Test Administrator: _____________________________________________ A9. Name of Grade 2 teacher interviewed: _______________________________________________ A10. What is the teacher’s highest educational attainment? a. Grade 7 b. Grade 9 c. Grade 12 d. Other. Explain: _____________________________ A11. What is the teacher’s highest teaching qualification? (Example: Primary Teaching Certificate) _______________________________________________________________________ A12. How many years of pre-service training has the teacher completed? a. 1 year b. 2 years c. 3 years or more d. None A13. How many years has the teacher been teaching? Number of years: __________ A14. How many teachers have taught the Grade 2 class for a week or more this year? Number of teachers: __________ A15. How many days of training in Learning at Taonga Market has the teacher completed? Number of days: __________ A16. How many years has the teacher used Learning at Taonga Market? Number of years: __________ A17. How many classes and pupils at each grade level is the teacher teaching? Fill in the table. Grade 1 2 3 4 5 6 7 Totals Number of Classes Number of Pupils Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 49 A18. How much time does the teacher usually spend with the Grade 2 class each day? Amount of time: __________ Note: Questions A19 and A20 are only for teachers who use Learning at Taonga Market. A19. How much time does the teacher usually spend with children on their learning tasks before the broadcast? a. No time at all b. 5 – 10 minutes c. 11 – 30 minutes d. More than 30 minutes A20. How much time does the teacher usually spend with children on their learning tasks after the broadcast? a. No time at all b. 5 – 10 minutes c. 11 – 30 minutes d. More than 30 minutes Section B: Language B1. What language do the pupils use when playing? 1. Chitonga 2. Cinyanja 3. Icibemba 4. Lunda 5. Luvale 6. Kiikaonde 7. Silozi 8. Other: ______________ B2. What Zambian language is used for literacy instruction in the teacher’s class? 1. Chitonga 2. Cinyanja 3. Icibemba 4. Lunda 5. Luvale 6. Kiikaonde 7. Silozi 8. Other: ______________ B3. Is the language of play the same as the language of instruction? 1. Yes 2. No B4. How well can the teacher speak the Zambian language used for literacy instruction? 1. Very well 2. Satisfactorily 3. Slightly 4. Not at all B5. How well can the teacher write the Zambian language used for literacy instruction? 1. Very well 2. Satisfactorily 3. Slightly 4. Not at all B6. What Zambian language is being used in this test of Zambian language literacy? 1. Chitonga 2. Cinyanja 3. Icibemba 4. Lunda 5. Luvale 6. Kiikaonde 7. Silozi 8. Other: ______________ B7. Is the Zambian language of instruction the same Zambian language used on this test? 1. Yes 2. No Section C: Resources Instructions: Questions C1 to C7 are only for schools that use IRI. When interviewing teachers at control schools, skip questions C1 to C7 and begin with question C8. C1. Is the radio working? 1. Yes 2. No C2. In general, how clear is the radio reception? 1. Very Clear: Loud with little or no noise from the radio 2. Acceptable: A little soft or noisy, but loud and clear enough to hear all words in the broadcast 3. Hard to hear: Occasionally too soft or noisy to hear some words 4. Impossible to hear: Frequently too soft or noisy to hear words C3. How many days per week is the reception Acceptable or Very Clear? All 5 days 4 days 3 days 2 days 1 day 0 days Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 50 C4. During 2006, were there particular times of the day or year when there was poor radio reception or no radio reception at all? 1. Yes 2. No If Yes, when was there poor radio reception or no reception at all? ________________________ ______________________________________________________________________________ C5. Does the teacher use antennae extensions to improve radio reception? 1. Yes 2. No C6. If the teacher does not use antennae extensions, why not? 1. Radio reception is very clear 2. Doesn’t know how to make an antennae extension 3. Other reason: _____________________________________________________________ C7. Has the teacher received the timetable that includes Taonga Market lessons? 1. Yes 2. No If Yes, how helpful is it to your teaching? 1. Very helpful 2. Helpful 3. Not helpful Reasons for answer: ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ C8. Did the school have enough materials to use during the third term? Chalk 1. Yes 2. No Pencils 1. Yes 2. No Exercise books 1. Yes 2. No Mentor’s guide 1. Yes 2. No MOE register 1. Yes 2. No C9. How many pupils share a Zambian language reader at a time? 1. Each child has a book 2. Two children share 3. Three children share 4. Four or more children share 5. No Zambian language readers to share C10. How many pupils share a SITE activity book at a time? 1. Each child has a book 2. Two children share 3. Three children share 4. Four or more children share 5. No SITE activity books to share C11. How many pupils share an English language story book at a time? 1. Each child has a book 2. Two children share 3. Three children share 4. Four or more children share 5. No English language story books to share C12. How many pupils share a mathematics book at a time? 1. Each child has a book 2. Two children share 3. Three children share 4. Four or more children share 5. No mathematics books to share C13. What do children put their exercise books on when writing in class? 1. Table 2. Desk 3. Bench 4. Floor 5. Other: ______________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 51 Section D: Use of Step Into English (SITE) D1. Was the teacher trained in SITE? 1. Yes 2. No If Yes, how was the teacher trained in SITE? (Circle each type of training received.) 1. Through school mentoring 2. Through a school workshop 3. Through a district workshop 4. Through a national workshop 5. No training D2. Does the teacher have each part of the SITE kit? Fill in the chart based on information from the teacher: Does the teacher have the . . . a. Pathway 2: Oral English? 1. Yes 2. No b. SITE Teacher’s Guide? 1. Yes 2. No c. 4 conversation posters? 1. Yes 2. No d. Learner’s Activity Books for Grade 2? 1. Yes 2. No e. Story books for Grade 2? 1. Yes 2. No D3. Does the teacher use the SITE kit to teach literacy? 1. Yes 2. No D4. If the teacher does not use the SITE kit, why not? 1. Doesn’t have kit 2. No time for it 3. Doesn’t see the need for it 4. Doesn’t know how to use it 5. Other reason: ______________________________________________________________ Note: Question D5 is only for teachers who use SITE. D5. If the teacher does use the SITE kit, how did the teacher organise the class when using the SITE kit during Term 3? 1. As a single group 2. As four social groups 3. As four ability groups 4. Other way of organising the class: __________________________________________________________________ Note: Question D6 is only for teachers who use Learning at Taonga Market and SITE. D6. How well do Taonga Market radio broadcasts and SITE support each other? 1. Very well 2. Somewhat well 3. Not well at all Reasons for answer: ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 52 ____________________________________________________________________________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 53 Section E: Use of Mathematics Rainbow Kit (MARK) E1. Was the teacher trained to use the Mathematics Rainbow Kit (MARK)? 1. Yes 2. No If Yes, how was the teacher trained to use MARK? (Circle each type of training received.) 1. Through school mentoring 2. Through a school workshop 3. Through a district workshop 4. Through a national workshop 5. No training E2. Does the teacher have the MARK Teacher’s Guide? 1. Yes 2. No E3. Does the teacher use the MARK Teacher’s Guide to teach numeracy? 1. Yes 2. No E4. If the teacher does not use the MARK Teacher’s Guide, why not? 1. Doesn’t have the book 2. No time for it 3. Doesn’t see the need for it 4. Doesn’t know how to use it 5. Other reason:___________________________ __________________________________________________________________ Note: Question E5 is only for teachers who use MARK. E5. If the teacher does use the MARK Teacher’s Guide, how does the teacher teach the class when using it? 1. As a single group 2. As four social groups 3. As four ability groups 4. As two separate streams 5. Other way of organising the class: __________________________________________________________________ __________________________________________________________________ Note: Questions E6 is only for teachers who use Taonga Market and MARK. E6. How well do Taonga Market radio broadcasts and MARK support each other? 1. Very well 2. Somewhat well 3. Not well at all Reasons for answer: ___________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 54 Section F: Use of Multigrade Teaching F1. Was the teacher trained to use Multigrade Teaching? 1. Yes 2. No If Yes, how was the teacher trained to use Multigrade Teaching? (Circle each type of training received.) 1. Through school mentoring 2. Through a school workshop 3. Through a district workshop 4. Through a national workshop 5. No training F2. Does the teacher have a teacher’s guide for multigrade? 1. Yes 2. No F3. Does the teacher involve the Grade 2 pupils in multigrade teaching? 1. Yes 2. No If Yes, describe the members of the multigrade class that included the Grade 2 learners, giving the grade level and the number of learners in the class. Grade 1 2 3 4 5 6 7 Total Number of Pupils Note: Questions F4 and F5 are only for teachers who use Learning at Taonga Market and multigrade teaching. F4. If the teacher does involve the Grade 2 pupils in multigrade teaching, how does the teacher combine the use of Learning at Taonga Market and multigrade teaching? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ F5. How well do Learning at Taonga Market radio broadcasts and multigrade support each other? 1. Very well 2. Somewhat well 3. Not well at all Reasons for answer: ___________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ Learning at Taonga Market: An Evaluation of Interactive Radio Instruction at Grade Two in 2006 Page 55 Section G: Enrolment G1. Instructions: Fill out this table based upon data from the register. Girls Boys Totals How many Grade 2 learners were enrolled in term 3? How many Grade 2 learners were enrolled in Term 1? What is the difference in the average attendance? (Use + to show an increase and – to show a decrease.) G2. If Grade 2 attendance has increased or decreased, explain why: ________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________