Zambia Quality Education Services Through 
Technology (QUESTT) Project
Impact of Interactive Radio Instruction
Grade 1
Prepared by: 
Republic of Zambia – Ministry of Education
Education Broadcasting Services
Education Development Center, Inc.
USAID Award No.: 690-A-00-03-00096-00
December 2001
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 ii
TABLE OF CONTENTS
EXECUTIVE SUMMARY...........................................................................................................iii
1.0 BACKGROUND................................................................................................................... 1
1.1. Purpose of the Evaluation........................................................................................... 2
2.0 METHODOLOGY................................................................................................................ 3
2.1 Sample................................................................................................................................ 3
2.2 Test Development ............................................................................................................ 3
2.3 Data Collection ................................................................................................................. 6
3.0 FINDINGS............................................................................................................................. 8
3.1 Demand for IRI................................................................................................................. 8
3.2 Attendance ...................................................................................................................... 10
3.3 Profile of the Learners.................................................................................................... 11
3.4 Performance in basic Literacy and Numeracy........................................................... 14
3.5 Impression on Effectiveness of IRI: Mentors and other IRI workers...................... 21
4.0 LESSONS AND RECOMMENDATIONS....................................................................... 23
4.1 Lessons............................................................................................................................. 23
4.2 Recommendations.......................................................................................................... 23
5.0 APPENDICES ..................................................................................................................... 27
Appendix A - Data collection sheet for students’ profile................................................. 29
Appendix B - Content Mapping for Grade 1 - English..................................................... 30
Appendix C - Content Mapping for Grade 1 - Mathematics ........................................... 31
Appendix D - Test Blueprint for Grade 1 - English........................................................... 32
Appendix E - Test Blueprint for Grade 1 - Mathematics.................................................. 33
Appendix F - Test Plan: Basic Literacy and Numeracy .................................................... 34
Appendix G - Test of Mathematics and English Language Skills................................... 35
Appendix H - Test Administration Training Notes .......................................................... 38
Appendix I – IRI Success stories .......................................................................................... 42
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 iii
EXECUTIVE SUMMARY
A mid-pilot evaluation was conducted in October 2000 after the first 50 programs of 
Grade 1 level Interactive Radio Instruction (IRI) programs. That evaluation addressed 
the questions of whether students were attending and remaining in the program, and 
how much they were learning. The present evaluation was conducted in August 2001 
after all 100 programs had been broadcast to a national audience attending radio￾learning centers in every province. It addresses the same fundamental questions as 
those asked in the evaluation of the pilot, but contains more information about the out￾of-school audience. Data for this evaluation was collected from a sample group of 30 
learning centers receiving the radio broadcasts daily over a 20-week period. The data 
provides insights into enrollment, daily attendance, learner characteristics, and student 
mastery of basic numeracy and literacy skills using a curriculum based achievement 
test. 
The 2001 evaluation demonstrates the persistent demand for IRI. EBS recorded a total 
of 251 organized learning centers in all nine provinces. The number of reported learners 
enrolled is significant (approximately 9,250). Numerous “spontaneous” IRI centers 
organized by a variety of people have also been reported. The composition of learners 
has become increasingly varied as new student groups from the early childhood and 
adult populations are becoming involved in IRI, signaling a growing interest in the 
program. The percentage of orphans reportedly enrolled at centers has increased from 
27.7 to 33.2 percent since the 2000 pilot, while enrolment by sex is estimated as equal 
with slight variance in the urban enrollment reflecting 51.2% female and the rural 
enrollment reflecting 51.4% male.
 Students are learning with IRI. Results of the Grade 1 level achievement test in basic 
numeracy and literacy indicated that students demonstrated more mastery in numeracy 
skills (71.5% mean score) than literacy skills (56.6% mean score). On the whole, student 
performance increased by age with learners aged 18 years and over scoring 99 to 100 
percent on numeracy test items. Variance in performances by gender was virtually 
nonexistent. Also, there was no significant difference in the performance of urban 
versus rural centers (while more urban centers performed above the mean, two of the 
best performing centers were in rural areas). 
The evaluation revealed, however, that the programme is facing a number of 
challenges. The following recommendations have been suggested: 1) due to the fact 
that the program is attracting a wider range of clientele than was originally intended, 
the Ministry of Education (MOE) should articulate a simple regulatory IRI strategy that 
provides the minimum requirements that should be met in order to ensure that the 
program reaches those who need it most; 2) mentors should be encouraged to keep 
attendance information and a journal documenting critical incidents; 3) in order to 
provide consistent descriptions of learning achievement and for year to year 
comparisons, the Educational Broadcasting Services (EBS) should consider a standard 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 iv
setting exercise; 4) to improve mentor retention, MOE should delineate the profile of an 
ideal mentor for the IRI program, his/her training needs, including coaching during 
monitoring visits, as well as in-service training where possible; 5) MOE should continue 
to seek partners in the development and delivery of IRI as well as to explore options for 
mentor incentives such as preferential admission to teacher training because the 
support of mentors and centers is critical and 6) due to the rapid expansion, there is a 
need for EBS to improve data management and to move from the current method of 
entering data on a spread sheet to developing a database of center, student and mentor 
profiles in order to improve efficiency and allow greater manipulation of data and 
easier linkage to the MOE’s Planning Unit. 
The test results attest to the effectiveness of IRI in providing sustainable, accessible, 
quality basic education for out of school children and adults. The strong demand for 
IRI is apparent in the proliferation of spontaneous centers established for a wide range 
of target and non-target learning groups. The Grade 1 test served its purpose as a 
valuable monitoring and evaluation tool to identify skill areas where learning is taking 
place and highlighted the need for exploring sustainable ways of supporting centers 
and suitable assessment procedures to more accurately ascertain content-specific, 
individualized student performance evaluations. 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 1
1.0 BACKGROUND
The present IRI project in Zambia is founded on a mandate to provide an alternative
delivery system for the existing curriculum. The IRI Zambia project team began its 
formal work in 1999 by training scriptwriters, developing an IRI masterplan for the 
Grade 1 curriculum, writing lessons and producing radio programs, as well as 
recruiting and training mentors who would run the learning centers. These initial 
activities were followed up by an official launch of the program in July 2000 when 
Grade 1 programs went on air for pilot testing. Twenty-two centers participated in the 
pilot, two in Monze, eight in Chongwe, and the rest in Lusaka. 
Several evaluation activities were undertaken as part of the pilot-testing phase. These 
included formative evaluation of programs, where writers took turns to test their 
programs at a learning center established at EBS for this purpose, collecting baseline 
data on how much knowledge the children had as they began the IRI program as well 
as an appraisal of their attendance and how well they had learnt after 50 programs. A 
summative evaluation was also conducted at the end of the pilot phase. 
IRI programs for Grade 1 went on air nationwide in March 2001. A total of 35 districts 
(out of a total of 72) in all nine provinces reported the existence of 251 learning centers. 
169 centers sent detailed information on the learners as indicated in the table below.
Table 1: Number of Grade One Centers in 2001 by Province and District
Province No. of Districts 
with IRI
No. of centers 
recorded
No. of centers 
data received
Central 5 32 17
Copperbelt 6 30 21
Eastern 7 22 21
Luapula 1 5 5
Lusaka 4 61 48
Northern 4 22 11
Northwestern 3 37 8
Southern 4 33 29
Western 1 9 9
TOTAL 35 251 169
The total number of learners that are enrolled at IRI centers is reported to be 9,250. 
Some centers, however, only reported the total number of students but did not provide 
the student profile data. Student profiles were received from only 169 out of the 251 
centers. The total number of learners for whom more complete data was received was 
7,782 as reflected in the Table 2 below. 
Table 2: Number of Grade One Students Enrolled in 2001
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 2
Province Female Male Total
Central 465 553 1018
Copperbelt 445 485 930
Eastern 426 400 826
Luapula 117 134 251
Lusaka 1257 1252 2509*1
Northern 306 365 671
Northwestern 186 178 364*2
Southern 486 503 989
Western 100 124 224
TOTAL 3788 3994 7782
IRI programs for Grade 1 (a total of 100 programs) were broadcast twice a day on 
weekdays, running for 20 weeks, and ending in August 2001. At the end of the 
broadcasts a sample of 30 learning centers was selected for a monitoring and evaluation 
study.
1.1. Purpose of the Evaluation
The purpose of this exercise was to monitor whether learners continued to attend at the 
learning centers for their daily broadcasts, and if those that were enrolled in the IRI 
programs achieved basic numeracy and functional literacy in English as (stipulated in 
the Zambian curriculum). Also investigated was whether the programs were benefiting 
the population for whom they were intended, out-of-school children experiencing 
socio-economic hardships. To that end, the evaluation focused on the following 
questions:
1. Is there a demand for IRI programs (i.e., are learning centers established and 
learners enrolled in them)?
2. Do learners attend daily broadcasts at the learning centers?
3. What are the characteristics of the children who attend?
4. Are learners achieving basic literacy and numeracy as expected at the Grade 1 
level?
5. What is the impression of the beneficiaries, individuals associated with IRI such 
as mentors, and the general public on the effectiveness of IRI as a means of 
imparting basic literacy and numeracy skills?
Data on enrollment and background characteristics of the learners was collected at the 
selected centers, while attendance data was collected in a smaller number of centers. In 
 1 2,509 indicates the actual number of students for whom detailed information was received. A total of 3,958 students was reported 
for Lusaka. However, some reporters provided information (of 1, 449 students) as total numbers and did not send in the student 
profiles. 
2 As with Lusaka, the number of students for whom detailed information was received is 364 but the actual number 
of students reported was 383.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 3
addition to this, learners were tested on mastery of basic numeracy and English 
language skills using a curriculum-based achievement test. Since programs are 
delivered through the medium of radio, some adjustments were made to the existing 
curriculum to make it suitable for radio.3 These differences were accommodated in the 
achievement test that was developed. The sections below present a detailed discussion 
of the methodology, followed by findings resulting from the evaluation study.
2.0 METHODOLOGY
2.1 Sample
The population of IRI learners for which data was received was 7 782 learners attending 
169 centers. This population was sampled in two stages, by center and by learner. 30 
centers were sampled, 14 of which were located in urban areas and 16 in rural 
communities. The total number of learners in the 30 centers was 1898, which amounted 
to more than 24 per cent of the sample. The sample of 1898 learners was used describe 
the profile of learners in the centers, in particular their sex, age, who their guardians 
are, and whether they have any living parents (orphan status). 
While an attempt was made to obtain data on learner characteristics from all learners at 
each of the 30 selected centers, only 20 percent of the learners were randomly selected 
for achievement testing at each center, yielding a total of 405 examinees. 53.3 percent of 
the examinees attended urban centers, while 46.7 were in rural centers.
2.2 Test Development
IRI methodologies emphasize a communicative learning approach, where instead of 
using the traditional learning approach of compartmentalizing into subject areas, 
presentation of new knowledge and skills is integrated. This has made it possible to 
develop a test of literacy and numeracy skills, that was packaged in the manner that 
learners are familiar with from their daily lessons, and presented to them orally. This 
section describes the rationale for developing a literacy and numeracy achievement test 
for Grade 1. A mastery test was developed for the assessment. The test development 
process entailed content mapping, test blueprinting, developing a test plan, item 
writing and pilot testing. The test administration procedure is described, followed by a 
presentation and discussion of test results.
2.2.1 Content Mapping
The test development process commenced by a content analysis for Grade 1, performed 
by the Senior Research Officer with the Examinations Council of Zambia (ECZ) and a 
test development specialist. Documents that were analyzed are the Mathematics and 
English syllabi which specify terminal as well as instructional objectives for the primary 
 3 The most significant adjustment has been the use of English as a medium of instruction for numeracy skills, and 
sequencing of the curriculum.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 4
level, and the mentors’ guide for the lesson broadcasts. Two content maps were 
developed, one for English and another for Mathematics. In these, instructional 
objectives that specify the content for the grade level were classified in a two-way 
matrix of terminal objectives by topic or skill area. In the absence of grade-level reading 
lists that usually indicate the reading levels of learners, the mentors’ guide was 
particularly useful in that it specifies new vocabulary and the numeracy skills that are 
presented in each lesson. It also formally specifies the life skills content, a deliberate 
effort to advance attitudinal learning alongside the cognitive skills that children have to 
master at this formative stage of being introduced to formal learning.
In the content map for English instructional objectives are classified into a two-way 
matrix of twelve Grade 1 terminal objectives by the skill areas of reading, writing, 
speaking, listening, phonics and spelling (see Appendix B). Out of the 106 instructional 
objectives that make up the content for Grade 1 English, 82 are terminal. These are all in 
the areas of listening and speaking. The remaining 24 that fall in the areas of reading 
and writing are developmental objectives to be mastered in later years. 
The first three terminal objectives for the primary Mathematics course were deemed 
terminal for Grade 1. Learners at this level are expected to master the skills of naming 
and classifying objects according to given conditions, demonstrate understanding of the 
concept of number and numeration, and perform basic operations on numbers and 
other simple measures. All the 22 instructional objectives which are specified in the 
content map are terminal, 10 of which require learners to demonstrate understanding of 
the concept of numbers and numeration (see Appendix C).
2.2.2. Subject Blueprinting
Test blueprints were developed for English and Mathematics (Appendices D and E). In 
these, instructional objectives were mapped into a two-way matrix of content topics by 
level of cognitive complexity using Bloom’s taxonomy4. Even though there application 
of what was learnt in both English and Mathematics is expected at this level, greater 
emphasis was placed on acquisition of new knowledge and comprehension of concepts 
for both subjects. The higher cognitive levels of Bloom’s taxonomy (analysis, synthesis, 
and evaluation) were not featured at all.
2.2.3. Test Planning
One test plan of literacy and numeracy knowledge and skills was developed. Again, a 
two-way matrix of topic/skill area by cognitive level was used. Entries in the body of 
the test plan were instructional objectives from both the Mathematics and English 
syllabi (see Appendix F). The skills area of the test plan (first column) presented 
terminal objectives, where the first four skills was a summary of the 13 terminal 
objectives of Grade 1 English. The last three skills (in italics) reflect the mathematics 
 4 Bloom’s taxonomy was the classification scheme that is used by the Ministry of Education. 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 5
terminal objectives that are suitable for Grade 1. 
The optimal test length that was decided upon by the curriculum and test development 
specialist was about 20 items, taking into consideration the method of administration, 
and the time that it would take to administer the test. The plan reflected total of 22-item 
test consisting of 12 literacy items and 10 numeracy items, sampled from 20 
instructional objectives. Four items were planned at the knowledge level, 13 at 
comprehension, and 5 at the application level. The last column of the test plan 
stipulated the number of questions to be tested per skill, and the different levels (using 
Bloom’s taxonomy) at which the items for each skill would be tested. Instructional 
objectives that were sampled for testing are presented in bold in the body of the test 
plan.
2.2.4. Test Construction
The purpose of the test was to assess and evaluate if learners have mastered basic 
literacy and numeracy skills, and whether they could understand simple 
communication at the end of the Grade 1 course. The guiding principle during test 
development was that assessment procedures should match the intentions of each 
learning target, hence the behaviors which were elicited from the learners included 
recalling certain facts, as well as performing certain tasks. For instance, the intention of 
the learning targets on language during the early stages of learning is that learners 
should comprehend language and begin to produce simple language. Their 
comprehension of language in the lessons is demonstrated by the acting out simple 
instructions, hence the assessment of language skills comprised mainly of requesting 
them to perform actions when given simple instructions. 
At least one item was constructed for the 20 objectives that were sampled from the test 
plan. The test development specialist was mainly responsible for generating the test 
items. The ECZ Senior Research Officer, an individual with a thorough knowledge of 
the intended curriculum, reviewed these for content coverage and readability. 
Members of the IRI team who are most conversant with the way the radio lessons were 
structured and delivered also participated in the item review. 
For most objectives it was possible to construct a set of parallel items from which the 
test administrator would select the item to present to the learner. These items were pilot 
tested by 4 test administrators in 2 centers in Lusaka. Pilot testing assessed whether the 
questions elicited the intended behavior/skills, whether the correct difficulty levels in 
terms of content and language were maintained, the amount of time it took to 
administer the test, and whether the proposed administration procedure was 
reasonable and adequate.
Modifications that were made after pilot testing included simplifying content for some 
of the questions, recasting other questions in simpler language, clarifying certain tasks, 
and inserting instructions, also to increase clarity on the administration procedure. 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 6
Table 3 below produces a summary of intended learning targets (also reflected in the 
test plan) and the corresponding test items in the final version of the test. The final 
version of the test consisted of 22-items (see Appendix G). 
 Table 3: Skill areas and corresponding test items for Grade 1 Test, 2001
Skill Area Intended Learning Target Test Items
1. Recalling names
2. Simple comprehension of language
11, 12, 
13, 14, 15, 19
3. Production of language (speaking)
4. Production of language (reading)
16, 17, 18
20, 21
Language
5. Production of language (writing ) 22
1. Counting and writing numbers 1, 2, 3, 4
2. Naming/classification/comparisons 9, 10
Numeracy
3. Number operations 5, 6, 7, 8
Twelve (12) items assessed language skills, while 10 items assessed numeracy skills. 
Translation into local languages was allowed for the 10 items that tested numeracy. In 
the interest of keeping the test short and simple, no items were included for the lifeskills 
component. Also, the objectives for the lifeskills component are not expressed explicitly 
in the curriculum, which means that a different strategy will have to be used to assess 
whether children are learning in this area.
2.3 Data Collection
2.3.1 Training of Test Administrators
Training of test administrators started with a small team of three individuals from ECZ 
(who were trained by the testing specialist in preparation for trial testing on a 
convenience sample of 2 centers in Lusaka). The team was briefed on the purpose of 
test, how the test was developed, how the test was to be administered, and features that 
were to be trial tested. The training session was also used as an opportunity to receive 
feedback on the reasonableness and appropriateness of the test for testing Grade 1 
literacy and numeracy skills. After trial testing, a debriefing session was convened at 
which additional feedback on how the test functioned and the interactions between 
learners, test administrators, and the test were noted for interpretation, and to improve 
the test. Upon finalizing the test, training notes were prepared for another ECZ Senior 
Research Officer (see Appendix H), who was responsible for training the live 
administrators.
The ECZ trainer trained six people to administer the tests. The District In-service 
Provider from Monze administered them in Monze; the former DEO from Chongwe 
administered them in Chongwe; a Peace Corps Volunteer administered them in Petauke 
in Eastern Province; and ECZ and EBS staff administered them in Lusaka. The 
philosophy behind this testing exercise was for administrators to ensure that learners 
understood the tasks clearly, hence the instructions in most questions, to give children 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 7
two chances to respond, and more chances if the test administrators was not show hoe 
to code the response. The sentiments of this philosophy are captured in the comments 
of one test administrator: 
Tests were administered in early August during the last week of broadcasting. A 
convenient sample of 405 Grade 1 learners were selected from the sample o 30 centers. 
There were14 centers in Lusaka urban, 7 centers in Chongwe (Lusaka rural), 4 in 
Petauke (Eastern Province) and 5 in Monze (Southern Province). 
 
2.3.2 Test Administration 
Of the 14 centers tested in Lusaka urban, five were part of the September, 2000 pilot 
sample. However, in August 2001, test administrators tested no more than five children 
that had been part of the pilot sample. All but three centers are part of the original pilot 
centers. Some new ones selected for testing had to be abandoned for logistical reasons, 
or after administrators failed to locate them (many new centers have not been visited by 
EBS or the MoE since the this year’s cohort of mentors were trained in February). Each 
test administrator was required to arrive at the center in time for the broadcast, test the 
learners and record the responses accordingly, as well as submit a report for the testing 
visit. 
2.3.3 Interpretation of test scores 
The purpose of this test was to monitor if learning is taking place and describe the 
learning patterns, first, for the whole group of Grade 1 learners, and secondly, for 
subgroups. Subgroups that have been identified for analysis include males, females, 
rural learners, urban learners, learners in specific centers, among others. The test is 
therefore suitable and valid for monitoring learning at a group level, but not equally 
valid for describing achievement for individual learners. 
In order to enhance the validity of the test as a monitoring tool, content domains or skill 
areas have been clearly delineated, hence it is possible to calculate means for domain 
“subtests”. This is the most valuable information for our purposes because conclusions 
can be made confidently about skill areas where students seem to have achieved the 
most, and where they have been found to be weakest. Data on individual items on the 
test is only useful to the extent that it provides an indication of what learning 
difficulties learners could be having on specific content matter that each item is testing. 
However, it has also been possible to draw conclusions about mastery for the group of 
learners at the item level. The test was built such that for most items, administrators 
could randomly sample the item to be presented from a list of parallel stimuli (see Items 
1, 2, 5, 6, 7, 8, 11, 12, 13, 17, 18, 19, 20, 21, 22). The implication of this practice is that 
We gave the students ample time to think, we gave them examples, we even 
gave them hints and contextual clues and non-verbal cues. We didn’t aim, in 
other words, to stump them. The benefit of the doubt lay with pupils.
Testing Report for the Eastern Province, Hunter Nielsen, 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 8
conclusions on content as presented in individual items were made not only on one 
item, but on a number of such items. The other strength was that there were different 
combinations of items that were presented to the students, which practice increased the 
validity of the test for making mastery decisions.
As a tool for monitoring learning and making summative evaluations about the 
learning environment, the test has performed its intended purpose adequately. 
Conclusive evidence on performance in content specific items at the individual student 
level would be derived from a different process, which necessitates that there be more 
than just one item on each topic area, that sampling of items be adequate, that the test 
be of a suitable length, and that testing time be more suited to learners at the Grade 1 
level. The usual practice for such individual testing is that the achievement test for the 
whole curriculum is broken down into manageable subtests that are administered in 
several sittings. 
3.0 FINDINGS
3.1 Demand for IRI
3.1.1 New Centers
EBS receives information on the new learning centers that open, as well as the number 
of learners who are enrolled at each center. Approximately 251 IRI learning centers 
opened up in the nine provinces when the program went to scale in March, 2001. 169 of 
these sent information to EBS indicating an enrolment of approximately 9,250 learners 
who received Grade 1 programs. 9,250 learners may seem to be a small number in 
comparison to the total number of Zambian children of school going age who are out of 
school (an estimated 700,000 in 20005). However, the fact that a significant number of 
children are taking advantage of this opportunity to engage in organized learning 
constitutes clear evidence that IRI programs are in demand.
People are showing the value they place on the programs in various ways. In Petauke 
(Eastern province) for example, communities are so determined to ensure the success of 
the program that they have decided to cultivate their mentors’ fields as their way of 
supporting the program. Near Lundazi (Eastern province), an individual who had no 
access to a radio would go to a neighbor where people congregated to listen to the radio 
program. He listened, took notes then went back to his organized learning group to 
teach what he learnt. 
In Chongwe (Lusaka rural) one community decided they wanted the program to stay 
after watching its success during the pilot and put extra effort to complete a shelter for 
the radio learners that had stalled for a long time. A similar story comes from Chipata 
 5 Basic Education Sub-Sector Investment Programme (BESSIP): 2000 Programme Performance Indicators," Republic 
of Zambia, Ministry of Education, November 30, 2001.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 9
(Eastern province), where the coming of the radio program provided one community 
with the incentive they needed to realize a dream they had had for several years: they 
built a school that can accommodate at least two classes where there was nothing but a 
tree before! In Garden and Linda compounds of Lusaka, mentors have been offered 
plots to build classrooms for their centers. The current Garden center is under a tree 
within the grounds of Ngwelele school while the center in Linda is in the mentor’s 
home. In Jack compound, the community has shown their support of the program by 
building a toilet for the children next to the church where the children meet to listen to 
the radio program. In Central province, despite not having been trained or received the 
mentor’s guides for some time, several communities went ahead and a total of 32 
centers. 
3.1.2 New Audiences
The emergence of IRI is having an effect on mobility between centers and schools in 
some places. In Monze District, the usually large numbers of children enrolling and 
waiting to enroll in Grade 1 did not occur in 2001. It is believed that a significant 
number of them took advantage of the opportunity provided by the radio programs 
and enrolled in Interactive Radio Learning Centers (IRLCs). However, in this survey, 
there was also evidence of the opposite movement in an urban center after the pilot 
phase.
This kind of movement reflects a vision articulated by the MOE: that the various 
alternatives being introduced to ensure that all Zambian children have access to basic 
education will allow easy access between government schools, community schools and 
radio centers depending on the circumstances of the child. 
But since the effect of and motivation for this mobility are not yet clear, it seems to be an 
important issue to investigate in the qualitative study proposed for 2002 in the EBS 5-
year Strategic Plan. It may simply reflect confusion in the minds of parents and 
guardians about where the best “deal” lies, differences in the number of options 
available to them in rural and urban areas, or it may reflect a general heightening in the 
desire for parents and guardians to invest in the education of their children. 
In addition, the program is attracting adult learners as can be witnessed from the 
establishment of centers which service adult learners in Monze, Southern province 
(Cheelo B), and in Chongwe, Lusaka rural (Chimbwete). In both cases, the mentors 
reported that the adults came to them and asked, “What about us? We want to learn too.” 
So, in addition to an IRI class for young learners that had already been established, 
adult classes were born. Observations during monitoring visits indicate that the adults, 
“Most of the children transferred to regular schools and community schools. There were only two 
remaining from the previous enrollment.”
Report of testing held at Garden Center by Teza Nakazwe, August, 2001
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 10
including mothers with babies, are just as eager to learn and participate even in 
activities intended for young learners. The findings of this study also indicate there is 
an increase in the number of learners who are still minors but are beyond the official 
age for lower primary (11- 17 year olds). This group constitutes 20.8 percent of those 
enrolled.
The IRI program seems to be filling yet another need: that of exposing children that are 
not yet of school-going age to organized learning. Reports received from centers 
indicate that there are younger children who attend with their older siblings who 
impact the program in various ways. 
While it cannot be argued that younger children are demanding education, it is clear 
that it is important for the older siblings, who are most likely the caregivers, to attend 
the radio learning centers. Much as it would undesirable to have learners miss 
attending because of their care-giving responsibilities, having learners who should 
belong to early childhood programs in IRI programs might interfere with the process of 
learning, either in the form of older siblings having to attend to their younger brothers 
and sisters during the time of the broadcasts, or mentors trying to accommodate the 
needs of these younger learners. It may be necessary, therefore, to articulate a strategy 
for dealing with the issue of age limits, especially at the lower end.
3.2 Attendance
Attendance data were obtained from 10 centers in and around Lusaka, and 4 centers in 
Monze after Grade 1 broadcasts had ended. This involved a number of steps: 
requesting the mentor to provide the register, recording the highest number of children 
who enrolled in the center, recording the number enrolled at the end of Grade 1 
(August 27, 2001). A total of 1284 learners were registered in the 14 centers at the 
beginning of the broadcasts, 1155 of whom were still attending lessons at the end of 
Grade 1 broadcasts as reflected in Table 4. 
Table 4: Enrolment
Center No. enrolled in 
March, 2001
No. enrolled in 
August, 2001
Garden 135 135
Kanyama 127 99
Mimosa (Linda) 64 71
ZACEF Kamanga 140 137
It is quite difficult for the mentor to control them. Children comprise families of 2-3. 
That is, the center has 2-3 children coming from one family. The youngest child can be 
about 3-4 years old.
Report of testing held at Ngombe Center by Teza Nakazwe, August, 2001
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 11
Jack 79 72
Misisi 120 120
Bauleni 87 90
George 120 92
 Ng’ombe (ZACEF)6 80 70
Kamanga 166 122
Hanamaila 30 28
Cheelo A 43 33
Chipembele 41 38
Naluca 52 48
TOTAL 1284 1155 (90.0)
Less than 10 percent of the learners have dropped out, compared to about 20 percent in 
the piloting sample. This could be an indication that parents and other stakeholders 
were becoming more familiar with the programs and were taking them much more 
seriously after the national advertising campaigns that EBS embarked on. Learners 
typically drop out of the program when families move to a different part of town, or 
when children are relocated of children to rural areas for various reasons. Some 
learners have daily responsibilities that make it impossible for them to attend the 
lessons, typically going to the market to sell foodstuffs, or looking after younger 
siblings. Rather than drop out of the programs, some learners do attend with their 
younger siblings.
Attendance data reflecting the number of programs that each child attended out of the 
100 Grade One programs was obtained for only 10 centers, a number we feel to be too 
small to allow meaningful conclusions to be drawn. Many mentors had either lost or 
misplaced the G1 registers, or had not been taking attendance on a daily basis, and this 
is a management issue that the Ministry and communities need to address. We do 
know that economic realities affect education quite profoundly in some provinces, such 
as Luapula where children migrate to the lake areas to fish between January and July, 
and where patterns of attendance are known, appropriate management decisions can be 
made. The radio centers could migrate with the children of Luapula, for example.
3.3 Profile of the Learners
Data collected on learner characteristics included sex, age, who the guardians of the 
learners are, and whether they are orphans. We also collected data on whether or not 
they had formal schooling prior to enrolling at the center, and whether they had 
siblings in the center (see Appendix C for data collection instrument). These data were 
collected from the sample of 30 centers, 14 of which were located in an urban area, and 
16 in several rural communities. The unit of analysis was the individual learner, 1898 of 
which were sampled. Generalizations on the profile of the learners are being made 
 6 This center is supported by a local NGO (ZACEF) which means that mentor is paid (unlike most others). Test administrator found out that 
children had missed almost a whole month of broadcasts in July when mentor went for voter registration activities. ZACEF appointed another 
mentor who abandoned the center after a few lessons.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 12
from the sample data to the population. Also, further investigations on learning 
patterns as a function of some of these variables will be investigated.
3.3.1 Sex and age of learners
Of the learners enrolled at the learning centers that were sampled, males are estimated 
to be 50.0 percent, the same as females. The implication of this finding is that even if the 
population may not replicate this breakdown in sex, no significant differences can be 
expected in the population parameters. The urban enrolment reflected slightly more 
females than males (51.2 percent compared to 48.8 percent), while the rural enrolment 
was the reverse (48.6 percent female, and 51.4 percent males). 
The mean age of the learners was 9.7 years, whereas the median age was 7 years old 
(22.2 percent of the learners). The wide age difference between the mean and the 
median is explained by the fact that there is a wide variation between the youngest 
learners (5 years old), and the oldest (51 years old). There were no differences when age 
was disaggregated by sex. With the mean age in urban areas being 8.9, compared to 10.6 
in the rural areas, however, a significant7 difference was observed when age was 
disaggregated by locality.
With the introduction of IRI on a nationwide scale, a wider variation on the age of the 
learners was expected. Compared to the 2000 pilot sample, 21.1 percent of the learners 
in the 2001 were still minors (11- 17 year olds), even though they were beyond the 
official age for lower primary. Some centers, typically rural centers, had more than 20 
percent of their learners being 13 years or older. For instance, at rural a center in 
Chimbwete (Chongwe, Lusaka), the mean age was 12.8 years, with 21.2 percent of those 
enrolled being adults whose ages ranged from 18 years to 49 years. In Monze district 
there was a center for adult learners, Cheelo Adult, which had learners of ages ranging 
from 17 to 51 years, with the mean age being 30.0 years old.
3.3.2 Parents and Guardians
1881 of the 1898 learners reported on this information. In that number, 66.8 percent of 
the children had both their parents living, while 33.2 percent were orphans as indicated 
in Table 2. Double orphans constitute 10.8 percent, while 22.5 percent learners had only 
one living parent. Compared to the pilot sample, the percentage of orphans is up from 
27.7 in 2000 to 33.2 percent. However, we believe orphans may be under-reported in the 
present investigation as they were in the previous evaluation. This derives from the fact 
that the official definition for orphans may not be consistent with the cultural definition, 
where children cease to be orphans by virtue of the fact that they have been adopted 
into the family of a close relative, albeit unofficially. Mentors have confirmed before, 
from personal knowledge of the children in their communities, that significantly more 
children in their centers were orphans, even though the adult guardians did not want to 
disclose that information about their protégées. 
 7 t = 7.60, p = .00
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 13
Table 5: Distribution of orphans vs. non-orphans by locality
Living Parents All learners 
(%)
Urban 
areas (%)
Rural
areas (%)
Both parents alive 1256 (66.8) 679 (67.2) 577 (66.2)
Only mother alive 321 (17.1) 189 (18.7) 132 (15.2)
Only father alive 101 (5.4) 40 (4.0) 61 (7.0)
No living parent 203 (10.8) 102 (10.1) 101 (11.6)
Orphans 625 (33.2) 331 (32.8) 294 (33.8)
Table 5 indicates also that there are slightly more orphans in the rural areas (33.8 
percent) compared to urban areas (32.7 percent). This pattern is a reversal from the pilot 
data (2000) where there were more orphans in the urban areas that in the rural areas. It 
was also expected that there would be comparatively more orphans in the urban areas, 
particularly in the case of HIV/AIDS orphans, because urban centers in Zambia have a 
higher prevalence of HIV/AIDS and HIV-related deaths. There is evidence that 
increasingly, orphaned children are ending up with grandparents or other relatives in 
the rural areas. The tracking of orphaned children and the reasons they are increasing in 
the rural areas is important and should be considered in the proposed 2002 study to 
measure the impact of IRI.
The information in Table 6 indicates who the guardians of the learners are. About 48 
percent of the learners lived with both their parents. A significant number of learners 
(21.3 percent) lived either with their grandparents or other relatives. Guardianship may 
also shed more light on the issue of who the orphans are, especially in the case where 
the primary guardians are grandparents who are also acting as adoptive parents. In 
cases where children were double orphans, the grandparents were the most popular 
choice among the relatives (11.3 percent). 
Table 6: Guardians of the learners by locality 
Relationship to 
learners
All learners 
(%)
Urban areas 
(%)
Rural 
Areas (%)
Both parents 866 (47.4) 485 (48.0) 381 (46.7)
Mother 362 (19.8) 252 (25.0) 110 (13.5)
Grandparent(s) 206 (11.3) 70 (6.9) 136 (16.7)
Father 152 (8.3) 75 (7.4) 77 (9.4)
Other relatives 183 (9.6) 120 (12.0) 63 (7.7)
Non-relatives 57 (3.1) 8 (0.8) 49 (6.0)
Total 1826 (100.0) 1010 (100.0) 816 (100.0)
There were significant differences in guardianship in the urban and rural areas. Many 
more children lived with either or both their birth parents in the urban areas than in the 
rural areas (80.3 percent, and 69.6 percent, respectively). The reverse was true for 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 14
grandparents who lived with their grandchildren (16.7 percent in rural areas, and only 
6.9 percent in urban areas). The reason for this may partly be that grandparents 
typically live in the rural areas. Also, for every child in the urban areas that is left in the 
guardianship of total strangers, there were 7 such children in the rural areas. 
Where both parents were alive, 90.5 percent of the learners lived with either or both 
parents. When single parent were separated and considered on their own, 70.4 percent 
of single mothers lived with their children, compared to only 41.9 percent single fathers. 
A significant number of children with single father parents, 22.6 percent, had their 
grandparents as primary guardians. From a psychosocial point of view, it is highly 
desirable that parents should take primary for responsibility for guardianship of their 
children. Parental guardianship is also known to have positive benefits in children’s 
school performance. However, there could be cultural and economic factors that make it 
equally acceptable that children remain under the guardianship of their grandparents.
3.3.3 Prior school attendance
17.3 percent of the learners had some schooling experience prior to enrolling in the 
learning centers, and the percentages are similar for female and male learners. Unlike in 
the previous evaluation, prior school attendance is higher in rural areas (21.9 percent) 
than urban areas (13.7 percent). A possible explanation for this may be that when 
people move to the rural areas they lose opportunities to continue with their education. 
This finding partly explains why the rural communities have been more proactive in 
setting up centers for the IRI program).
3.4 Performance in basic literacy and numeracy
Grade 1 learners attained a mean score 27.9 out of 44 possible points on the whole test 
(a composite score of the numeracy and language score) , a mean of 63.4 percent. The 
lowest score was 0, and the highest, 44. The mean score for the numeracy component 
was 14.3 out of 20 (71.5 percent), while the mean for the language component was 13.6 
out of 24 possible points (56.6 percent). There were no differences when the scores were 
disaggregated by sex.
Table 7: Test Means for Literacy, Numeracy and the Overall Test
Content N Minimum Maximum Mean Std. Dev.
Math 405 .00 20.00 14.3 4.9
English 405 .00 24.00 13.6 5.5
Composite 405 .00 44.00 27.9 9.7
Learners performed better on the numeracy component than on literacy. This is to be 
expected, particularly that they get the opportunity to practice numeracy skills in their 
daily lives. While it is true that learners were generally weaker in English language 
skills, part of the reason for low performance on English language items could be that 
some of the questions were not presented clearly enough for student to comprehend the 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 15
task. These are tasks that do not lend themselves to paper-and pencil testing, and are 
easily observed in situations where learners respond naturally to situations that arise in 
the learning environment. Table 8 below presents information on the percentage of 
learners who mastered each item, with items grouped under the different domain. It 
also provides, for comparison, the subtest mean for each domain (skill area).
In the literacy component, learners had the highest competence in recalling names of 
colors and items of clothing (at 77.5 percent). The least performance was registered in 
the skill area of production of language, where the mean for the domain was 46.7 
percent. In particular, most learners were not able to produce language that expresses 
personal preference (Item 17, with only 17.5 percent masters, and less than 50 percent 
combining masters and partial masters). Most test administrators expressed that they 
had difficulties in presenting the task to students on this particular item. 
Learners also had difficulties in reading sight words (Item 21 with 18.5 percent mastery) 
and spelling (Item 22, at 22.5 percent mastery). Even though items on reading and 
spelling were included in the test, reading is not a terminal skill at Grade 1, hence it was 
expected that most learners would have difficulties with reading. Also in the case of IRI 
instruction, learners have very little opportunity to practice this skill in that they do not 
have reading material at their disposal, They generally only read words written on the
board by the mentors, hence the low performance on this skill. It is a matter of 
considerable importance in the design of the IRI program how far we can continue with 
this method. For now, information on reading is useful in that gains in Grade 2 will be
measured against the present reading level.
Table 8: Mastery by item and (skill area) domain
Skill Area Items Non￾Masters
Partial 
Masters
Masters Domain 
Mean
Production of language Item 16 18.3 16.5 65.2
Item 17 56.0 26.4 17.5
Item 18 40.7 25.4 33.8 5.6 of 12
Item 20 26.7 34.1 39.3 (46.7 percent)
Item 21 47.7 33.8 18.5
Item 22 50.4 27.2 22.5
Item 13 28.0 18.3 53.7
Item 14 21.5 36.8 41.7 5.0 of 8
Item 15 21.2 16.5 65.2 (62.5 percent)
Comprehension of language
Item 19 35.3 23.0 41.7
Recall of names Item 11 10.4 25.2 64.4 3.1 of 4
Item 12 16.0 17.8 66.2 (77.5 percent)
Counting and writing numbers Item 01 5.7 13.8 80.5
Item 02 18.8 16.5 64.7 5.9 of 8
Item 03 21.2 24.9 53.8 (73.8 percent)
Item 04 23.7 16.3 60.0
Shapes Item 09 15.1 36.8 48.1 2.7 of 4
Item 10 14.6 30.9 54.6 (67.5 percent)
Number operation Item 05 24.0 27.7 48.4
Item 06 22.2 13.8 64.0 5.7 of 8
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 16
Item 07 19.8 14.8 65.4 (71.3 percent)
Item 08 19.8 4.0 76.3
All skill areas of the numeracy component were performed at a mastery level of over 50 
percent as indicated by the domain means. Given a range of between 10 and 99, more 
than 80 percent of the learners were able to count in 1s (Item 1). A significant number 
was able to write the two digit numbers in the same range (Item 2). Counting in 10s was 
a little easier than counting in twos for most learners (Items 4 and 3, respectively). On 
the whole, learners performed very well on numeracy, but only reasonably well on 
literacy. Performance differences by sex were negligible, a known strength of IRI 
programs worldwide that has been borne out for two consecutive years in Zambia.
3.4.1 Comparison with the previous Grade 1 tests
Table 9 below includes the domain description for the Grade 1 test in 2000. As can be 
expected in a test of basic literacy and numeracy at the lowest level, the internal 
structure of tests (See Table 3 on Section 2.2.4 for Grade 1, 2001 test) is the same. The test 
domain in both instances covers recalling of numbers names, counting, simple 
comprehension and production of language, and writing of letters and single digits. 
These are presented in similar proportions in both tests. Test results show a similarity 
between the patterns of the current test scores and both sittings of the previous test 
scores. This feature speaks to the internal consistency of the test, which in this case is 
very high.
Table 9: Skill areas and corresponding test items for Grade 1 Test, 2000
Skill Area Intended Learning Target Test Items
1. Simple comprehension of language 11, 14, 15, 16, 18, 20
2. Production of language8 1, 2, 12, 13, 19
3. Writing letters 17 
Language
3. Recalling names 7, 
1. Counting 3, 4,5
2. Writing numbers 6
3. Recalling numbers 10
Numeracy
4. Adding and subtracting 8, 9
Table 10 below includes, for comparison, performance scores for the current test 
administered after 100 programs, and those of the previous cycle of testing, 
administered midway into the broadcasts in the form of a pretest in July, 2000, and a 
posttest using the same test instrument in October, 2000. 
Table 10: Test Means for Literacy, Numeracy and the Overall Test
 8 All items under this learning target are subsumed under comprehension of language.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 17
Test Performance, August 2001
Content N Minimum Maximum Mean Std.Dev.
Math 405 .00 20.00 14.3 4.9
English 405 .00 24.00 13.6 5.5
Composite 405 00 44.00 27.9 9.7
Test Performance, October 2000
Content N Minimum Maximum Mean Std.Dev.
Math 143 5.00 14.00 12.6 1.8
English 143 6.00 26.00 19.8 4.8
Composite 143 11.00 40.00 32.3 6.2
Test Performance, July 2000
Content N Minimum Maximum Mean Std.Dev.
Math 190 .00 14.00 10.2 3.0
English 190 .00 26.00 14.2 5.8
Composite 190 4.00 40.00 24.4 7.8
Even though the scores show internal consistency, performance on the previous tests 
was far superior to performance on the current test. This is partly due to the fact that 
there were a lot more centers to deal with in the nation-wide Grade 1 broadcasts, which 
means that standardization of procedures, and supervision of the learning center was, 
at best, minimal. Differences between the test scores from one year to the next are to be 
expected, at least in the formative years of the program when systems have not yet 
stabilized. There should be benchmarking against which performance from year to year 
can be compared. To this end, cut-off points that define different levels of performance 
should be defined. A psychometric approach that is a hybrid of mentors and teachers’ 
judgments on individual item content, and judgments on examinees test scores would 
be preferable. More experienced teachers from community schools which tune in to the 
IRI programs would be invaluable to this exercise since judgments of mentors alone 
could be challenged on the basis that they not qualified teachers. 
3.4.2. Performance by locality and by center
Differences in performance both by locality and by center were detected and presented 
as qualitative data by administrators throughout the testing period. IRI staff has also 
noted these differences in their routine visits to the learning centers. At a general level, a 
conclusion can be made that urban centers were performing better that rural centers as 
shown in Figure 1, and was borne out by testing data. The overall mean for the 
composite test was 27.9 of 44.The mean for urban centers was 29.6, compared to 25.8 in 
the rural areas.
Figure 1: Centers, by locality and name, performing above and below the mean
Urban Centers Above the Mean Rural Centers Above the Mean
1. Bauleni 7. Mapepe
2. Chainda 8. Misisi
3. Chipata
4. Garden
1. Cheelo Adult
2. Chimbwete
3. Hanamaila
4. Nangombe
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 18
5. Kamanga
6. Kanyama
Urban Centers Below the Mean Rural Centers Below the Mean
1. George
2. Jack
3. Kabwata
4. Kamanga-ZACEF
5. Mimosa
6. Ngombe-ZACEF
1. Chanshaya 7. Kapovu
2. Chilando 8. Mwachilele
3. Chilyabwale 9. Mwanza
4. Chipembete 10. Naluca
5. Chipungu 11. Namakube
6. Kanakantapa 12. Nyangwena
Twelve centers9, those in the first and second quadrants, performed above that mean. 
The top two centers were Nang’ombe in Chongwe, and Cheelo Adult in Monze. The 
mean scores for the centers in this group ranged from 36.8 (83.6 percent) to 28.6. Even 
though there were differences in performance by locality, high performance and 
achievement was attributed less to the locality factor than center specific attributes and 
occurrences. For instance, for the best performing center, Nangombe Center, the test 
administrator had this to say about the center and the mentor: 
Generally, centers in this part of Chongwe, where there are fewer government schools, 
were noted for high motivation and higher than average performance. A possible 
explanation could be that IRI centers are the only means of providing education to 
children for most communities, hence these programs are taken more seriously than in 
places where people have the community schools option to fall back on. 
Cheelo Adult was reported to be a center that received a lot of support from the 
community and the area Catholic Church. High performance can be attributed to the 
involvement of the Ministry of Education In-service training provider, as well as the 
mentoring system that is in place. 
 
9 
p 
learners in the center, whose data is aggregated as a group. 
I am very impressed with this Center because after the radio lesson the mentor takes his time 
to go over the lesson and the previous day’s work. They spend up to one and a half hours in 
post broadcast activities….This mentor is very committed to his work. I hope the community 
is appreciating his work. The work at this Center is going on smoothly.
Report of testing held at Nangombe Center by F. Mubiana, August, 2001
Mrs. Lumang’ombe, the Monze DIP, works closely with Fr. Tadeusz, who has spearheaded the IRI 
effort in Chikuni area. They have a total of 19 centers. Mentors are chosen and supported by 
communities. With the help of the Choma DIP and headteacher from a nearby school, they visit 
and monitor mentors regularly. They also bring them to a central place (parish) for a monthly 
meeting. Halfway through Grade 1, they held a weeklong refresher training course for the mentors. 
Theirs is by far the best monitoring system developed so far and in many ways an ideal situation.
Notes from IRI Consultant about Monze centers, October, 2001
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 19
A peculiar feature of Cheelo B center is that it enrolls parents of children that attend at a 
neighboring center, hence high performance could also be attributed to the age of the 
learners, as will be discussed in the next section.
Eighteen centers performed below the mean. The least performing centers were 
Chilando and Mwanza, in the eastern Province, with overall means of 6.0 and 10.7, 
respectively. The sample sizes that were tested in these centers were too small for any 
meaningful statistical conclusions. However, feedback from the test administrator has 
shed considerable light on the difficulties in the learning environment, the setbacks in 
the mentoring process, and the childrens’ interaction with testing situation and the test. 
In addition to there being no reliable radios that the communities can use, the mentors 
do not seem to have received adequate training:
In fact, for factors that militated against good performance in the centers that he is 
working with, Mr. Nielsen cited features that were the strengths of the centers that 
performed well, such as Cheelo Adult and Nangombe. He also cited the age of children, 
an issue which will be dealt with in more detail in the next section.
3.4.3. Performance by age
The ages of learners in this sample ranged between 5 and 51 years old. However, with 
the mean age in urban areas being 8.9, compared to 10.6 in the rural areas, a significant10
difference was observed when age was disaggregated by locality. Significant differences 
were also observed when the age categories were used to disaggregate performance as 
reflected in Table 8. 
The first age category was that of children who are yet to reach the official school-going 
age and could be considered too young to be in IRI centers. The second age category of 
7 to 13 year olds captured those who are the official age for primary education. The 
third category, that of 14 to 17 year olds were minors who would, under normal 
circumstances, be pursuing secondary education. The last age category was that of 
 10 t = 3.89, p = .00
…mentors only reinforce language with rote methodologies, parents cannot reinforce English in 
the home, students are too young often to understand, and radios are poor so that broadcasts are 
unclear… The mentors here could really benefit from some sort of training designed to make them 
more effective instructors within the Taonga framework.
Testing Report for the Eastern Province, Hunter Nielsen, August 2001
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 20
individuals who are officially adults. 
Table 11: Performance by age category
Age 
Categories
Test N Minimum Maximum Mean Std. 
Deviation
6 years and under Numeracy 26 0 19.0 8.7 5.1
Literacy 26 .0 18.0 11.1 5.2
Composite 26 0 33.0 19.7 9.2
7 - 13 year olds Numeracy 280 0 20.0 14.0 4.8
Literacy 280 .0 24.0 13.4 5.8
Composite 280 1.0 44.0 27.4 9.8
14 – 17 year olds Numeracy 23 12 20 18.0 2.7
Literacy 23 7.0 24.0 16.7 4.1
Composite 23 20.0 44.0 34.7 5.8
18 years and older Numeracy 13 18.0 20.0 19.4 .87
Literacy 13 8.0 24.0 17.0 4.9
Composite 13 28.0 44.0 36.4 5.2
The results showed an increase in performance as age increased, with all learners in the 
category of 18 year olds and older missing only one item, or getting a perfect score in 
the numeracy subtest. The mean differences were significant11. The standard deviations 
were also increasingly narrower, which means that with increasing age, learners were 
becoming more similarly competent on the skills that were tested. Another conclusion 
that can be inferred from the fact that standard deviations were narrower for the 
numeracy subtest, in comparison to those of literacy skills subtest within each age 
group, is that examinees were more variant in literacy than in numeracy. The data 
seems to suggest that age has to be taken into consideration in IRI if returns are to be 
maximized. This finding will be important for future enrolment policy and practice as 
more IRI centers are organized.
 11 Composite: F = 14.3, p = 00; Numeracy: F = 23.5, p =.00; Literacy: F=5.8, p = .01, all at 
df = 341
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Rev 
3.5 Impression on Effectiveness of IRI 
IRI is becoming an effective alternative way of providing basic education in Zambia. 
Unlike in 2000 when the evaluation study included focus groups of mentors, evidence 
on the effectiveness of IRI was mostly gathered by MOE, EBS and EDC personnel who 
were closely associated with the program and by various partners. Reports from 
mentors, partners supporting the programs (e.g. churches and PCVs), test 
administrators, etc. have confirmed that IRI is becoming an effective and workable 
community alternative in providing basic literacy and numeracy skills, and directing 
attitude learning towards some intended goals. 
Other impressions were gathered from the high level of commitment to IRI work that 
has been exhibited by some of the mentors and communities as can be seen in the notes 
below which were provided by mentors in the Chikuni area during the mentor 
refresher course in July 2001, and the success stories in Appendix I. 
 
 
 
 
The community has managed to buy two radios, two chalkboards and some pieces of chalk … 
the other problem is that the community has failed to give us support over the fact that thy 
are the ones who are looking after the orphans. Therefore they cannot manage to support us 
and support the orphans at the same time.
Chrispin Simulimya Keembe J. C
Cheelo Centre (A and B), Monze
We have 38 pupils. The headmaster provides us with chalk. We are using a church as a 
classroom. The committee is busy working for this school so that it does not stop.
We do not have a radio. For now we are using a radio with batteries. The committee is able to 
buy batteries for us. For now they are donating maize so that we buy a radio.
Mavis Mudenda,
Kayola Centre, Monze
The community bought the radio, chalkboard and also some chalk. There are at least a good 
number of children who are performing very well, i.e about 25 of 41 children we have are able to 
read the words we taught them. For this reason many of the people are interested in Taonga 
more than the regular school … The center is facing a problem with the children who are 
orphans … some are staying with old people who cannot afford to support the children’s need as 
well as the mentors’.
Mentors, Center Unknown
Monze
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
 
 
We have a total number of 44 children. From the time we started up to date the attendance has 
been very encouraging. We have no problem with the children but on the side of the parents there 
is a bit of doubt because to them it is a new type of learning. But as time goes by we expect some 
changes …We still have a problem of purchasing the radio. We are still depending on a borrowed 
radio and we thank the person assisting us with a radio. Nevertheless parents have so far 
donated two bags and are still donating. Hopefully by the end of this month everything will be 
fine on the side of the radio. On the side of batteries we teachers are purchasing batteries for the 
benefit of the children
Mentors,
Kasilgili Taonga Market School
Our committee is trying by all means to help us mentors e.g organising even if we get nothing. 
Since we started we haven’t been paid. The committee is encouraging us to work accordingly. 
There is a problem at our center. We’ve got children who cannot afford to attend the original 
school, which they only pay k4500.00 per year so that’s why even paying for the radio is a 
problem they can’t manage.
Mentors,
Namakube Centre, Monze
At Sintemba Center we have bought the radio. We made the wall blackboard, chalk, local material 
such as stones, sticks etc. The community were able to provide a learning place but the community 
is not able to assist the mentors in any way.
The condition of the radio is good because it is still working … The pupils are in three groups, the 
performance of the first group is very good ….
Webster Hamonze,
 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 23
4.0 LESSONS AND RECOMMENDATIONS
4.1 Lessons
This evaluation, the second to be conducted on the IRI project since its inception two 
years ago, has revealed that IRI is fast becoming an invaluable feature of the Zambian 
education system. The preparation and hard work that everyone has put into the IRI 
project this year; preparing for nation-wide airing of the Grade 1 programs, developing 
and recording Grade 2 programs, visiting learning centers in Lusaka and rural/remote 
areas, as well as preparing for and collecting data for purposes of monitoring and 
evaluation, underscores the serious with which this mandate has been received and is 
being carried out. A few themes have recurred from the earlier evaluation.
1. The demand for IRI is still increasing. The program is also growing popular as 
can be gleaned from new groups of learners that are becoming interested in it.
2. It has been possible to achieve a lot with very little. Most centers survive with 
one radio each. In addition, several of the mentors who were in the piloting 
group have continued to help the children, mostly working for no pay, operating 
under very difficult conditions.
3. Mentor support and coaching as well as high levels of motivation seem to be 
making the difference between centers that record high learning gains and those 
that are not performing as well. 
4. There is a benefit in continuing to actively seek churches, NGOs and other 
partners in the development and delivery of IRI in Zambia
4.2 Recommendations
4.2.1 Demand for IRI
The demand for IRI is increasing. Many new centers were organized nation-wide 
throughout the period of the airing of Grade 1 programs. Some communities started 
new centers, but managed to organize themselves well enough to receive lessons well 
into the period Grade 1 broadcasts. Many of these centers missed a number of 
broadcasts at the beginning of the Grade. In some areas parents and guardians are 
pulling children from existing schools and registering them for IRI instead. In this 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 24
regard, IRI is also presenting an alternative form of education, an alternative that makes 
sense to most parents in that the students in IRI will be able to go through two grades in 
the same year (Grades One to Four). Also, IRI is attracting a wider range of clientele 
than was originally intended, a feature that speaks for the demand of the program. An 
issue that arises from this demand is that some of the newly established centers may 
begin operating when they are too ill-prepared to undertake any meaningful learning.
Recommendation 1: MOE should articulate a simple regulatory IRI strategy that 
provides the minimum requirements and criteria that should be met for a community to 
organize a formal learning center(have a radio, recruit a mentor, have shelter, etc). This 
strategy should not be inhibiting to communities who wish to establish centers, though.
4.2.2 Attendance
About 90 percent of the learners that enrolled at the beginning of Grade 1 completed the 
grade in centres where attendance information was available. The majority of learners 
attended regularly. However, attendance information was incomplete in most centers. 
It is clear that some mentors choose not to spend time on taking attendance, while 
others misplaced or lost the loose sheets on which they recorded attendance. It is 
possible that mentors perceive attendance taking to be a tedious, unnecessary and time 
consuming administrative duty.
Recommendation 2: MOE should articulate a mentor-friendly strategy for tracking 
daily attendance of learners. This strategy should include providing an official register 
that can be collected at the end of each grade. 
4.2.3 Learner characteristics
IRI attracts learners of various backgrounds. The majority of the children fit the 
definition of “out-of-school” children, while a few are being described by their parents 
as being out-of-school simply to be accepted at IRI centers. The sex breakdown of the 
learners is at the ratio of about 1:1. The age range has increased to include the very 
young and the very old, and this presents new challenges to the program. For most 
children, both parents are alive, which means they are attending IRI and not regular 
school due to hardships that their families may be experiencing, other than the fact that 
they do not have parents who can provide for them. Also, the majority of the learners 
live either with their parents, or grandparents, while only a few are in the care of non￾relatives. It is anticipated that as the number of children who are drawn to learning 
centers increases, centers are going to be faced with situations where they are unable to 
admit every prospective learner.
Recommendation 3: A deliberate attempt should be made to delineate the profile of an 
IRI learner, the ideal “out-of-school” learner for which the program was initially 
designed, such that the IRI service should be made available to those who need it the 
most. While the programme was never intended for older learners, their participation 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 25
yielded positive results. However, care should be taken not to mix groups of very old with 
younger learners as the older group is displaying a different kind of motivation and 
attitude allowing them to master concepts much faster, which could be a source of 
discouragement to younger learners. Also, monitors and supporters of the program 
should suggest some alternatives to mentors for the underage children who are showing 
up in the centers and are clearly not ready for formal schooling.
4.2.4 Student Learning
Learners exhibit more mastery in the area of numeracy than in literacy skills. Also, 
older learners are performing better than younger learners, their main strength being 
mastery of literacy skills. Students in the pilot centers achieved more than students in 
the centers nation-wide (80.1 percent on the composite test in 2000, compared to 63.4 
this year), as was expected because the mentor training and supervision was more 
rigorous in the smaller number of centers. These differences need to be explained since, 
general statements on learning will cease to be meaningful as the number of examinees 
increases. A number of actions may need to be considered to maximize learning at the 
centers.
Recommendation 4: EBS should, with the help of ECZ and/or an educational 
assessment specialist, design a standard setting exercise that would determine cut-off 
scores for different levels of mastery (and/or proficiency) for purposes of providing 
consistent descriptions of learning achievement, as well as for year to year comparisons.
As a result of the interactions that are taking place in the centers, learning is taking 
place in all centers, not only concept and skill learning, but attitudinal learning as well. 
However, learners at some centers are performing appreciably better than those at other 
centers. It has been suggested that some of the variance for performance at learning 
centers might be as a result of mentor characteristics, the assistance that mentors receive 
from their communities or from MOE, and mentor motivation.
Recommendation 5: A deliberate attempt should be made to delineate the profile of an 
ideal mentor for the IRI program, including his/her training and supervision needs (e.g. 
formal IRI training, coaching during monitoring visits, as well as in-service training 
where possible), and a mechanism to improve the chances that mentors are retained. 
4.2.5 Effectiveness of IRI
Evidence for the effectiveness of IRI can be gleaned partly from the fact that there are 
many learners who are transferring from regular schools to IRI centers as well as from 
the responses from communities and partners. Also, the program is becoming popular 
with the general public in Zambia. The evaluator enquired about the program 
randomly from a number of Zambian colleagues (in Botswana), some of whom were 
aware of the program and the need that it is attempting to address in the Zambian 
education system. However, systematic and comprehensive information about the 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 26
workings of an IRI center is needed for decision-making about many aspects of the 
program.
Recommendation 6: MOE should train mentors on how to keep a journal of critical 
incidents (aspects of the program that work well, and those that do not, as well certain 
occurrences that demonstrate that learning has taken place), to be used in monitoring, 
formative, and summative evaluations of the program.
4.2.6 Cost 
The cost of supporting IRI learning centers has grown tremendously with nation-wide 
implementation. Some of the more expensive activities have been the assessment and 
evaluation exercises that have been carried out. It is possible to cut down on cost by 
building into the program, data collection mechanisms such as the mentors’ journals 
mentioned above, which data can then form the basis for the qualitative inputs into the 
evaluation study. Also, with an increasing amount of data that should be kept for 
monitoring purposes, EBS record keeping practices need to be improved.
Recommendation 7: MOE should develop a cost-effective and sustainable monitoring 
and evaluation framework.
Recommendation 8: A database of center, student and mentor profile will be more 
desirable and efficient than the current spread sheet that is used. This will allow greater 
manipulation of data (e.g. tracking performance throughout the entire primary cycle). 
Ideally, this would be institutionalized into the operations of EBS and/or the MOE’s 
Planning Unit.
The MOE’s effort in providing alternatives to reaching disadvantaged learners is to be 
commended. Now that the program has gone nationwide, and there will soon be 
several cohorts of the program, policy decisions on the issues outlined above will need 
to be considered to ensure that IRI maintains desirable educational standards and that 
IRI remains accessible to disadvantaged learners in Zambia. 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 27
5.0 APPENDICES
Appendix A: Data collection sheet for students’ profile
Appendix B: Content Map for Grade 1 English
Appendix C: Content Map for Grade 1 Mathematics
Appendix D: Test Blueprint for Grade 1 English
Appendix E: Test Blueprint for Grade 1 Mathematics
Appendix F: Test Plan for Numeracy and English Literacy for Grade 1
Appendix G: Grade 1 Numeracy and English Literacy Test
Appendix H: Training Notes for the Administrators
Appendix H: Training Notes for the Administrators
Appendix I: IRI Success Stories

Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 29
Appendix A - Data collection sheet for students’ profile
Center:
C# ID Name Age Sex SchAtt Reside Live With? Sibling? Parent Alive?
__________________________________________
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 30
Appendix B - Content Mapping for Grade 1 - English
Terminal Objectives Listening and
Speaking
Reading Phonics Writing Spelling
1. Understand and respond to teacher’s 
instructions
7.1
2. Give instructions 7.2 7.3
3. Use language in social interaction, 
classroom organization, correction, and 
simple explanations
1.1
4.1 4.2
8.1 – 8.5
4. Answer questions
5. Make statements 2.4
5.1 5.2 5.4 5.6 – 5.10
6.2 – 6.4
6. Ask questions 3.1 – 3.16
5.3 5.5
7. Ask for information 6.1
8.8
8. Use simple descriptive language 1.2 1.4 – 1.9
2.1 – 2.3 2.5 – 2.8 
9. Listen and demonstrate 
understanding
10. Use language simple in interactive 
activities 
6.2 – 6.4
9.1 – 9.9 10.1 – 10.7
11. Demonstrate understanding
12. Pronounce words 11.1
13.1 – 13.4
C1
C2
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 31
Appendix C - Content Mapping for Grade 1 - Mathematics
Topics /Terminal Objectives/ Naming Classifying 
objects and numbers 
according to given 
condition
Demonstrate 
understanding of concept 
of number and 
numeration
Perform basic operations 
on numbers and other 
measures
Sets 1.1-1.2 1.3–1.5
Numbers 2.1; 3.1 2.2–2.3 2.4
Completing addition and 
subtraction of number sentences up 
to 100
4.1 4.1; 5.1
Shopping Activities 6.1
Counting 7.1; 10.1 10.1
Telling time 8.1
Comparisons 9.1 – 9.2
Shapes 11.1 11.2
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 32
Appendix D - Test Blueprint for Grade 1 - English
Topics Knowledge Comprehension Application
1. Identifying and classifying 1.1 – 1.2 1.3 - 1.19 1.20
2. Describing narrating and reporting 2.1 – 2.2 2.4 2.3 2.5 – 2.7
3. Finding out (inquiring) 3.1 – 3.16
4. Denying, confirming and correcting 4.1 – 4.2
5. Expressing and finding out 
intellectual attitudes
5.1 – 5.4 5.6 5.8 – 5.10 5.5 5.7 
6. Expression personal meaning 6.2 6.1
7. Getting things done 7.1 – 7.3
8. Using language socially 8.1 8.3 8.4 8.2 8.5
9. Expressing concepts and 
relationships
9.1 – 9.9
10. Talking about quality and quantity 10.1 10.2 10.6 – 10.7 10.3 – 10.5
11. Learning about language 11.1 – 11.3
12. Speechwork and phonics 13.1 – 13.2 13.4 13.3 13.5C1-C2 (P)
13. Reading 1 2 – 5
14. Writing and spelling 1 – 7 (W)
1 (SP)
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 33
Appendix E - Test Blueprint for Grade 1 - Mathematics
Terminal Objectives/ Topics Knowledge Comprehension Application
1. Sets 1.1 – 1.5
2. Numbers 2.1 – 2.3 2.4
3. Naming numbers (0 – 10) 3.1
4. Completing addition and 
subtraction of number sentences up 
to 100
4.1
5. Addition and subtraction 5.1
6. Shopping Activities 6.1
7. Counting in tens and ones 7.1
8. Telling time 8.1
9. Comparisons 9.1 – 9.2
10. Counting in twos up to 100 10.1
11. Shapes 11.1 11.2
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 34
Appendix F - Test Plan: Basic Literacy and Numeracy
Skill Area Knowledge Comprehension Application
Number of 
items
Recall of names 1.1–1.2 1.4 - 1.9; 1.10-1.11; 
1.12-1.16; 1.18-1.19
1.17; 1.20 3 (2K, 1C)
Production of Language 
(Speaking)
2.1-2.2; 2.4; 3.1-3.16; 8.1; 
8.3; 8.4;10.1-10.2
2.3; 2.5-2.7; 4.1-4.2; 5.1-5.4; 5.8-
5.10; 7.1-7.3; 8.2; 8.5; 10.6-
10.7;13.1-13.2; 13.4
5.5; 5.7; 6.1; 9.1-9.3; 
9.4-9.9; 10.3-10.5; 
11.1-11.3;13.3;13.5;
4 (2C, 2A)
Production of Language
(Reading)
1 2; 3; 4; 5 2 (1K, 1C)
Production of Language 
(Writing)
1; 2; 3; 4; 5; 6; 7 1 (A)
Comprehension of Language 6.2; 10.6; 10.7 2 (C)
Counting 7.1; 10.1 3 (C)
Number Operations (addition, 
subtraction and multiplication)
4.1; 5.1; 6.1 3 (A)
Classification and Comparisons 3.1; 11.1 1.1-1.5; 2.1; 2.2; 2.3; 9.1-9.2 2.4; 11.2 3 (1K, 1C, 1A)
Explanatory Notes: 
1. The skills area of this test plan (first column) presents a summary of terminal objectives. In the first four skills the 13 terminal objectives of 
Grade 1 English are restated in summary form. The last three skills (in italics) reflect the mathematics terminal objectives that are suited 
for Grade 1. The entries in the body of the test plan are instructional objectives from both the Mathematics and English syllabi.
2. The last column of the test plan stipulates the number of questions to be tested per skill, and the different levels (using Bloom’s taxonomy) 
at which the items for each skill will be tested. This is a 22-item test consisting of 12 literacy items and 10 numeracy items. There are 4 
items at the knowledge level, 13 at comprehension, and 5 at the application level.
3. Instructional objectives that are sampled for testing are presented in bold in the body of the test plan. From these, several items were set, 
The administrators will typically present one item to the learner, and a second item if the administrator is not certain about mastery/non￾mastery of the objective.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 35
Appendix G - Test of Mathematics and English Language Skills
GRADE 1
TEST ADMINISTRATION SCRIPT
Instructions: 
• Please read this information before the test and seek clarification where 
necessary.
• Everything that appears in italics in the test is for the information or 
direction of the mentor/test administrator. Words that are not in italics are 
to be read out to the learner.
• Administer the whole test (all questions) to each learner. The test is to be 
administered to one learner at a time, away from other learners. 
• The test item is to be presented to the learner a maximum of two times.
• Translation is allowed on all numeracy items except 9 and 10.
• Use the test administration grid. Record a capital C against the objective to 
indicate that the correct response was given. Write PC for a partially 
correct response, or W for a incorrect response, or a NR for non response.
 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 36
Hello, What is your name? 
How are you ____________? I am well too. How old are you? Good.
Now, I am going to ask you a number of questions. Sometimes I will ask you to just tell 
me the answer, and other times I will ask you to write your answer down, OK? Now, 
here the first question:
Question Stem Selection Instructions
1. Count from _____ 13-33; 21–41; 37–57; 41–51; 73–93; 35-
55; 70-90;29-49; 48-68; 11-31;
14-34; 25-55; 79-99; 62-82; 12-32
Select and present one of the 
items. Allow a second 
chance if necessary
2. Now write _____ 10 –99 Ask them to write any two 
digit numeral from 10-99 
(inclusive).
3. Count in twos from 2 to 10 Allow a second chance if 
necessary
4. Count in tens from 10 to 
100
Allow a second chance if 
necessary
5. ____ is the same as ___
e.g., 7+6 = 6+7 
7+6; 9+4; 6+7; 4+7; 5+5; 6+4; 9+8
9-4; 7-5; 8-3; 6-1; 9-2; 8-7; 7-3
 Ask them to complete the 
statement, presenting both 
addition and subtraction
6. Add the following 
numbers
7+6; 9+4; 6+7; 4+7; 5+5; 6+4; 9+8 Present in horizontal 
orientation on flash cards
7. Subtract ____ from ____ 9-4; 7-5; 8-3; 6-1; 9-2; 8-7; 7-3 Present in horizontal 
orientation on flash cards
8. A man has K___ and buys
____ for K___ . How much 
will you give back?
K500-K200; K700-K300; K900-K600
K400-K200; K600-K100; K200-K100
Present a story involving 
money and giving change. 
Allow a second chance if 
necessary
9. What is the name this 
shape?
Present a circle, square, 
triangle, and rectangle in 
any order
10. Now, draw a circle; a 
square; a triangle; a rectangle
Conceal the shape and ask 
learners to draw them from 
memory
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 37
I am now going to ask the remaining questions in English. Please don’t be worried 
because it is simple English, just like you always hear in the radio, when the Mrs 
Musonda tells you what to do. OK ? Here the first question:
Question Stem Selection Instructions
11. What colour is this? red, white, blue, green, black, 
brown, yellow, and orange
Present all colours in any 
order.
12. What is this?/What is _____ 
wearing?
dress; hat; shoes; socks; shirt Allow a second chance if 
necessary
13. What time of the day do you 
get up/go to bed
morning; evening; night, or any 
specific time
Allow a second chance if 
necessary
14. What can you see in this 
picture? What are they doing?
Present the picture on the 
learning center scene
15. Can you see ________?
Is __________________?
Ask several questions about the 
picture to elicit “yes” or “no” 
responses.
16. How many children can you 
see? women? radios?
Use language associated with 
counting.
17. Which animal do you 
like/do you not like?
animal, food, friends, colour Ask both the “like” and “do 
not like” questions about any 
of the items
18. Where is the book? Under, in, on top of, here, there, 
on, in front of, 
Present question involving 
prepositions or adverbs of place
19. Here is one stick, and here 
are two _____.
book; hand; pen; leg Present two words involving 
plurals. Allow a second chance 
if necessary.
20. Read these letters A – Z; a - z Present any 5 letters of 
alphabet randomly
21. Read this word 
What are these letters?
box; man; boy; pen; pot; dog; 
cat; table; shoe; hand; pen; car, 
leg; door; pot; spoon
Present two words and ask 
them to read. If not, ask them 
which letters make up the word
22. Spell _______ man; boy; pen; pot; dog; cat; 
pen; car, leg; door; pot; spoon, 
red
Present two of these. Allow a 
second chance at reading if 
necessary.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 38
Appendix H - Test Administration Training Notes
Numeracy Items (1 – 10)
The numeracy items may be presented in English, or the learners’ local language, just as it has 
been done in the teaching of these skills. Proficiency in English language should not be allowed 
to inhibit performance of the task at hand. Each of the items is described below.
Question 1
In this item, we want to test the learner’s counting skill. In order to increase the chances 
of beginning the sequences where indicated, tell the learner to repeat the number range. 
But, if the child starts at one still and covers the sequence that you have presented, give 
full credit for this item.
Question 2 
This item is meant to solicit information on whether the students can write a two digit 
numeral correctly. Some learners will reverse the numerals, for example, writing 15 
instead of 51. In that case, ask the learner to write 15. Then ask her/him again to write 
51. If he/she still reverses the numerals at this point, score the reversed item as an 
incorrect.
Question 3
This is a counting item, which also exhibits basic multiplication skill. Present the item 
using the language that is understandable to the learners. You can ask them to count in 
twos, or ask them to count “2 by 2”. Present the first 2 number of the sequence, and let 
them supply you with the remaining three number of the sequence.
Question 4
This is another counting item that exhibits basic multiplication skill. Present the item 
using the language that is understandable to the learners. You can ask them to count in 
tens, or ask them to count “10 by 10”. Present the first 3 number of the sequence, and let 
them supply y the remaining seven.
Question 5
This item test their knowledge of number facts, and also, basic reasoning using 
numbers. Present two items from the set, each item being presented horizontally. Ask 
the student to read the number sentence, and then solve it. While some learners will 
add abstractly, those who need to perform this task concretely should be allowed to do 
so.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 39
Question 6
This item tests addition of single digit, which yields answers is in double digits. Present 
two items from the set, each item being presented horizontally. Ask the student to read 
the number sentence, and then solve it. Some learners will add abstractly, while others 
will perform the task using concrete objects. This is acceptable. If it is not quite clear to 
you whether the learner has mastered addition, present a third item.
Question 7
The item tests subtraction of one single digit number form the other, with answers in 
single digits. Present two items from the set, each item being presented horizontally. 
Ask the student to read the number sentence, and then solve it. Some learners will 
subtract one number from the other abstractly, while others will perform the task 
concretely. This is acceptable. Present a third item if you need to convince yourself 
whether learner has mastered addition.
Question 8
The item tests application of subtraction in a commercial setting. The item should be 
presented in a number story involving denominations of 100. Present two items from 
the set. Most learners will perform the subtraction problem abstractly, while others will 
perform the task concretely. This is acceptable. Again, you may present a third item if 
you need to convince yourself of mastery..
Question 9
The item tests simple recall of names of shapes. All four shapes should be presented in 
any order. Please do not interpret the names of the shapes.
Question 10
The item tests a first level operation with properties of the four shapes. Conceal the 
shapes that you used for the previous item, and ask the learner to draw the shapes as 
you call them out. Precision in drawing is not important. However, the children need to 
clearly differentiate a square from a rectangle. Please do not interpret the names of the 
shapes.
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 40
Literacy Items (11 – 22)
All literacy items are to be presented in English, without any translation. Functional literacy at 
a very basic level of the English language is expected. 
Question 11
The item tests recall of names of colors. All the 8 colors in the list should be presented.
Question 12
The item tests understanding of language and use of language to recall and name items 
of clothing. Present an opportunity to name at least two items, and also be sure that the 
learner understands what you were asking.
Question 13
The item tests understanding of language and use of language in a conversation about 
different time periods in a day. Present another opportunity if you need to be ascertain 
whether or not the learner understands what you were asking.
Question 14
The item tests understanding of language and whether learners can use the language to 
describe what they see in pictures. Use the picture from the Taonga Market scene. 
Present several questions so as to have a good indication of the level of mastery (none, 
partial, or full mastery).
Question 15
The item tests understanding of language and use of language to confirm or deny 
certain facts. Using the picture from the Taonga Market scene, present several questions 
so as to have a good indication of the level of mastery (none, partial, or full mastery). 
Question 16
The item tests understanding of language and use of language associated with 
counting. Using the picture from the Taonga Market scene, ask several questions that 
would require the to count objects in context. 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 41
Question 17
The item tests understanding of language and its use to express personal meaning using 
reference to the first person, and possessive pronouns. Present several questions so as to 
have a good indication of the level of mastery (none, partial, or full mastery). 
Question 18
The item tests understanding of language and use prepositions and adverbs of position 
or place. Present several questions so as to have a good indication of the level of 
mastery (none, partial, or full mastery). 
Question 19
The item tests turning singular words into plurals. Present several words from the list 
so as to have a good indication of the level of mastery (none, partial, or full mastery). 
Question 20
The item tests recognition of letters of the alphabet. Present the letters at random first, 
and then consecutively if the learner is not able to recognize them at random. Present 
several strings, so as to have a good indication of the level of mastery (none, partial, or 
full mastery). 
Question 21
The item tests reading of words with single syllables. Present several so as to have a 
good indication of the level of mastery (none, partial, or full mastery). 
Question 22
The item tests spelling of words with single syllables. Present several so as to have a 
good indication of the level of mastery (none, partial, or full mastery). 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 42
Appendix I – IRI Success stories
There are many stories from IRI centers around the country. Most come from places 
that are struggling but where mentors, communities, churches, MOE officials, are doing 
an impressive and commendable job. A few selected stories are presented below.
“IT’S ACADEMIC!” RADIO QUIZ
At the end of Grade One, Chikuni Community Radio Station decided to organize an 
“It’s Academic!” radio quiz show for all centers and also invited local schools to 
participate. Fr Thadeusz asked a couple of teachers from local schools in Monze to 
develop questions based on the Grade One curriculum. The children from the centers 
answered virtually all the questions correctly, and since no centers were being knocked 
out of what was supposed to be a competition, the teachers were asked to make the 
questions harder! The gratifying aspect for the IRI program was that rural centers did 
just as well as centers closer to towns, and in fact won the competition.
MENTORS WITH A DIFFERENCE: MR MVULA
Garden Center began in July 2000 as part of the Learning At Taonga Market pilot 
program. The mentor, Mr. Mvula, originally taught the children in a Hope for Africa 
building in Garden Compound. After a few months, Hope for Africa closed its doors to 
Taonga Market suddenly, forcing Mr. Mvula to look for a new place in which to teach. 
At the same time, he was evicted from the home he and his family had been renting. 
The families of the 100+ children supported him and the center, but they could not give 
monetary or even in-kind support to him. 
The headmistress of Ngwelele Basic/Primary School had heard about and visited Mr. 
Mvula and the children as they learned at Taonga Market. She was very supportive and 
encouraging all along. When she heard that Hope for Africa had sent them away and 
that they were now squeezing into a tiny shelter behind a house in the market, she
offered some space on her school grounds. Since then, they have been learning from 
within the school compound under a tree. During holidays, they learn from an empty 
classroom. At one point during Grade One, Mr. Mvula realized he had far too many 
children to teach alone. Also, the children had to walk long distances from other 
compounds since he was the only one teaching for free. He met a woman who 
volunteered to teach some of his children in the neighboring Chaisa Compound, and so 
he invited her to observe his classes until she felt comfortable enough to try it herself. 
Ms. Phiri did this, and under his guidance, training and encouragement, she opened 
another center in neighboring Chaisa market, which they have named Garden 2. 
MENTORS WITH A DIFFERENCE: MR MATIMBA
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 43
Mr. Matimba of George Compound has trained at least two assistants to help him with 
his Taonga Center of over one hundred children. Even on surprise visits, we always find 
him at the center. Mr. Matimba has given the lead role to one of the assistants but does 
not leave the children even when an assistant is there. He is more than willing to help 
mentors in other communities and is extremely hard working and diligent. The children 
love him, because he treats them fairly and takes extra time to teach them when it’s 
necessary. 
MENTORS WITH A DIFFERENCE: MS MUMBIKA
Ms. Mumbika from Misisi Learning Center came to EBS in search of support for her 
center. For three cycles she has continued volunteering but now she is really struggling 
to make ends meet. She has no income and the community is not helping. Chawama, a 
neighboring center, had their radio stolen, so she invited them to join her class in Misisi. 
Now they share a structure and a radio. Ms. Mumbika told us that several times when 
she has been sick or too discouraged she tried to stay home from the center. She could 
not stop—the children followed her home and begged her to teach them! 
A LITTLE HELPFROM FRIENDS
The headmaster of Mununga School in Mpika is supporting an IRI center. Though he 
did not have too many resources, he donated his own radio, provided a venue (in his 
home where there is access to electricity because batteries were too expensive) and also 
provided a mentor – his wife! He is also providing chalk and a board and giving the 
children, most of them orphans, a chance to learn!
CREATIVE MINDS
One mentor could not find chalkboard anywhere. Yet, he really needed one. Then he 
realized that he actually had all the resources needed to make one. He and other 
mentors gathered soil from termite mounds and plastered it onto boards and portions 
of walls to create chalkboards for the learning centers. 
Pride should not spring just from having expensive things. Mentors in Mporokoso, 
Northern province were so proud of their center that they made a sign announcing its 
existence and posted it on the nearest road. . . it just so happens that the nearest road is 
thirty kilometers away from the center! But people are finding them, and they know 
that the community is working together to teach our children. 
In another center, because they had no money to buy chalk and other things for the 
center, they started a vegetable garden right next to the center. Now they sell the 
produce to buy chalk, pencils and books. Because chalk was too expensive and town 
Evaluation Report, Grade 1, 2001 Educational Broadcasting Services
Revised, December 2001 44
was too far away, another mentor decided to use what he had in his area. He found 
white clay soil and molded it into sticks of chalk, which he used to teach. A kind head 
master finally provided him with chalk.