NEW HORIZONS FOR PRIMARY SCHOOLS
YEAR 2000 PROJECTPERFORMANCE
PREPARED FOR:
UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT
KINGSTON, JAMAICA
PROJECT CONTRACTORS:
JUÁREZ ASSOCIATES,INC.
IN COLLABORATION WITH
IDEAS, LTD.
PREPARED BY:
RAY CHESTERFIELD, KJELL ENGE AND VIKKI FRANK
JUÁREZ AND ASSOCIATES,INC.
JUNE 13, 2000
CONTRACT NO. NHA-532-C-00-98-12345-00
i
TABLE OF CONTENTS
EXECUTIVE SUMMARY ....................................................... ii
A. Background ....................................................... 1
B. Methodology ...................................................... 2
1. Indicators ................................................... 2
2. Design ..................................................... 2
C. Assumptions ...................................................... 3
II. FINDINGS ............................................................ 5
A. Student Performance ................................................ 5
B. Teaching Quality ................................................... 9
B. Teaching Skills .................................................... 11
C. School Visits by NHP Specialists ...................................... 14
D. System Support ................................................... 15
III. CONCLUSIONS AND IMPLICATIONS .................................... 18
A. Conclusions ...................................................... 18
B. Implications ...................................................... 19
IV. BIBLIOGRAPHY ....................................................... 21
ii
“We thought New Horizons would be a cadre of persons
constantly visiting the schools and helping the
teachers.” NHP Principal, May 2000
“The teachers keep asking ‘When is New Horizons
EXECUTIVE SUMMARY
Formative Evaluation of the New Horizons Program
The comments above, made by school principals during the course of the formative evaluation
of New Horizons Program (NHP), illustrate a challenge for NHP as it enters it third year of
implementation. Findings showed that schools visited by project specialists had somewhat more
positive classroom environments, and greater relative numbers and usage of instructional materials than
did program schools which were not visited. However, visits directed at improving instruction took
place in a limited number of schools and totaled less than one day during the school year, on the
average. This was the result of what the NHP Project Director referred to as “a leveling strategy” of
working with several MOEC units to implement the new primary school curriculum in the program’s 72
school, and thus, provide a common base for all schools. This document summarizes the methodology,
the findings, and the implications of formative evaluation work that examined project implementation on
a number of indicators, compared to a 1999 baseline year. 
Formative Evaluation Procedures
A three-person team of experienced educational evaluators carried out the evaluation during
May-June 2000. The initial step in the evaluation was to obtain and analyze the 1999 student
assessment data for third and sixth grade mathematics and language arts. Subsequently, fieldwork was
undertaken in NHP schools using a multi-method design consisting of inventories, checklists, classroom
observation forms, and focused interviews to measure the conditions in place for effective learning in
NHP classrooms. Data analysis consisted of calculating the absolute and relative frequencies of each
behavioral indicator and making comparisons by gender and across different types of schools. Special
indices were created to examine complex issues such as teaching quality. 
Sample
Evaluation schools were a random, stratified sample of 16 schools and 27 classrooms. Schools
in which specialists had carried out workshops and classroom observations were slightly over-sampled
iii
to provide the best possibility to measure program effects.
Major Findings
NHP has been successful in building elements of system support at the local level. Virtually every
indicator has had a positive change over the course of the 1999-2000 school year, as shown by the
following:
• Schools with school development plans increased from 30% in 1999 to 100% in 2000;
• Schools with PTAs meeting regularly increased from 33% in 1999 to 94% in 2000;
• Parent participation in their children’s academic endeavors increased from 36% to 42%;
• School officials use of computers increased for administrative purposes increased from 0 to
20%;
• Schools with mathematics and language arts resource teachers increased from 15% to 88%
• Teacher participation in project professional development activities rose from 0 to 85%; and
• Teachers with special incentive strategies to improve attendance increase from 50% of the
sample to 70% 
NHP has been less successful in contributing to change in classroom practice. On most
indicators related to classroom performance there has been little change from the 1999 baseline. For
example:
• Teacher quality, as measured by student performance, classroom environment, and student
participation, remained unchanged at 43% out of a possible 100%
• Teaching skill, as measured by the quality of teacher-student interactions dropped slightly, from
14% in 1999 to13% in 2000.
• Teacher mastery increased from 12% to 28% but remains far below the 40% projected for
2000 and the 100% ideal. In addition, less than 10% of teachers could articulate the key
objective and strategies pertaining to children who have had limited academic success.
• The average number of students using instructional materials decreased from 25% to 13% for
mathematics and 27% to 20% for language arts.
• Use of instructional materials was higher in schools where formal visits had been made by NHP
specialists, but remained relatively low, at about 20%.
• The percentage of children reaching near mastery increased from 1998 to 1999, and surpassed
projections in every case but girls’ third grade language arts. However, this appears to be a
result of overall system improvement, as non-NHP schools showed similar increases and the
gap between the two types of schools was maintained. Further, as the 1998-1999 school year
was the first year of NHP implementation, interventions that might affect learning outcomes
were not yet in the schools.
iv
Lack of change at the classroom level seems to be related to the NHP team’s focus on laying
the groundwork at the system support level which limited time providing technical support in individual
schools and classrooms. This is indicated by the following:
• Schools that received formal professional development visits by NHP specialists had more
child-friendly learning environments, and greater availability and use of learning materials in the
classroom than schools not provided visits of this type.
• Project documents show that only 57% of the schools (a total of 41) received formal visits that
included structured activities related to program learning outcomes from specialists during the
year.
• The NHP team spent a total of 328 hours in these schools or a total of 41 days which is an
average of one day per visited school.
• Visits focused on smaller schools, thus only about 43% of the student population were attended
by the formal visits. 
Implications
The impact of school visits by NHP specialists argues for increased hands-on support in
schools and classrooms in order to assist teachers to develop appropriate learning strategies within
local contexts. The planning might begin with an NHP retreat to determine strategies to maximize the
effectiveness of technical support. The implementation of such hands-on support should be carefully
monitored so that all schools are attended and the impact on project outcomes can be determined.
The lack of student participation and the highly teacher-centered nature of NHP classrooms
suggest that NHP professional development efforts should be highly targeted to deal with these areas,
especially in terms of the importance of all children’s participation to the acquisition of numeracy and
literacy.
Teachers lack of knowledge about the specific targets, objectives, and strategies of the New
Horizons for Primary Schools project suggest that a concentrated effort must be made to increase
teachers’ knowledge of the program in professional development efforts.
The differences in performance of students by grade level and school type implies that special
strategies may be necessary for different populations of students. Special emphasis should be placed
on the teaching-learning situations in the upper grades of primary schools, as the gap between NHP and
non-NHP students grows between third and sixth grade.
Installation of computers, training in the use of administrative software, and strategies for
maintenance of computers and computer environments must be carried out as soon as possible, if
school administrative capacity is to be improved.
1
I. INTRODUCTION
This document describes the results of the second year of formative evaluation of the New
Horizons for Primary Schools (NHP) Project. The evaluation is carried out at the end of the Jamaican
school year (May-June) to provide a barometer of the progress that NHP is making on a series of
indicators. It had two purposes. First, the formative evaluation results inform the implementation of
NHP interventions and permit NHP staff to target interventions in critical areas of the program. The
results complement those of ongoing assessments of the implementation process undertaken informally
through school visits, feedback on professional development efforts and periodic communication with
school administrators and teachers. Second, the formative evaluation process serves to measure
project results from an established baseline. As it provides systematic monitoring of performance over
time, formative evaluation contributes to the measurement of final project results. (Baseline indicators
and projections of change over time derived from the 1999 baseline study are found in Attachment A to
this report). The 2000 evaluation was conducted during May-June by a three-person team consisting
of Dr. Ray Chesterfield, Dr. Kjell Enge, and Ms. Vikki Frank. All of these individuals are experienced
education evaluators who have worked extensively with primary school programs in the Latin America
and Caribbean Region.
A. Background
The primary objective of New Horizons for Primary Schools (NHP) is to enhance the
performance of Jamaican primary school students in numeracy and literacy. The focus of the technical
assistance component of the project is on those children who, because of poverty and a lack of other
enabling conditions, have had little academic success in school. Increased academic success is to be
accomplished through the development of model interventions that, when tested, can be used to
improve the performance of low-achieving children throughout Jamaica. Thus, the products of the
contractor’s work will be changes in schools and classrooms that result in individual students having
greater academic success in primary school. Such results will include measurement of the indicators for
the USAID strategic objective. 
Systems, such as computerized administrative and student tracking systems, are also being
implemented over the life of NHP. These systems will assist schools to monitor their own performance. 
The results of such individual school monitoring can be aggregated to examine project performance. 
Similarly, NHP will integrate Ministry of Education and Culture (MOEC) databases to provide
additional data sources for monitoring performance. Until such systems are fully operational, however,
monitoring will be carried out as part of the formative evaluation effort designed to provide feedback to
program technicians implementing the interventions. As formative evaluation requires in-depth data
collection, a representative sample of NHP schools are selected each year for evaluation purposes.
Many of the indicators for monitoring performance are complex concepts that require the
combination of qualitative and quantitative methods to measure accurately. The following pages discuss
2
the procedures used to collect data on NHP indicators. Subsequent chapters present the findings of the
formative evaluation, in terms of change from the baseline data, and provide conclusions and
implications drawn from these findings.
B. Methodology
1. Indicators
The indicators are taken largely from the U. S. literature on school/classroom effectiveness and
on the growing body of international literature on classroom interaction and educational quality (see
attached bibliography for examples). Three levels of indicators were used. The first relates to student
performance in terms of mastering the curriculum. The second consists of indicators of teacher
performance that are generally associated with greater quality in terms of students’ academic
performance. Third, are indicators of system support or enabling factors such as efficient school
management, professional development opportunities for teachers, and parent participation in the
education of their children, that must be in place to improve the performance of individual children.
2. Design
A multi-method design consisting of inventories, checklists, classroom observation forms, and
focused interviews was employed to measure the conditions in place for effective learning. This design
allows for the measurement of the impact of the interventions implemented to improve learning,
especially among students who have had limited success in school. Evaluation efforts focused on both
females and males. This is important not only to ensure that initiatives are equitable but to identify
initiatives and strategies that are successful regardless of gender. 
Sample. A stratified random sample of 22% of project schools was drawn from the universe
of 72 schools. Schools were stratified by size and type (primary or all age) then randomly selected
within strata. As the focus of the project is a “ground-up” approach that begins with needs identified by
participating schools, those schools that had been most involved in NHP activities during the year were
over-sampled. The final sample consisted of 16 schools and 27 classrooms for intensive data collection
and analysis. The focus of the formative evaluation was on third grade. The purpose of the formative
evaluation was to obtain in-depth, systematic data, in a limited amount of time. Thus, it concentrated on
one grade as an indicator of general progress. Third grade was chosen, because there are test scores
available which allow greater diagnostic ability and permit the monitoring of change in the cohort of third
graders serving as the baseline over the life of the project. This is important because both the1998 and
1999 NAP scores suggest that NHP children fall behind principally between third and sixth grade.
First, second, fourth, fifth and sixth grade classrooms were also observed. The data from these
classrooms showed the same general patterns as those for the sample as a whole. This suggests that for
monitoring purposes, third grade results can be used as a general indicator of progress.
Instruments. Instruments included classroom maps, materials inventories, classroom
3
observation forms, classroom environment assessments, and interviews guides for use with teachers,
students and school principals. Maps were employed to identify children and to examine the context in
which they interact with teachers. Materials inventories measured both the presence and use of all
materials at different times during math and language arts lessons. Observational sweeps were made at
three points in time during each academic context. At each sweep, the number of books and ancillary
materials available and in use were counted. Classroom interaction was measured through a teacher￾student interaction protocol. This instrument focused on teachers' interactions with individual students
and the nature of those interactions in different academic classroom activities. In order to ensure
consistency and control for contemporaneous events that might influence behavior patterns, the form
was used for ten minutes at four different times during the instructional day in third grade classrooms. 
Two observations took place during mathematics lessons and two during language arts. Thus, a
behavioral sample of 20 minutes for each of the target content areas was created. Researchers used
the classroom environment instrument to rate the appropriateness of the classrooms for child-centered
learning. Teachers’ perceptions of the interventions, as well as their mastery of and commitment to the
new approaches implemented under NHP, were tapped by a teacher interview schedule. Similarly,
changes in the school management planning and systems were measured through an interview with the
principal. Students were queried about activities in the home and involvement of parents in the
children’s reading.
Fieldwork Procedures. A three-person research team made up of researchers, experienced
in school, classroom and community research, conducted the formative evaluation. They synchronized
observations through a one-day training exercise prior to entering the field. This training included
exercises with the instruments using videotapes of classroom interaction in schools to ensure
consistency in observations and interviewing. Parallel observations were conducted with the
instruments until an inter-observer agreement coefficient of above .80 was reached for all observational
instruments. The researchers worked individually in small schools and in teams in larger schools. They
spent up to one full day at each school. Procedural guides and operational definitions were attached to
specific instruments as references to ensure consistency in field procedures during the investigation.
Data Analysis. The principal unit of analysis was the classroom. As the interventions are
focused largely on improving teaching, it will be changes in classroom-level environments and behaviors
that affect student learning. Data analysis consisted of calculating the absolute and relative frequencies
of each behavioral indicator and making comparisons across the two evaluation years. Differences by
types of schools were also examined. Special indices were created to measure complex issues such as
teaching quality. Where appropriate, non-parametric statistics such as chi-square were used to
examine differences among the sample.
C. Assumptions
The study was based on several assumptions. First, the school and the class are the key units
of analysis in planning and intervening to improve the quality of learning. Second, the school is a social
system and the interaction of all of the elements within a school has an influence on student learning
4
beyond that provided individually by inputs to the school. This is not to suggest that the uniqueness of
each school makes aggregate measurement impossible, but rather that accurate measurement of the
impact of schooling is a complex undertaking requiring the integration of a variety of data collection
approaches.
5
II. FINDINGS
R. Student Performance
Jamaica is promoting pupil-centered “everyone can learn” concept of teaching rather than a
norm-based “cream of the crop” approach. Thus, the focus is shifting to all children’s mastery of the
curricular content. This means that the array of individual scores will shift from the normal distribution
or “bell shaped curve” associated with a norm-based assessment and mean scores, toward a “J￾curve”with a few students falling at the low end and the middle and most scores reflecting a high degree
of learning. However, with the current inverse J-curve, the first step will be to move students to “near
mastery” levels. NAP defines less than 50% of the items in each domain correct as “no mastery” at the
third grade level. Although NAP does not designate mastery levels for the sixth grade GSAT, the
criteria used at the third grade level was employed in determining student progress (less than 50%
correct = “no mastery,” 50% to 75% correct = “near mastery” and above 75% = “mastery”. 
Owing to lags in data collection and processing, 2000 student performance data were not
available at the time of the evaluation. Thus, change over time is presented in terms of changes from
1998 to 1999. The analysis will be updated when 2000 test data become available.
Table 1: Student Performance on Third Grade Maths Test
Math 3 Near Mastery % change Mastery % change Total % change
NHP 1998 1999 1998 1999 1998 1999
Female 37.9% 45.1% 7.2% 9.3% 19.4% 10.1% 47.2% 64.5% 17.3%
Male 28.8% 37% 8.2% 4.1% 11.8% 7.7% 32.9% 48.8% 15.9%
Total 33.2% 41% 7.8% 6.6% 15.6% 9% 39.8% 56.6% 16.8%
Non-NHP 1998 1999 % change 1998 1999 % change 1998 1999 % change
Female 43% 45% 2% 12.7% 28% 15.3% 55.7% 73% 17.3%
Male 33.8% 38.5% 4.7% 7.9% 19.5% 11.6% 41.5% 58% 16.5%
Total 38.4% 41.6% 3.2% 10.3% 23.8% 13.5% 48.7% 65.6% 16.9%
Source: NAP 1998 and 1999 database
Table 1 shows the change in the percentage of children reaching near mastery of the third grade
mathematics curriculum, as measured on the diagnostic tests for that grade. Changes in student
performance from 1998 to 1999 are compared to all primary schools not participating in the NHP
program. As can be seen, there is significant change from one testing period to the next. This may in
part be due to change in the tests themselves. A 70-item test was used in 1998 whereas a 60-item test
was employed in 1999. However, as the change in tests were the same for both groups of children, it
can be seen that NHP children showed similar overall gains to the children in non-NHP schools. NHP
6
showed greater gains in the percentage of children reaching near mastery whereas non-NHP schools
had slightly larger gains in mastery.
Table 2 presents the change over time in the percentage of girls and boys reaching near mastery
and mastery in third grade language arts. For both sets of schools, the percentage of children reaching
the near mastery level decreased. This was, however, offset by children obtaining mastery of the
language arts subject matter. As with mathematics, higher percentages of NHP students are at the near
mastery level whereas greater percentages of children in non-NHP schools have developed mastery.
Table 2: Student Performance on Third Grade Language Arts Test
Language
Arts 3
Near Mastery % change Mastery % change Total % change
NHP 1998 1999 1998 1999 1998 1999
Female 46.9% 42% -4.9% 26.2% 31.1% 4.9% 73.1% 73.1% 0%
Male 37.8% 37.9% .1% 13.5% 16.5% 3% 51.3% 54.4% 3.1%
Total 42.2% 39.9% -2.3% 19.7% 23.7% 4% 61.9% 63.6% 1.7%
Non-NHP 1998 1999 % change 1998 1999 % change 1998 1999 % change
Female 40.7% 34.6% -6.1% 37.7% 46.1% 8.4% 78.4% 80.7% 2.3%
Male 40% 34.8% -5.2% 21.9% 29% 7.1% 61.9% 63.8% 1.9%
Total 40.3% 34.7% -5.6% 30% 37.6% 7.6% 70.3% 72.3% 2.3%
Source: NAP 1998 and 1999 database
 
Table 3 compares sixth grade mastery in mathematics among children attending NHP schools
and those in other schools. The trends are similar to those found in third grade, as NHP has greater
relative gains at near mastery whereas there is a greater overall increase in non-NHP children reaching
mastery levels. Lower percentages of NHP students are, however, evident at both levels.
7
Table 3: Student Performance on Sixth Grade (GSAT) Maths Test
Math 6 Near Mastery % change Mastery % change Total % change
NHP 1998 1999 1998 1999 1998 1999
Female 13.9% 22.2% 8.3% .3% 1.7% 1.4% 14.2% 23.9% 9.7%
Male 5.5% 8.8% 3.3% .2% .8% .6% 5.7% 9.6% 3.9%
Total 9.7% 15.9% 6.2% .3% 1.3% 1% 10% 17.2% 7.2%
Non-NHP 1998 1999 % change 1998 1999 % change 1998 1999 % change
Female 26.5% 31.9% 5.4% 2.9% 6.9% 4% 29.4% 38.8% 9.4%
Male 13.8% 17.3% 3.5% 2.3% 4% 1.7% 16.1% 21.3% 5.2%
Total 20.6% 25% 4.4% 2.6% 5.5% 2.9% 23.2% 30.5% 7.3%
Source: NAP 1998 and 1999 database
Table 4 presents sixth grade progress in language arts. Again, non-NHP schools have more
children reaching the targeted levels than NHP schools. However, there is actually a slight reduction in
the gap between the relative frequency of success. Relatively greater numbers of NHP students reach
both near mastery and mastery. Non-NHP schools have a slight overall decline at the near mastery
level. 
Table 4: Student Performance on Sixth Grade (GSAT) Language Arts Test
Language
Arts 3
Near Mastery % change Mastery % change Total % change
NHP 1998 1999 1998 1999 1998 1999
Female 34.2% 37.3% 3.1% 8.3% 9.3% 1% 42.5% 46.6% 4.1%
Male 15.2% 17.3% 2.1% 2.1% 1.6% -.5% 17.3% 19.9% 2.6%
Total 24.6% 28.4% 3.8% 5.2% 5.7% .5% 29.8% 34.0% 4.2%
Non-NHP 1998 1999 % change 1998 1999 % change 1998 1999 % change
Female 39.6% 42.3% 2.7% 20.4% 18.2% -2.2% 60% 60.5% .5%
Male 23.5% 25.3% 1.8% 8.2% 7.4% -.8% 31.7% 32.7% 1%
Total 32.2% 34.2% 2% 14.7% 13.1% -1.6% 46.9% 47.3% .4%
Source: NAP 1998 and 1999 database
With the exception of sixth grade language arts, where there is a drop of over three percentage
points (17.1% to 13.3%) in the overall difference between NHP and non-NHP schools, the gap
between to the two groups of students is maintained from 1998 to 1999. As in 1998, the gap is greater
8
between sixth graders (13.3% for both maths and language arts) than for third graders (10% and 8.7%
for maths and language arts, respectively). The differences are similar for males and females, except for
third grade language arts, where there is no appreciable gap between NHP and non-NHP male
students.
In order to examine possible difference by school type, the percentage of children below near
mastery in the formative evaluation sample schools was calculated and aggregated by school size. As
can be seen from Table 5, the general improvement noted previously is found for all schools. However,
males’ performance at the third grade level differs dramatically by school type. Much higher
percentages of male students in small- and medium-sized schools fall into the category of no mastery of
the curriculum than males in large schools. 
Table 5: Percentage of NHP Children Below Near Mastery in Sample Schools
1998 Third Grade NAP Scores - Students below “near mastery”
Size/Subject Area Language Arts Mathematics
Males Females Males Females
1998 1999 1998 1999 1998 1999 1998 1999
Small 70.5% 53.4% 32.5% 28% 77% 60.4% 66.6% 36.3%
Medium 71% 53.3% 29% 33.5% 95% 59% 60% 45.5%
Large 49.7% 39.5% 31.2% 24% 63.8% 44.2% 47.2% 31.2%
1998 Sixth Grade GSAT Scores - Students at/below 50% correct
Language Arts Mathematics
Males Females Males Females
1998 1999 1998 1999 1998 1999 1998 1999
Small 95% 85% 69.6% 59% 100% 92% 95% 79%
Medium 88.7% 80% 54.3% 55% 98% 92% 85% 79%
Large 69.7% 79% 63.7% 51% 95.7% 89% 87.5% 73%
Source: NAP 1998 and 1999 database
There are a number of possible explanations for these results. Children in smaller schools are
often in multi-grade classes, in which teachers who have not had experience dealing with such classes
may teach each class individually, thus limiting overall instructional time. As the small schools in the
sample are relatively isolated, children may not have access to printed matter outside of the school or
may miss large amounts of school time to help in the agricultural pursuits of their parents. Finally,
children may not come to school fully proficient in oral English which can limit comprehension of all
subject matter. 
9
At the sixth grade, the difference between male students in large schools and small or medium
schools persists. In mathematics, however, the general lack of mastery by all students negates any
influence of school type. As with the NHP schools as a whole, there is a consistent trend toward
greater percentages of children showing less mastery of subject matter as they advance in grade.
B. Teaching Quality
Teaching quality was measured through an index made up of three generally accepted
standards for determining teacher performance: content knowledge of students; environment for student
learning; and teaching for student learning. The first of these dimensions has been discussed in the
previous section. Third grade performance, measured as the percentage of children reaching near
mastery over all children taking the NAP diagnostic tests is used. Both mathematics and language arts
performance are used in the index. 
Learning environment standards relate to the social and emotional components of learning as
prerequisites to and context for academic achievement. Thus, the focus is on the physical setting
created by the teacher and the resources available. A six-item scale dealing with the fostering of a
positive self-concept, the creation of a nurturing environment that supports gender equity, and the
organization of space and materials to allow a variety of learning opportunities was used to measure the
quality of the environment . Researchers used the assessment instrument after a complete series of
observations in a classroom. Specific criteria were provided with each item to ground the ratings. 
Ratings were made on a three-point scale of “not met,” “partially met,” and “fully met”. Thus, scores
ranged between a minimum of six and a maximum of 18. Scores were expressed as a ratio of the
actual score over the total possible score.
Table 6 compares the classroom environment scores for 1999 and 2000. There has been
some improvement in scores for each type of school. This improvement is reflected in the overall
increase in the classroom environment score. However, scores remain relatively low. Classrooms
generally met criteria of lack of physical punishment and interacting with individual children often. Equal
lighting, ventilation, and furniture for boys and girls was also generally met. Other criteria such as
displaying children’s work, creating a variety of learning opportunities within the classroom, encouraging
children to express themselves with peers and adults, using materials that showed males and females in
traditional and non-traditional roles, were usually not met. In many of the classrooms, especially those
in larger schools, the lack of space contributed to a less than optimal classroom environment. Children
in these classrooms usually were wedged tightly into desks and the only space for displaying materials
were blackboards that served as partitions between classrooms.
10
Table 6: Mean Classroom Environment Scores by School Size
Mean/School Size 1999 2000
Small .5929 .6389
Medium .5900 .6588
Large .4867 .5490
Total .5464 .6115
 
 Teaching for student learning is concerned with the act of teaching and its overall goal of helping
students understand the content that they are imparting and the ability to present the content in a manner
that is consistent with the knowledge, interests and abilities of the students. For the purposes of
monitoring, the focus is on interactions in the classroom between teachers and students. Student￾initiated interactions were taken as an indicator, as such interactions show teachers’ willingness to
recognize student input. Student-initiated interactions were found to be a very low percentage of all
interactions in teacher-centered classrooms. As mentioned, a corpus of 40 minutes of observations of
academic lessons were collected in each classroom. These observations were divided equally between
mathematics lessons and language arts lessons.
Table 7 presents the percentage of observed interactions initiated by teachers and students in
the normally occurring contexts of the classroom in 1999 and 2000. The table shows the percentage of
interactions initiated by each actor in the contexts observed taking place in the classroom. The bottom
row provides the overall percentage of interactions initiated by teachers, boys, and girls. Teacher￾initiated interactions predominate in both years. They make up at least 88% of all interactions. 
Table 7: Interaction Initiator by Classroom Context
Context Interaction Initiator
Teacher Boy Girl Total Context
1999 2000 1999 2000 1999 2000 1999 2000
Teacher-led small
group
9.1% 2.6% 1.3% 2.0% 10.0% 0 9.2% 2.4%
Student-led small
group
2.3% .3% 0 0 0 .9% 2.1% .3%
Large group 51.2% 77.2% 29.2% 65.0% 16.0% 52.3% 49.2% 75.5%
Seatwork 32.4% 17.7% 55.4% 25.0% 49.0% 38.5% 34.4% 19.4%
No instruction 4.0% 2.2% 13.7% 8.0% 2.5% 3.4% 5.1% 2.5%
Total Interactions 92.5% 88.7% 3.8% 5.2% 3.6% 6.1% 100% 100%
Student-initiated interactions increase somewhat from the baseline year. In 1999, they totaled
slightly more than seven percent of all interactions, whereas in 2000, they make up about 11 percent of
the total. Such interactions are, however, low as in a child-centered environment, up to one-half of all
11
interactions between teacher and students might be expected to be initiated by the children. Little
difference is noted by the gender of the students, as both boys and girls initiate interactions with similar
frequency.
The principal change is in the types of contexts in which the majority of interactions occur. 
Whereas seatwork predominated for child-initiated interactions in 1999, classrooms in 2000 appear
more oriented to large group contexts. This may be a result of teachers attempting to involve children
to a greater extent in classroom activities, as called for by the new curriculum, and their lack of
familiarity with organizing small group work. Small group interaction actually decreases in 2000,
despite strategies promoted by both NHP and the new primary school curriculum to encourage small
group work.
Despite the slightly higher results on the three measures making up the index of teacher quality,
overall teacher quality scores remained similar for 1999 and 2000. The index score for each year was
.43 or 43% of a possible 100%. This may be a result of not all schools reporting test scores, which
eliminated some schools from being included in the overall index. 
A subsample of six classrooms was used to examine the number of children in the classroom
who actually engage in interactions with the teacher. Detailed observations of which children had
contact with the teacher were made. It was found that in mathematics lessons, 41% of the boys in the
classroom had interactions with the teacher compared to 34% of the girls. The percentages were
similar for language arts lessons where 35% of the boys present and 34% of the girls interacted with the
teacher. Across lessons, 59% of boys and 55% of girls interacted with the teacher. This was a total of
57% of the students in the classroom, on the average.
B. Teaching Skills
Several indicators of teaching skill are important to the NHP project. Obviously, the ability to
effectively create an environment that instills self-confidence in students and allows them multiple
learning opportunities, discussed previously under teaching quality, is related to pedagogical ability. The
focus here is on specific behaviors engaged in by teachers that encourage children to participate in the
learning process. Included are: the quality of teacher-student interactions and the use of materials by
students; teachers’ mastery of and commitment to the interventions introduced by NHP; and teachers’
strategies for encouraging student participation through regular attendance. 
Quality of teacher-student interactions. Teachers’ ability to impart information and encourage
inquiry rests largely with the types of verbal and non-verbal interactions that they use to engage
students. To be effective, such interactions create situations that allow students to apply their
knowledge and not merely memorize facts. Teachers must also monitor learning to make certain that
students assimilate information accurately and can use what they have learned. Permitting students to
expand ideas, together with providing feedback and explanation as needed, are generally considered
manifestations of these skills.
12
The structured observations of mathematics and language arts, described previously, were used
to collect data on the quality of student-teacher interactions. The percentage of all interactions that
involved explanation and feedback was used as the measure of teaching skill. As shown in Table 8,
teachers provided relatively little explanation or expansion of ideas. This type of behavior was found in
only 13.4% of all interactions in 1999 and 7.3% of such interactions in 2000. Feedback in the form of
either praise or punishment was similar for years. In each, it made up about 5% of all interactions.
Table 8: Quality of Interactions
Context/Interaction 1999 2000
Questions 37.3% 64.1%
Expands 13.4% 7.3%
Orders 40.6% 30.3%
Dictates/Lectures 20.3% 18.1%
Praises 2.9% 3.3%
Punishes 1.5% 1.3%
Use of materials. A principal focus of the project is on improving the availability and use of
instructional materials. Both texts and supplementary instructional materials provide children with a
channel for interacting with academic content on an ongoing basis. Often, however, it is assumed that
children have books available and that teachers are trained in using instructional materials effectively. 
Teachers may lack practical experience in using texts and when working in a development situation may
face overcrowded classrooms, children without books and little in alternative instructional resources. 
Thus, they resort to extensive lecture and use of the chalkboard. The purpose of this indicator is to
confirm the provision to classrooms of project schools of sufficient supplementary materials to enrich
the teaching and learning of literacy and numeracy. However, availability of materials alone is not an
adequate measure, as materials must be used by student in order to enhance academic achievement. 
Use of materials was measured by three visual sweeps of the classroom during both
mathematics and language arts lessons. During the sweeps, the number of available books and
supplementary instructional materials and manipulatives were counted separately then the number
actually in use was noted. The average number of materials available per child, as well as the average
number of materials in use were calculated.
As shown in Table 9, both mathematics texts and supplementary materials such as
manipulatives, and reading materials increased in the classrooms. This was in part due to the
supplementary materials provided by NHP which were present in a number of sample classrooms. 
However, in several schools these materials were found stored in the teacher’s office rather than
13
present in classrooms. The availability of reading materials increased to the extent that almost a text per
child, on the average, were observed to be readily available in the sample classrooms. 
Table 9: Availability and Use of Tests and Other Learning Materials
Subject Availability Use
1999 2000 1999 2000
Maths .2 .4 .25 .13
Reading .4 .9 .27 .20
The use of materials had, however, declined somewhat over the year. Only about one child in
ten were observed to use math materials, whereas two children in ten used reading materials during the
observations. The decrease in use of materials may be a result of teachers transitioning to the new
curriculum. Available materials are largely those used with the old curriculum. Thus, teachers trying to
implement the new curriculum appear to be doing so without employing the older texts. 
Mastery of the intervention. There is consensus in the international literature on educational
innovation that mastery of new instructional approaches by teachers is a critical factor in adoption and
sustainability. As NHP interventions are not yet in place, mastery was measured by asking teachers
about the general objectives of the program. This proxy measure will be adjusted to tap understanding
of specific interventions as these are developed. A second factor closely associated with mastery of the
innovation is commitment to the new approach. This aspect of teaching skill were measured through a
series of hypothetical questions in the teacher interview on circumstances that might deter a teacher
from using an approach.
Teacher mastery improved somewhat from 1999 to 2000. Teachers were able to identify 28%
of the major objectives of NHP as compared to an average of 12% in 1999. However, the increased
understanding of the program appears to be tied to the dual implementation of the NHP innovations and
the new national primary curriculum in NHP schools. Sixty-three percent of the sample teachers
identified elements that the programs have in common, whereas only 7% of the sample teachers were
able to identify elements unique to the NHP program such as a focus on less successful children, and
strategies of mixed skill and age groupings aimed at increasing the participation of these children. 
Commitment to the combined emphasis of the two programs was strong with 85% of the teachers
stating that they would continue to use what they had learned even if the program were discontinued.
Strategies for encouraging attendance. The purpose of this indicator is to measure the extent
to which project activities influence absenteeism rates among students. Attendance was examined by
gender, as male attendance is traditionally lower than female attendance throughout the country. As
official school attendance may run the risk of inflation or deflation, an correction factor of observed
attendance recorded by the evaluation team was built into the measure. The key to the success of
incentive programs will be their integration with the teaching-learning process, thus, classroom teachers
14
are the appropriate source of information about incentives. Teachers were asked to list all of those
which they were using in their classrooms.
There was an increase in the percentage of teachers using incentive strategies. In 1999, half of
the teachers interviewed stated that they used incentives to increase attendance. In 2000, 70% of the
sample described strategies used to encourage students to come to school. Despite the greater use of
incentives, there was an observed decrease in attendance from 1999 to 2000. Overall attendance went
from79% to 65% and male and female attendance had similar declines. This appears to be tied to the
unique events that took place during the data collection period and suggest that 2000 attendance figures
are an anomaly that should not be reported as a performance indicator. The effect of midterm break as
a result of labor day occurring on a Tuesday, decreased attendance in schools on both the Monday
preceding and the Wednesday following the break. Likewise, the administration of fourth grade literacy
tests during the data collection period may have resulted in lower attendance of sample children who
did not understand that their classrooms would not be affected by testing. 
C. School Visits by NHP Specialists
In order to examine the impact of working directly at the school level, NHP records were
examined and a variable for school visits was created. Project records show that formal visits related
to the objectives of the program with regard to numeracy and literacy were of three types - visits to
discuss school development plans as they related to literacy and numeracy, classroom observations by
the project subject matter specialists, and professional development workshops with school staffs. 
According to the Project Director, visits were conducted using the strategy of giving priority to schools
with the lowest tests scores. As shown in Table 10, this resulted in a high concentration of the work in
small schools, with 85% of schools of this type in the program being covered. Fewer than half of the
medium size schools and less than one-fourth of the large schools were reached for these types of
activities. Types of coverage also differed by school type as small schools received the bulk of the time
spent in workshops and large schools received only school development plan consultation.
Table 10: Time Spent in School Visits by NHP Specialists
Schools Number* Visited Hours
SDP
Hours
Observation
Hours
Workshop
Total Hours
Small 33 28 (85%) 56 25 194 275
Medium 25 11 (44%) 22 10 15 47
Large 13 3 (23%) 6 0 0 6
Total 71 42 (59%) 84 35 209 328
 Source: NHP monitoring documents
* Schools in monitoring document totaled 71
15
Even in the small schools, the amount of time spent with each of the schools averaged slightly
more than one eight-hour day. Whereas the time spent with the other types of schools averaged 2.1
hours and 2 hours for the medium size and large schools, respectively. When the school populations
were extrapolated from enrollment data at sample schools, findings suggested that only about 43% of
the student population of NHP had been attended as a result of the strategy to concentrate on small
schools.
Despite the limited number of hours spent by specialists at project schools, work at the school
level appears to have had a positive impact in the classroom. Table 11 presents comparisons on
selected classroom variables between sample classrooms in schools visited by the specialists and in
NHP schools that were not visited. The classroom environments were more responsive to children in
schools where the specialists had worked. In addition, there was greater availability of materials on the
average, and slightly greater use of materials in the classrooms of schools that had been visited. 
Overall, however, the classroom environments remained relatively unsupportive to child-centered
learning and less than two children in ten used learning materials. 
Table 11: Comparison of Visited and Unvisited Classrooms on Selected Variables
Type Classroom
Environment
Availability
Maths 
Use Maths Availability
Language Arts
Use Language
Arts
Visits .66 .76 .18 1.3 .23
No Visits .57 .14 .08 .62 .18
D. System Support
In order to improve the success of children, teachers must be supported by an infrastructure at
the school and national level. This includes support for professional development that will contribute to
successful teaching and learning, effective management of the local learning institution to ensure that
teachers can focus on teaching, and participation of community members in the education of their
children.
Professional Development. Training to upgrade skills and knowledge is one of the main ways
that a school system provides support for teachers. Such training can come about through in-service
courses and workshops or through interaction with colleagues who have specialized knowledge in a
particular subject area such as mathematics or language arts. This indicator establishes the number of
teachers that have engaged in professional development activities as a consequence of their
participation in New Horizons. The indicator takes into account training in Jamaica and abroad. 
Schools with resource teachers are also used as an indicator. All professional development activities
are coordinated with the Professional Development Unit of the MOEC.
As shown in Table 12, at the time of the formative evaluation baseline data collection, no
teacher had participated in training offered through the New Horizons project. By the end of the 1999-
16
2000 school year, 85% of sample teachers stated that they had participated in such training. Similarly,
the availability of resource teachers had risen from 15% in 1998-1999 to 94% of schools in 1999-
2000. 
Table 12: NHP Professional Development
Professional Development/Year 1999 2000
Teachers in Workshops 0 85%
Schools with Resource Teachers 15% 94%
School Management. Tracking of school resources and students is an important function of
school management. Such tracking should be undertaken within a framework of specific objectives and
activities. Thus, the utilization of school management plans in regard to NHP activities together with the
utilization of the computer and accompanying administrative software, which can speed principals’
decision-making and ease reporting burdens, are the indicators of effective school management. 
Effectiveness of school boards is an additional indicator of school management. Measures for this
aspect of management will be developed by the National Council on Education (NCE).
As part of the NHP program, principals were asked to design development plans taking into
consideration school needs, teacher training, curriculum design and parent/community involvement,
especially as related to improving student literacy and numeracy. Among sample principals, 30% had
completed this task at the time of 1999 formative evaluation data collection. Since most of those
interviewed mentioned progress in completing the plans, it was expected that the number would
increase rapidly. As can be seen from Table 13, all principals were implementing their development
plans by May of 2000.
Table 13: NHP School Management
Professional Development/Year 1999 2000
School Development Plan 30% 100%
Computer present 25% 68%
Computer used for administration 0 20%
The percentage of schools with computers increased by 43% during the year. This was largely
a result of the NHP program which was delivering computers to the schools at the time of the 2000
data collection. Eighty-one percent of the schools with computers had received them from New
Horizons. Although installation and training with administrative software had not yet taken place,
several of the principals were already beginning to input information in the new computers. This makes
proper installation and training imperative to ensure that consistent administrative decision-making and
reporting will take place.
17
In both years of data collection, principals stated that equipment was in disrepair owing to lack
of funds for ongoing maintenance. A similar phenomenon occurs not only with computers acquired
prior to NHP, but with other audio-visual equipment and support equipment such as copiers. 
Community Involvement. The body of research on parent participation shows positive effects
brought about by parental emphasis on literacy and other academic pursuits in the home. As the focus
of the project is on improved student learning, parental participation in learning is measured. In
addition, parental participation in management is important to assure that schooling is relevant to
community interests. Thus, the presence of parent-teacher associations and the frequency of their
meetings are other indicators monitored through formative evaluation. Other indicators, such as the
number of schools with parent participation programs, and training for parent and community leaders,
will be monitored in partnership with the NCE.
Samples of NHP students were asked about parental involvement in their studies. In 1999,
these interviews were conducted as part of the NHP school survey, whereas in 2000 data were
collected as part of the formative evaluation. Table 14 shows that there has been a slight increase in the
number of students who stated that either their father or their mother assisted them in their reading. 
Table 14: NHP Community Involvement
Year 1999 2000
Parent Participation in Learning 36% 42%
PTA present 89% 100%
PTA meets regularly 33% 94%
Eighty-nine percent of the NHP schools had PTAs in 1999. However, only 33% meet on a
regular monthly schedule. In 2000, all of the schools had PTAs and almost all were meeting regularly.
18
III. CONCLUSIONS AND IMPLICATIONS
The purpose of the study was to assess the progress made by the New Horizons in
implementing activities that will lead to increased numeracy and literacy for students who have had
limited success in school. The comparisons made from the baseline year of 1999 with the results of the
formative evaluation in 2000, allow certain conclusions and implications to be drawn that can help to
guide further implementation of the program.
A. Conclusions
Mastery of the curriculum by NHP students has improved. However, students mastery of
the curriculum remains low and gains appear to be a function of overall system improvement
rather than specific interventions by NHP. The percentage of NHP students reaching near mastery
increased by 16% and 1.7% for third grade mathematics and language arts, respectively. Similarly, the
percentage of students reaching at least near mastery in these subjects on the GSAT increased by 7.2%
and 4.2%. However, non-NHP schools showed similar levels of improvement. Lack of significant
improvement specific to the NHP schools is not surprising as test data were only available for the first
year of program implementation. Data for the 1999-2000 school year, expected to be available in
early August, will be a better measure of impact as the initial project interventions in professional
development and supplementary learning materials took place in this period. It is important to note that
percentages of students in the “no mastery” category continue to be high and the percentage of those
who achieve mastery is very low.
The final three years of primary school appear to be critical for improving the mastery
levels of NHP students. The gap between NHP and non-NHP students reaching near mastery was
higher in1998 and 1999 test data at the sixth grade level. Differences at least three percentage points
greater were found in near mastery for both maths and language arts at sixth grade than at third grade,
when NHP and non-NHP students were compared.
Special interventions targeting children, especially boys, in small NHP schools are
required if the project is to address those most in need. Much higher percentages of male students
in small- and medium-sized schools fall into the category of no mastery of the curriculum than males in
large schools. This is especially true in regard to language arts. Both third grade boys and girls in small
and medium schools have less success in mastering the mathematics curriculum than children in large
schools. 
The provision of hands-on professional development and other technical assistance at the
school level has a positive effect on teacher performance. Although formal school visits, involving
professional development activities, were limited to slightly more than half of the NHP schools and to a
maximum of about a day in length, they were related to improved classroom environments as well as to
availability and use of materials. More than double the number of learning materials were available in
19
classrooms visited by NHP specialists and these materials were used with greater frequency by
students. 
Despite the work of NHP and core curriculum specialists, student participation is low. 
Students initiate only about 6% of the interactions with teachers and only slightly more than half of the
student population participate in lessons on a given day. Classrooms generally do not display children’s
work, do not have learning centers for children to use, seldom encourage children to express
themselves with peers and adults, and lack materials that show males and females in traditional and
non-traditional roles. In many of the classrooms, especially those in larger schools, the lack of space
contributed to a less than optimal classroom environment. The limited learning opportunities in the
physical and affective environment are exacerbated by a teacher-centered approach to instruction that
focuses on questioning and allows little spontaneous participation by students.
NHP has been successful in building elements of system support at the local level.
Virtually every indicator has had a positive change over the course of the 1999-2000 school year. All
schools have school development plans and most have PTAs that meet regularly. Over 80% of the
teachers have participated in professional development activities and the vast majority of schools have
resource teachers. More than two-thirds of the schools have administrative computers, although in
most installation and training have not yet taken place.
Despite teachers’ high participation in professional development activities, many are
confused about the specific mission of New Horizons and equate the program with the
implementation of the new primary school curriculum. A majority of the sample teachers identified
elements that the programs have in common, whereas only 7% of the sample teachers were able to
identify elements unique to the NHP program such as a focus on less successful children, and strategies
of mixed skill and age groupings aimed at increasing the participation of these children.
Training in the use of administrative software and maintenance of equipment are a key
issue to the success of management efforts in project schools. All of the principals in the sample
identified installation, training and maintenance of equipment as having contributed to less than optimum
implementation of previous projects. They stated that they had neither the financial wherewithal nor the
available local expertise to maintain computers, audio-visual equipment or copiers.
B. Implications
The impact of school visits by NHP specialists argues for increased hands-on support in
schools and classrooms in order to assist teachers to develop appropriate learning strategies within
local contexts. Such support could begin with an NHP retreat to determine strategies to maximize the
effectiveness of technical support. Given the relative lack of attention to medium and large schools by
specialists, a school level professional development plan that assures that the NHP student population is
fully attended should be put in place. This plan might include a certain number of schools being
20
assigned to a particular specialist for regular visits, with support from other NHP team members as
required.
The lack of student participation and the highly teacher-centered nature of NHP classrooms
suggest that professional development efforts should be highly targeted. This means that specialists
should follow the pedagogical precepts of the curriculum and interventions that they are trying to
implement and build on what teachers already do to change classroom practice. This might include
showing teachers that their focus on questions can be made child-centered by allowing children to work
in small groups to develop questions from assignments, then asking each other the questions that they
have created. Similarly, simply having each student put a stone on one side of their desktop and
moving the stone when called on increases student participation and permits teachers to monitor their
strategies to involve students and can help to ensure that all students participate. 
Teachers lack of knowledge about the specific targets, objectives, and strategies of the New
Horizons for Primary Schools project suggest that a concentrated effort must be made to increase
teachers’ knowledge of the program in professional development efforts. This should be possible as
part of hands-on assistance provided at the schools which provide teachers with materials and
strategies to assure that all students master the curriculum.
The differences in performance of students by grade level and school type implies that special
strategies may be necessary for different populations of students. Special emphasis should be placed
on the teaching-learning situations in the upper grades of primary schools. Similarly, qualitative
diagnostic research to determine the causes of lack of academic success among male students in small
schools should be carried out. Such research could be in the form of action research conducted by the
teachers and students together.
Installation of computers, training in the use of administrative software, and strategies for
maintenance of computers and computer environments must be carried out as soon as possible, if
school administrative capacity is to be improved. Similarly NHP must work with schools to develop
the means to maintain any audio-visual and administrative support equipment to be used for enhancing
instruction. Strategies for financing equipment maintenance should be made explicit in planning
documents.
21
IV. BIBLIOGRAPHY
Arboleda, J., C. Chiappe, and V. Colbert. 1991. “The New School Program: More and Better
Primary Education for Children in Rural Areas in Colombia.” In Levin, H. and Lockheed, M. (eds.),
Effective Schools in Developing Countries. Washington, DC: The World Bank.
Bellew, R. and E. King. 1993. “Women’s Education in Developing Countries: Barriers, Benefits, and
Policies.” In King, E. M. and A. M. Hill (eds.), Educating Women: Lessons from Experience.
Published for the World Bank. Baltimore, MD: Johns Hopkins University Press. 
Blum, Robert E. 1990. Effective Schooling Practices: A Research Synthesis. Portland OR:
Northwest Regional Education Laboratory.
Chesterfield, R. 1992. Basic Education: Review of Experience. Washington, DC: LAC Education
and Human Resources Technical Services (EHRTS) Project, Bureau for Latin America and the
Caribbean, USAID.
Chesterfield, R., and K. Enge. 1998. “Gender, Cognitive Categorization and Classroom Interaction
Patterns of Guatemalan Teachers.” Human Organization, Vol. 57, No. 1.
Christensen, P., A. Badra Doukoure, P. Laugharn, T. Moreau, J. Moulton, J. Muskin, and M.
Welmond. 1997. Kids, Schools, and Learning: African Success Stories: A Retrospective Study
of USAID Support to Basic Education in Sub-Saharan Africa. Technical Paper No. 56. 
Washington, DC: Health and Human Resources Analysis for Africa Project, Office of Sustainable
Development, Africa Bureau, USAID.
Chung, Fay. 1990. “Recent Developments in Research into School Effectiveness in Zimbabwe.” 
Paper presented at conference on Schooling Effectiveness: Cross-National Findings. Cambridge, MA:
Harvard University.
Dalin, P., T. Ayono, A. Biazen, B. Dibaba, M. Jahan, M. B. Miles and C. Rojas. 1994. How
Schools Improve: An International Report. School Development Series. London: Cassell Villiers
House.
DeStefano, J. 1996. Community-based Primary Education: Lessons Learned from the Basic
Education Expansion Project (BEEP) in Mali. SD Publications Series, Technical Paper No. 15. 
Report for USAID. Washington, DC: Africa Bureau Information Center, USAID.
Fuller, Bruce. 1986. “Raising School Quality in Developing Countries: What Investments Boost
Learning.” Discussion Paper No. 2. Washington, DC: World Bank.
Hawes, Mark., et al. (Eds.) 1990. Questions of Quality: Primary Education and Development. 
Essex, England: Longman.
22
Henevald, Ward., and Helen Craig. 1996. Schools Count. Technical Paper No. 303. Washington,
DC: World Bank.
Himmel, E. 1998. “Social Impact of Educational Performance Evaluation Systems: The Case of
Chile.” In B. Álvarez and M. Ruiz-Casares (eds.), Evaluation and Educational Reform: Policy
Options. Washington, DC: USAID.
Huberman, M. and M. Miles. 1984. Innovation Up Close: How School Improvement Works. 
New York: Plenum Press.
Lockheed, M. E. and A. Verspoor, with others. 1991. Improving Primary Education in
Developing Countries. New York: Oxford University Press.
Mallon, N. 1989. The Community Classroom. World Education Reports. 28: 15-18.
Restrepo, G. 1998. “The System for Evaluating the Quality of Education in Colombia.” In B. Álvarez
and M. Ruiz-Casares (eds.), Evaluation and Educational Reform: Policy Options. Washington,
DC: USAID.
Rosenthal, G. 1988. How Important is Sustainability in Evaluating Projects? Report for USAID. 
Arlington, VA: Resources for Child Health.
Rugh, A. and H. Bossert. 1998. Involving Communities: Participation in the Delivery of
Education Programs. Report for USAID. Washington, DC: Creative Associates International.
Shann, M. 1990. Making Schools More Effective: Indicators for Improvement. Boston, MA:
Boston University.
Windham, D. M. 1988. Indicators of Educational Effectiveness and Efficiency. Tallahassee FL: 
Educational Efficiency Clearing House.