PROJECT BEST 
EVALUATION
REPORT TO UNITED STATES AGENCY FOR INTERNATIONAL 
DEVELOPMENT, H ONDURAS MISSION
Richard Dye, Chief of Party
Barbara Hunt, Miriam Meza, Allan Broehl
Edward Kissam and Gilbert Valverde
Tegucigalpa, Honduras
JULY 20, 2000
AGUIRRE INTERNATIONA L PROJECT BEST EVALU ATION TEAM
THIS REPORT WAS PREPARED FOR USAID/HONDU RAS BY AGUIRRE INTER NATIONAL UNDER GEM 
TASK ORDER 801, CONT RACT NO. FAO - I- 00 - 99 - 00010 - 00. THE FIND INGS, CONCLUSIONS, A ND 
RECOMMENDATIONS ARE THOSE OF ITS AUTHORS A ND DO NOT NECESSARIL Y REFLECT THE 
VIEWS AND/OR POLICIE S OF USAID.
TABLE OF CONTENTS
EXECUTIVE SUMMARY...........................................................................................................................................................III
I. OVERVIEW, KEY FINDINGS AND RECOMMENDATIONS.......................................................................................... 1
A. SUPPORT FOR EDUCATIONAL REFORM ..........................................................................................................................1
B. ABEDS (NOW EDUCATODOS).........................................................................................................................................1
C. PILOT PROGRAMS.................................................................................................................................................................2
D. VOCATIONAL TRAINING WITH BASIC EDUCATION..............................................................................................2
E. LESSONS LEARNED AND CROSS-CUTTING ISSUES............................................................................................................3
II. BRIEF DESCRIPTION OF TASK........................................................................................................................................4
A. BRIEF HISTORY AND SUMMARY OF THE BEST PROJECT AND ITS GOALS ...................................................5
B. CHANGES AND ADDITIONS TO THE ORIGINAL BEST PROJECT.........................................................................5
1. ABEDS (now EDUCATODOS).....................................................................................................................................5
2. Vocation Training with Basic Education ..................................................................................................................6
3. Policy Reform and Project Promotion: Pilot Activities.........................................................................................6
III. EDUCATODOS......................................................................................................................................................................7
A. CURRENT STATE OF IMPLEMENTATION .......................................................................................................................7
1. Development of Educational Materials......................................................................................................................7
2. Project Management and Operations........................................................................................................................7
B. ACCOMPLISHMENTS IN RELATION TO PLANNED OBJECTIVES....................................................................................7
1. Overall Accomplishments..............................................................................................................................................7
2. Achievement of defined objectives..............................................................................................................................8
C. FINDINGS REGARDING KEY PROJECT COMPONENTS ..................................................................................................10
1. Curriculum and Materials. .........................................................................................................................................10
2. Delivery System-Audio Materials..............................................................................................................................11
3. Project Management and Operations.......................................................................................................................12
4. Financial Management............................................................................................................................................... 14
5. Learning Support Systems...........................................................................................................................................14
6. Systems for Programmatic Accountability............................................................................................................... 16
7. Collaborative Linkages and Partnerships .............................................................................................................18
8. USAID Role....................................................................................................................................................................19
D. PROJECT ISSUES TO BE ADDRESSED................................................................................................................................19
1. Tracking and Managing Student Progress and Achievement ...........................................................................20
2. Maintaining and Improving Instructional Qual ity................................................................................................20
3. Upgrading and Updating Curriculum Materials..................................................................................................20
4. INTEGRATION OF GRADES 1 -6 AND GRADES 7 -9 CURRICULUM.................................................................21
5. Resources to Enhance Student Learning .................................................................................................................21
6. the measurement of educational opportunities and outcomes, and valid achievement benchmarking .....21
7. Future Management and Implementation Mechanisms .......................................................................................23
8. Systems to a ssure accountability and maximize impact.......................................................................................23
E. LINKAGES, PARTNERSHIPS, AND SUSTAINABILITY ......................................................................................................23
IV VOCATIONAL TRAINING WITH BASIC EDUCATION............................................................................................24
A. OVERVIEW...........................................................................................................................................................................24
B. FINDINGS REGARDING KEY PROJECT COMPONENTS....................................................................................................26
1. Construction and equipping of centers .................................................................................................................... 26
2. Provision of relevant training and the development and use of the basic training modules.........................26
3. Provision of technical assistance and staff training to the centers.....................................................................27
4. Assistance on trade certification of the skill levels of students............................................................................27
5. increasing basic education levels through ACCESS TO ET and providing training in small business 
DEVELOPMENT................................................................................................................................................................27
6. data on student success in training and in the labor market and Management systems to measure 
efficiency/ensure accountability ....................................................................................................................................28
C. POLICY ISSUES TO BE ADDRESSED ...................................................................................................................................29
II
1. Adequacy of project design to address the skill training needs of low income, out-of-school youth and the 
needs of the labor market ................................................................................................................................................29
2. CADERH income sources and the provision of quality private sector oriented training services............... 30
3. Identification of critical training..............................................................................................................................30
D. LINKAGES, PARTNERSHIPS AND SUSTAINABILITY.......................................................................................................30
2. Plans of other international donors to assist in the development of vocational training programs ...........31
3.SUSTAINABILITY OF VOCATIONAL PROGRAMS, the Endowment Fund, AND USAID'S RELATIONSHIP 
WITH CADERH ..................................................................................................................................................................31
V. POLICY REFORM AND PROJECT PROMOTION (PILOT PROJECTS)................................................................32
A. OVERVIEW .........................................................................................................................................................................32
1. CURRENT STATE OF IMPLEMENTATION .............................................................................................................33
2. Accomplishments: Educational reform.....................................................................................................................33
B. FINDINGS REGARDING KEY PROJECT COMPONENTS .............................................................................................34
1. Support for Educational Reform................................................................................................................................34
2. Pilot Projects.................................................................................................................................................................35
C. PROJECT ISSUES TO BE ADDRESSED................................................................................................................................38
1. Support of Educational Reform..................................................................................................................................39
2. Pilot Projects.................................................................................................................................................................40
3. Management structure and implementation methods............................................................................................40
VI. CONCLUSION......................................................................................................................................................................40
Notes ................................................................................................................................................................................42
III
EXECUTIVE SUMMARY
The BEST project, which focused primarily on Alternative Basic Education Delivery 
Systems but also included an educational policy component, was successful in the 
majority of its components, but had mixed success in one, CADERH. 
EDUCATODOS, an interactive radio instruction program, has successfully build an extensive 
service network, reached its intended target population, met many of its learning goals, and clearly 
has had an impact on more than 200,000 out of school children and young people, as well as 
illiterate or barely schooled adults. Weaknesses in evaluation methods and a need for updating of 
program methods and materials are detailed in the report. The Team recommends a final 
period of gradually declining assistance, during which technical assistance would be 
provided to assist EDUCATODOS in strengthening its academic and administrative 
systems; during this time phased absorption into the Secretariat of Education’s regular 
structure would be accomplished.
The CADERH-coordinated vocational educational education network provides quality training 
services to low-income youth and young adults. It is currently being expanded through a contract 
with DevTech, Inc. CADERH itself, however, while its materials development, training, and 
certifications services represent real value added, continues to have serious management and 
financial problems unrelated to the project. It is an open question whether it will be able to 
resolve those problems; in addition, it is not clear whether CADERH will be able to become a 
sustainable, self-funding organization. The Team recommends that CADERH be informed 
that AID will provide final, tie-off assistance, provided that CADERH is prepared to 
undertake a series of actions designed to produce sustainability by the end of the grant 
period. Assuming CADERH qualifies, assistance would be terminated December 31, 
2001. 
After Hurricane Mitch, important work was done under the policy component of BEST to lay the 
groundwork for comprehensive educational reform in Honduras. USAID Honduras played an 
active and important role in helping create a consensus in favor of reform, supporting the planning 
and public discussion of reform plans, and supporting it through effective donor coordination. In 
addition, the Mission assisted several pilot projects, two of which utilize distance learning, to 
provide inputs to the anticipated reform program. A national educational reform effort has now 
been launched. The Team recommends that USAID give priority to supporting selected 
components of this effort.

1
I. OVERVIEW, KEY FINDINGS AND RECOMMENDATIONS
In this section we present a brief description of the key findings and recommendations resulting 
from our evaluation of USAID’s Basic Education and Skills Training (BEST) Project. 
A. SUPPORT FOR EDUCATIONAL REFORM
USAID has played a key role in providing support for efforts to reform the Honduran educational 
system. Shortly after Hurricane Mitch, a consensus developed among Honduran educational 
leaders and international donors that the disaster had created a rare opportunity to initiate a 
comprehensive reform program. This led to a series of actions, which culminated in the 
presentation and dissemination of a report outlining the reform. The process was accompanied by 
successful, informal cooperation among interested donors. By early 2000, these informal donor 
arrangements had evolved into a formally structured organization called the Mesa Redonda de 
Cooperantes en Educacion (Education Donors Roundtable), generally known as MERECE. 
USAID played a key role in the process of developing this valuable institution, which also meets 
regularly with a more recently formed counterpart group of Honduran educational authorities 
known as La Cumbre (The Summit).
USAID should be commended for its work to date in this area. We recommend continued 
work in support of reform, as well as an integrated package of programs to assist with the 
execution of key reform components. Our recommendations for specific programs to be 
considered will be found in our forthcoming assessment and strategy paper.
B. ABEDS (NOW EDUCATODOS)
The major component of ABEDS is EDUCATODOS (ET), a project using Interactive Radio 
Instruction (IRI) to provide basic education to out-of-school youth and young adults. In its broad 
outlines, this project has done what it set out to do. An extensive service network has been 
developed in 11 departments, and 223,897 participants have been provided with an opportunity 
not previously available to them to learn basic reading and mathematics skills. The program is 
known and respected. It has involved thousands of volunteer learning facilitators in helping build 
literacy skills in their communities. The program is obviously of great importance to participants, 
and has been successful in establishing a growing number of agreements with local municipalities 
and private industry. The existence of these agreements and the potential for many more bode 
well for the future sustainability of the project.
In spite of its apparent overall success, we found deficiencies in several aspects of the ET 
program. We believe that these deficiencies must be remedied. They include:
· A seriously outdated curriculum (radio broadcasts and texts) that is inadequate to prepare 
its participants either for future learning or for success in the labor force.
· A system of evaluation (both of student progress and of the program itself) which fall short 
of meeting basic standards of evaluation and research design, so that it is difficult to 
determine exactly what has been achieved by program participants.
· An overall focus on quantity rather than on service quality and flexibility in responding to 
the needs of individual learners and educational stakeholders.
2
We recommend that the ET program be continued, but only if a solid commitment is 
made to expend the time and money needed to improve its quality. The first priority must 
be to design and implement a new testing and evaluation system that adheres to accepted 
standards, so that reports on progress both to participants and to USAID will be more meaningful 
and reliable. Concurrently, it will be necessary to revise and upgrade instructional materials for 
grades 4 -6 and assure they adequately prepare students to successfully continue on to the new ET 
grades 7 -9 series. It will also be necessary to design and implement a process for ongoing revision 
and updating of curriculum and materials. Attention must also be given to quality in other aspects 
of the program, ranging from standard expectations for the quality and consistency of training and 
supervision of promoters and facilitators, to provision of adequate transportation to enable regular 
and careful supervision of the facilitators in remote rural areas. ET is a program that has clearly 
made an impact on the lives of its participants. With the necessary improvements, ET can 
become an outstanding program that truly prepares its students for successful participation in the 
labor force of the 21st century, one that can serve as a model for programs elsewhere. 
C. PILOT PROGRAMS
ESCUELAS CON EXITO (EE) was a pilot program designed to improve the quality of 
instruction and student learning in the first grade, in order to reduce the high rate of student 
failure and repetition in this key grade. School directors and parents were to be encouraged to be 
active participants in the program. Our review of the program indicated that the techniques
introduced were valuable, although they could be improved. A revised version of the program has 
now been adopted by the Secretaría de Educación and renamed “Salvemos Primer Grado”. 
Despite the name, its coverage is grades 1-3. Efforts are now being made to implement the 
program throughout the country. We will recommend that the program be among those 
considered for future assistance from USAID in its plans for the next five years.
APRENDAMOS MATEMÁTICA (AM) is a program designed to use IRI to provide mathematics 
instruction in the early primary grades. Our findings indicate that this program has generally been 
well received by school directors, teachers and parents, although not infrequently teachers were 
initially quite resistant to its incorporation into their daily program. We believe the program 
provides a useful supplementary mathematics experience in a system in which many teachers feel 
very uncomfortable with mathematics. With regard to sustainability, it appears likely that 
increasing levels of support can be generated at the local level from NGO's, municipalities, and 
the private sector. We recommend that continued support be provided to this program, 
with improvements as noted in the body of the report, and pending curriculum decisions 
in the context of the reform.
TELEBÁSICA (TB) is a program using educational television and texts purchased from the 
Mexican government to provide instruction in all key subjects for grades 7 through 9. Our review 
of the textual materials indicated that they ranged from acceptable to high quality. However, in 
order for the program to be successful, we recommend that teachers be given more training and 
continued supervision and coaching. An evaluation of TB is currently being conducted by IEQ-II. 
At this time it is unclear whether TB should be designated for use in formal programs or ABEDS 
programs, or in both. Given the number of grade 7-9 curricula currently available or under 
consideration, as well as the pending evaluation, we recommend that TB continue as a n 
interim program in Basic Education until evaluation results are received and final 
decisions are made in the SE regarding a standard grade 7 -9 curriculum for Honduras.
D. VOCATIONAL TRAINING WITH BASIC EDUCATION
3
BEST Project assistance was provided through CADERH (Comité Asesor para el Dessarrollo de 
Recursos Humanos) to locally managed and financed centers that provide skills training to low 
income, out-of-school youth. Training centers were built and equipped with project assistance 
under CADERH's administration. In addition, the project provided funds for CADERH to 
enhance its capabilities for providing centers with technical services and training materials.
CADERH's management of the project was poorly carried out. Project results fell far short of 
those anticipated and several of the buildings constructed under CADERH's management are 
seriously flawed. While CADERH has performed poorly, the training centers receiving assistance, 
along with many others in the CADERH network, are providing quality training services to low 
income populations and to the labor markets they serve. They operate with local financing, so 
sustainability is not in question. Facilities and equipment are well maintained, students are 
disciplined, teachers are trained in the use of competency based instructional materials, and school 
directors are dedicated to providing the best training they can with their limited funds..
We recommend that the current program managed by DevTech to expand the number of 
centers and to reinforce existing centers be implemented as planned. It would, however, 
be unwise for USAID to go beyond this level of effort without: (1) having an overview of the 
vocational training efforts of all Honduran institutions compared to estimates of the requirements 
for labor force entrants to ensure that training resources are contributing to national economic 
development as well as to the needs of low income, out-of-school youth; (2) engaging in 
longitudinal studies of the results of training for a sample of graduates from the training centers to 
measure the longer-term benefits of these programs as well as identify improvements that should 
be made; and (3) determining if technical support from CADERH or a similar organization will be 
available in the future to assist training centers in delivering quality services.
In spite of its poor management of BEST Component II, CADERH still fills a critical role in 
contributing value added to the training provided by the centers. This is achieved through staff 
training, re visions and distribution of training modules, provision of certification exams, and 
technical assistance. Training centers indicated that these support services are very important to 
them for the delivery of quality programs. CADERH's major weakness is its apparent lack of 
interest in ever becoming financially self-sustaining. There seems to be a general consensus within 
CADERH that USAID should be a continual source of funding, since the training centers that 
CADERH supports are built with USAID funding. During CADERH's 17 year history, USAID 
has provided institutional support to CADERH for nearly 14 of those years.
We recommend that USAID continue to provide institutional support for CADERH during 
the initial period of DevTech Project implementation. During this period, CADERH and the 
Mission should come to closure on what level of CADERH support is required to adequately 
serve the affiliated training centers, what sources of funding CADERH can expect to generate in 
addition to the Endowment Fund, and how the deficit can be addressed. A change in attitude on 
CADERH's part about attaining self-sufficiency will be the critical factor in determining whether 
this additional support, which should be the last, will enable CADERH to become self-sustaining. 
E. LESSONS LEARNED AND CROSS -CUTTING ISSUES
The team identified the following issues which were evident in most areas of the BEST Project:
· Needs for training and supervision to accompany changes. We note the need for additional 
training and continued support and supervision for those teaching in the various 
components of the project, whether they be teachers or facilitators. Related issues include 
4
the difficulties in providing transportation for effective supervision, and the need to 
involve school directors in provision of ongoing support, supervision and instructional 
leadership in their schools and communities.
· Coordination with other donors in assisting with specific key reform efforts without 
overlap of efforts. It is clear that an explicit organizational structure for donor coordination 
and collaboration has great promise as a mechanism for articulating a coherent vision and 
setting a steady forward pace toward educational reform.
· Should USAID work within the Secretaria de Educación (SE) or outside it? Our 
conclusion is that in order for USAID to maximize its strategic role in the educational 
reform, it must attempt to address some of the educational system's greatest needs by 
targeting a small number of strategic entry points. Possibilities in addition to those listed 
here include targeted assistance to the proposed new Unidad para la Transformacion del 
Sistema Educativo (UTSE), strategic entry points in the SE to promote effective 
classroom instruction at the pre-primary level and the early grades, strengthened student 
assessment and evaluation systems, as well as targeted staff training, and experimentation 
with new models for the use of technology.
· Curriculum, evaluation and program standards. We identified significant deficiencies in 
curriculum planning and materials development, in combination with low expectations for 
field staff’s compliance with program guidelines and commitment to specific program 
objectives. In addition, we noted inadequacies in evaluation research design and 
implementation that underrmine decision-makers’ ability to refine strategies, limit program 
managers’ ability to take corrective action and prevent teachers and facilitators from using 
the results of formative and summative evaluation to meet the needs of their students 
more effectively.
· The need for increased responsiveness to customers. An emphasis on responsiveness to 
customers as a guiding principle in educational service delivery has potential as a tool both 
for policy reform and improvement of service quality.
Please see Appendix A, Lessons Learned and Cross Cutting Issues, for additional detail.
II. BRIEF DESCRIPTION OF TASK
The BEST Project Evaluation Team was asked to determine if the project achieved its purpose 
and results and identify the factors that have contributed to its success or failure. The Team was 
tasked with noting key lessons learned as a result of the project and suggesting how these lessons 
might relate to a new Mission education strategy.
The following specific questions were posed to the Team about the ET program. Should the 
program for grades 1 -6 continue and, if so: (1) which interventions should be included at what 
funding level; (2) what management structure and implementation methods would be most 
appropriate; (3) what must be done to assure longer-term sustainability for the program; (4) should 
the program materials be revised, how extensive should the revision be, and how should these 
revisions be carried out; and (5) should the program for grades 1 -6 be integrated with the program 
for grades 7-9? Similarly, specific questions were raised with respect to the vocational skills 
training component of the project. These include: (1) should future support be devoted to 
vocational skills training; (2) is there a reasonable expectation that the BEST sponsored 
institutions will become sustainable without continued Mission support and is there anything that 
can be done to assure sustainability over the longer-term, and (3) if continued support is 
5
recommended under a new Mission education strategy, what should CADERH's role be in 
implementing that strategy?
Finally, the following questions were raised about the final BEST component--educational policy: 
(1) which, if any, of the current activities funded by this activity should continue under a new 
Mission strategy and what should their funding levels be, and (2) what management structure and 
implementation methods are most appropriate?
A. BRIEF HISTORY AND SUMMARY OF THE BEST PROJECT AND ITS 
GOALS
The project was developed in 1995 with the goal of "Enhanced economic participation and 
increased incomes of the poor through investment in people" which was synonymous with the 
Mission's Strategic Objective No. 1 and Program Outcome 1.3. The Intermediate Results 
attributed to the project were "Improved Basic Education" and "Increased Number of Vocational 
Center Graduates Employed." The project was designed as a five-year effort to expand access to 
high quality basic education and vocational training for out-of-school youth and young adults.
The project has three interrelated components. The first, which was to develop an alternative 
basic education delivery system for some 250,000 out-of-school youth and young adults, evolved 
into the ET program managed by a Project Implementation Unit (PIU) under the Secretaría de 
Educación . The program provides a flexible learning system based on a wide variety of learning 
sites and class times utilizing written materials, programmed texts, cassettes and radio broadcasts 
for learning groups led by volunteer facilitators.
Component II, implemented by CADERH, provides for the construction of 15 new non-formal 
training centers individually managed by NGOs or municipalities, 12 upgrades to training centers 
previously constructed with USAID resources under an earlier project with CADERH, and the 
development and distribution of competency based training modules as well as the application of 
certification exams by CADERH. By Project completion, 12,600 youth and adults were to have 
received vocational skills training in an estimated 27 training centers throughout the country. An 
endowment fund was created to permit CADERH to continue to provide technical services to 
training centers after the Project's completion.
Component III was designed to support educational research towards improving the equity, 
quality and efficiency of education and to disseminate the results of these and other program 
initiatives financed by BEST. The funds were included in the cooperative agreement with 
CADERH, but were subsequently brought back into the Mission when concerns over CADERH 
management surfaced. Later, some portion of the funds was assigned to ET, but these also were 
not used. From then until Hurricane Mitch struck, the only use made of Component III was for 
technical assistance run out of the mission.
B. CHANGES AND ADDITIONS TO THE ORIGINAL BEST PROJECT
1. ABEDS (NOW EDUCATODOS) 
Because of the devastation to schools caused by Hurricane Mitch, some BEST funds were made 
available to accelerate the development of the Grade 7-9 curriculum for ET, so that the needs of 
many unserved students could be met. This curriculum is being developed under a contract with 
6
IEQ-II. As a result of difficulties with CADERH management, ET has also assumed the 
management of the funds for pilot activities and research under Component III
2. VOCATION TRAINING WITH BASIC EDUCATION
The Cooperative Agreement with CADERH, signed on October 16, 1995, remained unchanged 
until early 1998 when payments to CADERH were suspended for several months because the
accounting system in CADERH's San Pedro Sula "Centro Modelo" was found inadequate. By 
mid-year, USAID payments to CADERH resumed but problems related to CADERH's inadequate 
administration of funds and poor project management continued to slow down the 
implementation of project activities. By the end of 1998, CADERH had only completed four of 
the anticipated 15 new training centers, plus a multi-purpose room for one existing center. Given 
that the project was to terminate on December 31, 1999, USAID and CADERH agreed, in 
December 1998, to the completion of four additional centers as well as to supply some additional 
equipment for several existing ones. There were many difficulties in completing these centers due 
to the post-Mitch construction boom when workmen and materials were hard to obtain and due to 
CADERH's problems in getting financial resources to the work sites on a timely basis.
As a result of Hurricane Mitch in October 1998, USAID/Honduras decided to provide funds to 
build 11 new training centers as well as upgrade some existing ones in order to expand 
opportunities for vocational training. Furthermore, the Mission wanted to increase the quality and 
effectiveness of the training centers through instructional upgrades, certification programs, and 
greater orientation and support for business development. Because of CADERH's poor record of 
project management, the Mission selected DevTech Systems Inc. to provide the above services. 
While the DevTech activities are not directly under CADERH's administration, they provide 
technical services and training materials to the centers and funds to CADERH to make these 
services available.
3. POLICY REFORM AND PROJECT PROMOTION: PILOT ACTIVITIES
In addition to the policy reform activities described in Section V, three pilot projects have been 
developed with funds from this component:
TELEBÁSICA (TB) is a major activity under Component 3, which operates under the 
management of a unit in ET. It is based on the Mexican Telesecundario Program, using 
educational television and texts to provide an entire program for grades 7-9. It is currently being 
piloted in 35 sites, and will be evaluated in the coming months under a contract with IEQII.
ESCUELAS CON ÉXITO (EE) was designed to improve student learning and thereby reduce 
failure and repetition in the first and second grades by improving classroom instruction and 
working with parents. The project was piloted in 39 schools in Francisco Morazán and Intibucá 
during the school year of 1998. Hurricane Mitch caused serious disruption to the project and all 
records in Initbucá were destroyed. The project was discontinued after the 1998 school year. 
However, the Ministry perceived this to be a successful program, and this year has initiated the 
process of incorpora ting it into first grade classrooms nationwide, under the name of “Salvemos 
Primer Grado”. Although the Ministry requested funding for the new program from USAID, no 
funding has been provided under the BEST Project.
APRENDAMOS MATEMÁTICA (AM) is a supplemental mathematics program for the first three 
grades using IRI. Its pilot activities bagan in the spring of 1999 using local currency managed by 
CADERH. The program is implemented by the Fundación para la Educación Ricardo Ernesto 
7
Maduro Andreu (FEREMA). The first grade program has now been introduced nationwide, and 
piloting of the second grade program is now taking place in 10 departments.
III. EDUCATODOS
A. CURRENT STATE OF IMPLEMENTATION
1. DEVELOPMENT OF EDUCATIONAL MATERIALS 
Alternative educational materials for grades 1-6 were developed in 1995-1996 as planned.1
However, development of materials for grades 7-9 was substantially delayed. IEQII began a pilot 
of the 7 th grade materials at one site on June 30, 2000. During the coming months, this pilot 
project will expand to about 30 sites (some with multiple classes). It is likely that most of the 7th
grade materials will have been piloted by the end of the year.
2. PROJECT MANAGEMENT AND OPERATIONS
ET is administered by the Program Implementation Unit (PIU), a management and program 
support team of educational specialists and support staff housed in an independent facility on El 
Picacho. This central unit is responsible for distribution of audio material and textbooks, as well 
as recruitment, supervision, and training of project departmental coordinators. Working with the 
coordinators, it is also responsible for training promoters, and learning facilitators., student testing, 
reporting, and program evaluation.. 
ET operates primarily in 11 departments of the country, a region with approximately two-thirds of 
the universe of need (youth and adults needing learning opportunities for basic skills 
development). In the current year service has been initiated in 4 additional departments in 
response to requests from municipalities, NGO’s, and private sector businesses. Audio materials 
are delivered via 16 radio stations and via cassette. Fourteen Departmental coordinators recruit 
and supervise about 180 program promoters who are, in turn, responsible for recruiting about 
4,500 learning group facilitators. 
B. ACCOMPLISHMENTS IN RELATION TO PLANNED OBJECTIVES
1. OVERALL ACCOMPLISHMENTS
ET has made impressive progress toward meeting its overall service delivery objectives with 
respect to learners served and has clearly made an important contribution to increasing basic 
reading and math levels in rural Honduras. Program piloting of materials and phase-in was 
systematic and the service delivery system continues to expand in an orderly way. The success of 
any distance learning program rests on the sponsoring agency’s ability to build and manage a solid 
learner support system to recruit students, train and supervise local learning center volunteers, and 
to work to find better ways to facilitate student learning. The project has successfully built such a 
system. ET’s learning support network is a valuable institutional resource for the future. 
For rural areas where working youth and adults have virtually no possibility of attending night 
school, ET represents an extraordinary opportunity to allow them to realize their dream of finally 
getting ahead in life—for older adults, learning to read, for youth, the first rung on an upward 
8
ladder of career mobility. 2
 ET is seen nationally as an initiative that has done what it set out to 
do. However deficiencies in the program’s internal monitoring and evaluation system make it 
impossible for the evaluation team to determine whether the specific impact objectives were 
achieved. We are also concerned about the quality of instruction and learning opportunities 
delivered by ET, due to the limitations in the design of the educational materials (text and audio). 
These materials do not conform to generally-accepted principles of instructional methodology for 
adult learning programs and, in terms of curriculum content, are not well-suited to achieve the 
Honduran goal of workforce preparation in the national strategy of poverty reduction. The design 
limitations of these materials have served to decrease the cost-effectiveness of an alternative 
educational delivery system that is otherwise sound.
The record regarding women's participation in ET is excellent. In general, slightly more women 
than men have participated in the program, and more grade levels have been passed by women.
It is well established that formal education of women in developing countries provides many 
additional benefits, such as increased years of schooling for their children, better nutritional and 
health practices, etc. Two studies have provided data on such issues specifically for women 
participating in ET classes.3
 Findings suggested several positive differences for women 
participating in ET. Although both men and women reported more participation in civic activities 
and voting in elections after attending ET classes, the difference was more marked for women. 
Children of women attending ET classes were more likely to remain in school, and positive 
differences were reported regarding several important health practices. These findings are 
important, and we recommend that units in the curriculum dealing with knowledge of AIDS, 
family planning, and general health care should be amplified, in order to strengthen even further 
the impact of ET studies in these important areas.
Women in ET compared with those in a control group reported reading and writing more outside 
of class; they were also more likely to discuss their course materials with family and community 
members. A particularly interesting finding is that ET mothers spent more time reading their 
children's homework. Therefore, one might conclude that the positive benefits accruing to 
women from participation in ET are similar to those generally reported for formal schooling of 
women in the developing world.
2. ACHIEVEMENT OF DEFINED OBJECTIVES 
ET has made excellent progress toward achieving its enrollment objective of 250,000 students. 
As of June 30, 2000, it had served 223,897 students. We project that by project end, it will have 
achieved about 98% of the initially projected cumulative enrollment 4
. At that point, the project 
will have achieved about 64% of its objectives in terms of grade-level graduates. However, 
because the project MIS does not track individual students it is not possible to determine the 
unduplicated number of students served (since it is likely that many students have re-enrolled for 
several semesters). 
ET has also substantially exceeded its objective of working collaboratively with at least 24 
NGO’s and 12 municipalities—with 29 formal agreements and about 30 informal ones. In 
particular, ET has apparently had notable success in developing collaborative agreements with 
business co-sponsors to offer workplace literacy programs.
Planning documents and progress review, at points, imply that the project was meant to provide 
alternative basic education for out-of-school Hondurans throughout the country. Currently ET’s 
“sphere of influence” is in the 11 western departments, but it has also responded to requests for 
9
learning groups in 5 additional departments (Atlantida, Comayagua, El Paraiso, Yoro, and 
Olancho). The northeastern Departments are served primarily by the SE's Adult and Continuing 
Education’s PRALEBAH Project. We consider this regional focus for both programs to be a wise 
implementation strategy and believe this “gentleman’s agreement” may have had a positive impact 
on implementation of both programs because it allows a better focus on quality implementation. 
It is likely that the program had a positive cost-benefit ratio since instructional costs per student 
were very low and it is reasonable to expect that even modest increases in earnings would accrue 
over a period of 10-20 years since the service population is so young.5
 However, we cannot 
reliably determine whether ET has succeeded in achieving the specified 150% increase over 1994 
levels in the aggregate ABEDS “standardized” scores because of deficiencies in the design and 
implementation of the testing system. Nor is it possible to accurately determine students’ earnings 
increases as a result of basic skills. 
ET has done an impressive job of recruiting volunteer facilitators to convene learning groups in 
communities throughout the service area. However, it is not possible to determine whether the 
specific objective of securing 15,000 volunteers over the life of the project has been achieved 
because the project’s MIS does not provide an unduplicated count of volunteers over the life of 
the project. More important than the total volunteers recruited, is the extent to which the project 
has minimized volunteer turnover as a way to preserve its investments in recruitment and training. 
We consider the program to have achieved the implicit objective of leveraging about 7,500 
volunteer-years of effort.6
Although ET was promoted as a flexible customer-oriented program, the instructional design 
permits few opportunities for adaptation of course content to respond to the individual learning 
needs of a study group, in part because text and audio materials are highly programmed and 
integrated, in part because of the design of the overall instructional “package”. However, it 
would be possible to make local adaptations to the program by adding new facilitator-guided 
learning activities for learning groups. Nor is ET a true “open entry-open exit” system. Students 
can, as was originally envisioned, be placed into the system at any grade level. But they generally 
cannot progress at their own pace since learning is synchronized to a set schedule of 100 radio 
broadcasts over the course of a standard semester. 7
 However, where material is available on 
cassette there is more flexibility.
With respect to its primary, over-arching goal, i.e. contributing to the development of the basic 
skills needed for a productive Honduran workforce, the ET accomplishments are mixed. More 
than 160,000 ET students probably made gains in reading and arithmetic competencies.8
 
However the overall instructional design and course materials do not provide a adequate basis for 
basic sustained basic skills growth. Contemporary frameworks for adult learning, including those 
articulated in a wide range of Honduran educational policy documents, consistently emphasize the 
importance of sustained lifelong learning. This requires developing flexible “foundation skills” in 
analytic thinking, practical problem-solving, mathematical reasoning, use of information resources, 
teamwork, leadership, and understanding systems. 9
 These “higher order” basic skills are 
considered critical in preparing learners to respond to the demands of a work place in a business 
competing in a global economy, for participating meaningfully in a democratic society, and for 
effectively carrying out a self-directed plan of lifelong learning. 10 Redesigned ET instructional 
methods and materials would contribute much more to the development of these competencies. 
ET’s circumscribed delineation of the role of volunteer facilitators has not yet allowed it to 
become a “student centered” program as was originally envisioned. It will be necessary for the ET 
10
project to reconceptualize the instructional methodology, curriculum framework, and the 
strategies used by facilitators to assure the development of the basic skills needed for: 
· moving well-prepared students (many of them still teenagers) into the vocational education 
system or allowing access to secondary and university-level education 
· sustaining learning and skills development throughout a lifetime of technological change and 
escalating workplace demand
· realizing their potential for contributing to local community development, institutional 
strengthening and participating fully and effectively in a democratic society. 
The design of the ET instructional materials reflects a consensus that the ET Level 1-6 adult 
learning materials should be consistent with the corresponding elementary school system 
framework as articulated in its official Rendimientos Basicos framework. We believe this decision 
should be rethought.
C. FINDINGS REGARDING KEY PROJECT COMPONENTS
1. CURRICULUM AND MATERIALS. 
The ET package consists of two coordinated sets of materials—the audio material and the student 
workbooks for grade levels 1-6. The format of the distance learning package provides a 
standardized way to build learning group activities around the radio broadcast (or cassette 
playback). The instructional package design represents an enhancement on an instructional model 
consisting solely of radio broadcasts but does little to fulfill the promise of “interactive radio 
instruction”.
The ET materials package and instructional design fail to provide a sound basis for building out￾of-school youth and adult learners’ competencies in communication, mathematical reasoning, 
social and civic relationships, and understanding of natural systems. The strongest themes in 
terms of adult learning needs are those relating to personal health, public health, and 
environmental protection. Perhaps the most serious shortcoming is that the materials are not 
adult-oriented in terms of tone, style, or content. The overall tone of the material is bland and 
patronizing, sharply out of touch with the harsh realities and day-to-day challenges faced by 
learners living and working in poverty. 
The linkage of course content and learning objectives to the official Honduran Rendimientos Basicos
framework for primary education is not necessarily a problem (because linkage does not mean 
simply replication) but, in the context of ET it becomes one because the Rendimientos Basicos
curriculum framework has not been interpreted in terms of adult learning needs..11
ET's lack of support for active learning is a problem for both practical and educational reasons. 
Without an instructional design which encourages student interaction and collaborative learning 
and activities which engage them in trying out and extending their skills, adult learners will not 
spend much time on task beyond the 45 minutes a day of “class time”. To make real gains and 
master newly-introduced concepts and ideas they must be engaged in skills-building in every facet 
of their lives. Also ET's lack of strategies to engage learners in sustained skills-building does not 
provide its students with the skills needed for them to become effective, self-directed learners.. 
Another reason why active learning is critical and ET’ emphasis on passive behavior so 
problematic is that the 21st century workplace is one which places a premium on what might be 
11
called “generative” literacy—proactively setting out to find information for solving the problems 
one faces and communicating solutions to co-workers.
The ET instructional design and materials should provide out-of-school youth and adults with 
opportunities to engage in challenging, and enjoyable learning 12. Despite the cost of redesigning 
ET instructional design and curriculum materials, we believe this task is an essential one because 
otherwise the goal of bringing an illiterate and semi-literate population of youth and adult learners 
into the world of a 21st century global economy will not be met.
2. DELIVERY SYSTEM-AUDIO MATERIALS
ET uses two delivery systems for distributing audio material—broadcast radio and delivery of 
cassettes by ET drivers to departmental coordinators, who then make them available to 
promoters, and then to facilitators. About 95% of learning centers rely on radio broadcasts and 
5% on cassettes.13 There are advantages and disadvantages to each of these delivery systems. 
Broadcast radio is cost-effective.14 ET did very well in this aspect of operations, securing steep 
discounts from commercial and NGO-owned stations airing the programs (an average of 30% in 
the agreements we reviewed). These agreements also included donated promotional time. This 
time was used innovatively to recruit students and encourage them through the course of the 
semester. Radio is particularly important as a way to deliver daily lessons to remote areas. ET 
wisely involved departmental coordinators in identifying the best stations to reach learners in their 
service área--taking into account station popularity, signal coverage, and signal quality. At the 
sites we observed, reception was good, and the students and facilitators we talked to reported no 
problems except for infrequent outages when a lightning strike to a transmitter temporarily puts a 
station off the air. Broadcast schedules are also tailored to come up with the best time slots for 
each area. The disadvantage of relying on broadcast radio is that it is not possible for groups to 
proceed at their own pace and it is not usually possible to deliver all grade levels to all areas 
(although in areas which receive two broadcast signals, it is possible to deliver five or six grade 
levels during the 2 -hour slots from 4PM to 6PM).
Distribution of lessons on cassette has the great advantage of making it possible for groups using 
cassettes to schedule classes at the time that best suits their participants whether or not that 
conforms to the broadcast schedule. This was a critical need for the workplace-based program we 
observed in San Pedro Sula and was, also, an important consideration in La Lima where the 
municipal corporation agreed to provide funding for cassettes and players in order to make the 
program accessible at a later time than the regular ET broadcasts—7PM—since workers in the 
urban areas work later than in the rural areas. Facilitators in cassette-based programs also were 
able to stop the tape if things were moving too fast for participants (e.g. while students finished 
writing a sentence which was dictated to them as part of the lesson). The disadvantage of 
cassettes is that they are sometimes not delivered, they are lost, a few may break, or local funding 
for cassettes or players runs out or is not available. However, since ET maintains an extensive 
learning support network and must deliver texts to departmental coordinators, the marginal cost of 
increased distribution of cassettes would be minimal. ET strategy for delivering audio materials to 
learning groups is a sound one and we believe the current mixed-mode distribution system should 
be continued. 
12
3. PROJECT MANAGEMENT AND OPERATIONS
a) Staffing. ET’s basic management structure is well-designed and implemented. In particular, 
the scaling of the overall service delivery system is appropriate. The supervisory ratio of 1:15 
persons at each level of decentralization is one which generally allows the time and networking 
needed to develop solid working relationships at each level. The fiscal management contractor has 
lead responsibility for contracting with outside vendors and agencies. This seems a wise 
delineation of responsibilities, allowing PIU staff to focus more on issues of supervision, training, 
and reporting.
AT THE PIU LEVEL, we saw evidence of collaborative teamwork and good communication 
among central management staff. The planning devoted to training at all levels of the project is 
impressive; moreover, there are ongoing efforts to “customize” training to respond to the learning 
needs of individual staff. We were also impressed with the results of semi-annual project-wide 
self-assessments which elicited a wide range of productive suggestions. Project management also 
follows sound principles requiring from each coordinator a monthly workplan specifying 
objectives, resources needed, and persons assigned to any collaborative initiatives. At the end-of￾month meetings coordinators report accomplishments in relation to those planned, problems 
encountered, and corrective action taken. The project is also beginning to stress peer networking 
among departmental coordinators. Overall, the project is working very systematically and 
intelligently toward developing a functional, semi-decentralized system. 
AT THE DEPARTMENTAL LEVEL, we saw good working relationships between ET 
coordinators and external collaborators such as local “mainline” Departmental Directors and staff, 
private sector employers, and municipal representatives. We also saw good working relationships 
with promoters. Our reservation regarding project management at this level is simply that we did 
not get a sense that the coordinators have yet achieved the management skills to systematically 
engage in ongoing problem-solving, and to budget their time carefully among different competing 
demands
AT THE MUNICIPALITY LEVEL, we saw promoters working effectively with both facilitators 
and with collaborating organizations 15. We were impressed by the energy, dedication, and ease 
with which promoters moved in the community and observed that most seemed well qualified 
educationally and personally for their roles
AT THE SITE LEVEL, we saw facilitators who were, indeed, respected in their communities and 
assiduous in fulfilling their important roles. Not surprisingly, it is at this level where we saw the 
greatest level of variability in terms of individual competency. We saw extraordinarily good 
facilitation, for example, in the worksite program in San Pedro Sula (since the co-facilitators were 
university-trained staff from the company’s human resources department) and much less 
impressive facilitation in a remote hamlet in the San Juan area of Intibucá. Training will need to 
continue to enhance the facilitators’ ability to help learners.
b) Other Aspects of Operations. Transportation for field supervision staff to go to rural areas is 
an endemic problem—not just in ET but throughout the Honduran education system. ET has 
been wise in providing motorcycles to its promoters and our sense was that these were being 
deployed fairly well at the local departmental level. However, the combination of 6-months 
delays in payments to promoters and lack of budget for transportation for those who do not have 
access to a motorcycle is a disastrous one since promoters responsible for supervising 15-20 sites 
are not likely to achieve this objective under these conditions. Standardized reimbursements to 
13
promoters for transportation are used successfully in the PRALEBAH program and might be 
useful in the ET context.
Reliance on PMA food deliveries as an incentive to facilitators is not the ideal way to support 
them. We heard that the food (which is delivered twice per year) is sometimes spoiled—with 
maggots in the flour or rotting beans. Some of the food items (notably, sardines) are not popular, 
however nutritionally worthwhile they may be. The twice a year delivery of heavy food packages 
to remote areas is a real logistical burden. Modest cash payments instead of food would be a 
much sounder operational way to support volunteer facilitators.
We are aware that some departments have and use an MIS system which, at least, provides 
contact information on facilitators, a listing of sites, enrollment on each site, and student names. 
But this MIS, a tool which is potentially useful as a means for building more systematic regional 
and local service delivery, is not being used routinely by all departmental coordinators and should 
be enhanced to be still more useful. A serious MIS shortcoming is that ET does not have a 
systematic way of tracking the value of cash and in-kind donations provided by collaborating 
organizations. It would be desirable for ET to track these at the local level—both as part of its 
commitments under the USAID agreement and in preparation for increasing project self￾sufficiency. Systematic fund-raising initiatives typically benefit greatly from empirical data on the 
level of past donor contributions, favorite projects, particular concerns, etc. Such systems should 
be in place at the departmental level.
Finally, placement procedures for ET students pose operational problems as well as instructional 
ones. Our understanding is that persons interested in enrolling in a course are asked the last 
school grade they attended. They are then asked to secure a certificate that they completed/were 
approved at that grade level. If necessary, ET staff helps them get this certification. This is a 
waste of time and distraction for both program a pplicants and staff. Self-report of grade level 
attained would be adequate and would help both students and staff to focus on higher priority 
issues.
c) General Assessment of EDUCATODOS Management and Operations. ET is generally well￾managed. Decentralization seems to be working well for ET but there remains a good deal to be 
done to strengthen service quality management, and to make operations more efficient. Project 
management’s heavy investment in high-quality “train the trainer” staff development is a wise and 
promising area of emphasis. The next step will need to be to give closer attention to the quality of 
implementation by departmental coordinators. We believe the skills and potential of facilitators 
and promoters are still under-utilized and could make a greater contribution to overall program 
effectiveness. The facilitators we talked to expressed eagerness to receive further training and a 
willingness to work longer after radio broadcasts to tutor, encourage, and coach learners.
Field staff (coordinators, promoters, and facilitators) play a key role in raising project matching 
funds. They currently do well in this area—facilitators by assuring that “customers” are happy, 
promoters by responding to needs for service, and coordinators by making final agreements. 
However, because this is such an important function, it will need to be managed more 
systematically in the future.
We were impressed by the project management efforts to improve overall project functioning by 
sponsoring program-wide self-assessment retreats and in initiating a process of strategic planning. 
The project’s self-assessment addressed a number of bottlenecks in a productive and proactive 
way and we believe the overall emphasis on more flexibility and accountability is a promising one. 
14
We believe the project’s strategic planning will have to look further into the future, work further 
toward articulating strategies, and engage in a more systematic and quantitative analysis of 
strengths, weaknesses, opportunities, and threats, but what has been done so far provides a solid 
foundation for future efforts.
4. FINANCIAL MANAGEMENT
ET financial management appears to be sound. We have reviewed the budgets and records of 
expenditures of USAID funds for 1996-2000. Expenditures appear reasonable—although 
consistently lower than budgeted. There are, however, difficulties in fully assessing the extent to 
which financial management contributed positively to overall project cost-effectiveness. One 
problem is that budget line item categories are too broad to yield meaningful insight into the 
justification for key operational decisions or guidance in making them16. Another problem is that 
there is not a composite format linking together all funding sources and all expenditures—USAID 
funds, Secretaría de Educación funds, and locally-donated in-kind from municipal government, 
private business, and NGO’s17. While USAID projections regarding sustainability established 
budget targets for 3rd party support from NGO’s, private sector firms and donations from 
municipal government, these cash and in-kind contributions are not tracked.18 Even though it is 
difficult to project in-kind and cash donations for this sort of project, it would be wise to establish 
budget revenue targets and book actual in-kind and cash donations as revenue and expenditures.
We inquired into the causes of underspending and were told that there are some problems, in that 
USAID funding is seldom approved before April or May, although the Secretaría de Educación ’s 
fiscal year is a calendar one. However, the delay in receiving USAID authorization for current 
spending seems not to be a significant problem, because funds from the previous year can be 
carried over to support the first months of operations in the new calendar year. We suspect this 
may, however, exert some downward pressure on spending during at least the first quarter of each 
year as project managers tend to be nervous about this sort of situation.
The more serious problems are those which stem from the Honduran government’s cash flow 
problems. We were told that there are seldom delays in budget authorization and appropriation 
but that funds are simply not available in a timely fashion from the Department of Finance. 
5. LEARNING SUPPORT SYSTEMS
All distance learning systems make heavy demands on the learning support sub-system of the 
overall educational service delivery system, whatever technology is used. ET is no exception. In 
particular, promoters’ and facilitators’ work is crucial to learning success. In many ways, the use 
of broadcast technology such as radio is simply a way to structure group and individual learning 
activities which are then catalyzed and moved along by facilitators and promoters. ET learning 
support systems are adequate but that they should be improved. Project management has been 
proactive in making heavy investments of staff time and effort in providing training oriented to 
further improve the network of staff and volunteers who are charged with service delivery.
a) Student Placement. Placement procedures should generally be considered as part of learning 
support because the idea is to put the learner in the learning environment where he or she will do 
best. ET placement procedures rely on educational attainment (last grade attended) for 
placement. This is a problem because educational attainment is known to correlate very poorly 
with actual competencies and knowledge in the out-of-school population of youth and adults. 
Thus, since no placement test is given, there is the likelihood that a student will be placed above 
15
or below the “correct” level.19 However, we were pleased to see there has been training on 
informal assessment of student progress.
We observed that completely illiterate beginners at the 1 st grade level had particular difficulty in 
keeping up with the pace of lessons. Consideration should be given to the provision of a special 
preparatory level for students who have never before been to school. Since these students have a 
higher drop out rate, more attention to their learning needs could contribute to their initial feelings 
of success, possibly encouraging them to stick with their difficult task. 
b) Learning Group Facilitation. We do not agree with the ET stated premise that “facilitators 
should not act like teachers”. We do agree that the learning environment should not be teacher￾centered but, instead, a cooperative learning situation, with high levels of student interaction and 
active participation by all in the group. Our observations of facilitators was that some did 
extremely well and others much less well in carrying out their roles in support of learners. A key 
task for them is simply to help the group through the activities of any given lesson. Our 
observation was that several groups had learners at very different functional levels. This is a 
challenging situation for even experienced teachers so we believe it will be essential to provide 
more training to facilitators on how to observe and track different students’ ability to function in 
different areas (e.g. math, writing, reading) and how to provide individualized followup tutorial 
assistance. 
We observed situations in which facilitation took place with a team of two co-facilitators and 
were told that this model is fairly often used. This is an excellent solution to the problem of 
assisting facilitators in their role of supporting students—since it provides peer support, as well as 
making a substitute available if a facilitator must miss a class, etc
In general, learning facilitation can be strengthened by providing further guidance to facilitators to 
help them understand what the specific learning activities in a lesson, or a cluster of lessons, are 
“about”, that is, what skills the activities are designed to strengthen and where the learning 
program is headed next. Typically, in integrated distance learning program packages, one means 
of providing this guidance is in a facilitator’s guide organized to provide insight into the overall 
structure of the curriculum. Such a resource package might also provide ideas about what to do if 
a transmitter were off the air, how to develop out-of-class projects and assignments for students, 
and how to encourage peer support for self-directed learning.
c) Learning Tools-Blackboards, Writing Tables, Reading Materials. Throughout each lesson, 
some students are left with unfinished copying or mathematics tasks. Facilitators should be 
provided with blackboards so that mathematics problems or other tasks announced over the radio 
would be remembered, and could therefore be done when the lesson is over. (Obviously, 
blackboards would have many other practical uses, providing extra teaching opportunities as well 
as opportunities for participants to read and write.)
It is impossible to build reading ability without reading materials. Such materials are crucial for 
several reasons—to encourage students to spend more time reading, to give new readers a chance 
to see diverse text formats in different kinds of print materials (newspapers, magazines, fiction, 
poetry, manuals, reference books), and to encourage self-directed choices of what to read and 
when. Such efforts to promote sustained reading are a critical element in workforce preparation.
It is just as difficult to build writing skills without encouragement and respect for writing. 
Assignments to students to write short personally-produced texts and share them with fellow 
16
learners are needed. Ideally the ET curriculum should be changed to include such writing 
assignments, but even before the curriculum is changed, such assignments could be added by 
facilitators. Student writing might then even be published in a newsletter circulated throughout 
the ET system, thereby recognizing the value of skillful self-expression.
d) General Assessment of Learning Support Systems. The current learner support system provides 
a necessary foundation for student learning. There are two important constraints on this system’s 
ability to fulfill its function—inadequate appreciation of facilitators as resources and inadequate 
investment in ancillary materials to support learning. Investments in these areas need not be 
costly and can be assured to improve cost-benefit ratios. Further development of ET learner 
support systems are necessary to make the model one of genuine “interactive radio instruction”. 
The over-arching objective should be to promote active learning and informal, ongoing, self￾directed learning among students.
6. SYSTEMS FOR PROGRAMMATIC ACCOUNTABILITY
In this section, we assess ET systems for accountability, including certification of student 
competencies, reporting, and estimates of project impact. We consider the current systems for 
accountability acceptable with respect to reporting on project activities but not so with respect to 
estimating educational impact. Given USAID’s emphasis on outcomes, not simply inputs, we 
consider the lack of adequate systems for accountability a serious problem. 
a) Tracking of Activities. ET has an MIS which is designed to track the following aspects of 
ongoing service delivery activities: aggregate student enrollment by site, department, and 
nationally by gender, current roster of facilitators and sites, current roster of promoters and area 
assignments, and attendance sheets for individual sites. These systems provide a sound basis for 
reporting regarding enrollment, level of effort, and number of sites. However, we have not seen 
any viable system for generating system-wide attendance data although, in principle, such data 
would be available if existing attendance sheets were tabulated by hand. 20
Sound management of educational programs requires ongoing attention to the impact of the 
program on different sub-groups of students. Such information is a crucial program management 
tool because it provides a basis for service quality management initiatives to improve educational 
equity. The current MIS is not designed to generate reports of basic student characteristics by 
site, department, and nationally. It should, because this could provide invaluable information for 
accurately assessing program impact—since benefits are likely to be different for different sub￾groups of students. In order to fully understand the nature of project impacts, it would be 
necessary to engage in at least two types of research: a) analysis of the experience and learning 
gains of a random sample including sub-groups of learners, and b) longitudinal tracking of a 
sample of students over several years (to examine and better understand outcomes). Neither is 
feasible without a student MIS which contains individual records for uniquely identified students. 
Although this sort of system was envisioned in the original BEST project paper, it was not 
developed.
b) Assessment of EDUCATODOS’ Educational and Economic Impacts. The most serious 
shortcoming of the ET system of accountability is that, although the MIS can aggregate reports of 
student enrollment and students certified as successfully completing a grade level, the system of 
testing and student certification does not generate valid data for making these reports meaningful 
performance indicators or for estimating the impact of services provided—either in terms of skills 
development or likely economic impact.
17
Because the project relies on only one prime indicator of project performance, it is particularly 
critical for this to be a valid indicator because overall project management is driven by this 
indicator. In the case of ET, the use of this performance indicator may have generated 
complacency. Also, because of the focus on just this one indicator, ET has not taken into account 
other dimensions of positive program impacts. Although measuring other dimensions of impact 
are technically challenging, this sort of research provides additional valuable guidance for fine￾tuning program operations21
c) Technical Deficiencies of the EDUCATODOS Testing and Certification System. The lack of 
depth in the curriculum (i.e. questions seeking only factual responses drawn directly from text or 
preceding instructional material) is mirrored in the items used in testing – they are exclusively 
multiple-choice items with 3 distracters and 1 correct response. Students are not stimulated to 
generate solutions; they are evaluated solely on their ability to identify a correct solution from 
options generated by the test developers who, in turn, appear to have simply pulled items from the 
course materials. There are no ‘higher order skills’ being assessed22. Also, despite the assertions 
that the ET testing system consists of criterion-referenced tests, the system has not followed valid 
procedures for developing and using criterion-referenced tests. Some of the observed technical 
flaws that should be addressed are:
d) Non-definition of achievement domains. Technical documentation on test development in 
PEEP / BEST is thin, but it appears that testing is based upon the Rendimientos Básicos ,which do 
not constitute technically valid definitions of achievement domains.23 There are a number of 
possible options for addressing this problem. The ET evaluation methodology proposes to 
evaluate the higher order skills in the Rendimientos Básicos, but, at the same time, assumes that they 
are such simple skills that they do not merit elaborate development techniques. Following 
accepted standards in the definition and delimitation of the domains to be measured, and proper 
item specification, development and selection techniques are necessary conditions for valid 
testing, not unnecessary 'elaborateness'. Lacking the technical work to define and delimit the 
domains to be tested, and valid procedures of item specification, development and selection –
valid inferences regarding learner achievement in these domains are not possible.
e) Lack of valid samples of items to assess domains. ET has opted to include only one item 
related to each ‘rendimiento básico’ in their tests. This is problematical. In modern test theory, 
each item used to assess knowledge or skill in a domain is understood to be systematically tested 
itself and chosen from an infinity of items that could, in principle be used to measure the 
domain.24 Items in a test are samples of the various behaviors that the test developer has chosen 
as indicators of mastery of a particular domain25. Having only one item per domain does not 
provide students with an opportunity to demonstrate what they know or do not know26
. 
Assessing a soundly defined achievement domain requires from 3 to 5 items. As it stands, none of 
the "rendimientos básicos"– even if they were properly operationally defined achievement 
domains – can be said to have been measured validly in ET tests. It is maintained that sampling 
of behaviors is not necessary because of the simplicity of the areas assessed, and that if a student 
can select the appropriate answer from four options once, they have provided sufficient evidence 
that they have mastered that content or skill. There is no substantiation in educational 
measurement literature for such a claim27
. 
f) Non-Conformance to Professional Standards. ET testing departs from recognized 
professional standards28 in a number of important ways:
18
· ET does not collect or report evidence of test validity. This is evidence regarding the degree 
to which inferences made from these tests can be supported. There is a tendency to confuse 
validity with reduction of test difficulty .This diminished quality of its measurement efforts.
· ET similarly does not attend to the accumulation and assessment of data regarding reliability, 
a necessary but not sufficient condition for test validity.
· ET does not report data regarding the standard errors of its measures; information concerning 
which helps determine which inferences can be validly made from its data. It is essential to 
have these data.29 From a practical program management perspective, theyare necessary in 
order to compare the performance of different operating units or different sub-groups of 
program participants.
g) Flaws in the Evaluation Research Design for Interpreting EDUCATODOS Outcomes as an 
Impact of Program Participation. Comparison of pre- and post-test scores is a standard strategy 
for assessing program impact or learning. The difference between scores is considered to 
represent the “value added” to the knowledge base and/or skills inventory of program 
participants. However, in order to draw such conclusions, it is necessary to validly measure not 
just outcomes but also pre-program knowledge and/or competencies. Because ET students are 
not always given a pre-test when they enroll in the program but are, instead, deemed to have the 
competencies associated with the grade level they last attended, no valid estimates of “value 
added” could be made even if the post-program testing system were sound.30 Because the tests 
used to determine the educational outcomes for each level are, strictly speaking, invalid, it is, for 
example, difficult to determine the “value added” for a participant who has completed the ET 2nd
grade course and then moved on to complete the 3rd grade course. It is also important to consider 
the full range of program benefits, including non-economic ones. ET research on these kinds of 
benefits has moved in the right direction but further attention should be given to analyzing such 
data in terms of formative evaluation implications and disseminating such findings to guide 
practical efforts by coordinators and promoters to improve instructional quality. 31
h) Consequences of the Flawed System of Accountability. The consequence of the weaknesses in 
the ET system of accountability is that the project has not been generating the kind of information 
it needs to assure educational service quality. Testing systems such as those used by ET for 
reporting program performance to funders such as USAID are most cost-effective when they can 
be used both to make reliable decisions about program changes and when they generate the best 
possible evaluation results to guide efforts to strengthen the program.32
7. COLLABORATIVE LINKAGES AND PARTNERSHIPS
ET has done an excellent job in establishing collaborative linkages and partnerships. The project 
has exceeded its objective of establishing partnering agreements by at least 20% and more than 
11,000 enrollees have benefited from these collaborative working arrangements. Cash and in-kind 
contributions include the following sorts of support: bonuses and incentive payments to 
facilitators, financial support for radio transmission of lessons, broadcasters' in-kind donations of 
air time, cassettes, educational materials, transportation support and per diem for training of 
facilitators, student notebooks, motorcycle maintenance, purchase of radios and cassette players, 
and collaborative assistance in setting up learning centers. Our understanding is that these 
agreements are quite stable and that most of those which have been established are still in place—
although there are some instances in which a collaborating organization (e.g. a municipality) has 
difficulty in keeping its commitments.
19
We were particularly impressed by the extent of support from municipalities and business. Almost 
half of the agreements (44%) are with municipalities and 10% are with businesses. The number of 
formal agreements somewhat understates the extent of support because there are more than 30 
additional donors of cash or in-kind identified. 
The collaborative partnerships developed by ET are likely to have made significant progress 
toward the initial objectives established in the USAID project planning documents regarding 3 rd
party support.. 
8. USAID ROLE
In addition to being the primary funder of the ET project, USAID has played an important role in 
the development of the project. In terms of project management, an important accomplishment of 
USAID has been to arrange for support to be provided to ET by using Fernandez, Fortin, Bogran 
and Associates. This has stabilized project management and allowed the PIU to focus on 
operations rather than fiscal issues. USAID has also been proactively involved in efforts to 
overcome the chronic problems with payments to promoters (due to these funds being part of the 
counterpart contribution).
From a technical perspective, USAID involvement has played an important role in focusing ET's 
attention on educational outcomes. Constant attention to educational achievements (expressed as 
numbers of students successfully completing at least one grade level—“graduates”) was well￾placed as were efforts to empirically examine the reasons why participants dropped out of the 
program.
However, USAID’s technical assistance in program evaluation and assessment of student progress 
has not been adequate and is a significant factor in the project’s inability to demonstrate the 
educational outcomes and economic benefits of program participation. Moreover, the focus on a 
single indicator of program success (i.e. certification of grade level completion and maximization 
of proportion of completers) has distracted project staff from what should have been the top 
priority, “diagnostic” program evaluation efforts oriented toward generating practical strategies to 
improve service quality and service delivery.
USAID’s attention to radio distribution of audio material is well-justified and this component of 
the project seems to be functioning smoothly and cost-effectively. However, it would have been 
valuable for a similar degree of attention to have been given to development of high-quality 
instructional materials despite the urgency of getting ET into the field. This is a matter of current 
concern because the Grade 7-9 materials are being developed on a very rapid timeline and it is 
possible that they will not fully benefit from current insights into how best to provide accelerated 
learning opportunities for educationally disadvantaged populations 33
D. PROJECT ISSUES TO BE ADDRESSED
 In the following section, we summarize priority issues which need to be addressed in subsequent 
strategic planning regarding the directions in which ET should move. We propose to address 
these issues in more concrete details in the project strategic planning document.
20
1. TRACKING AND MANAGING STUDENT PROGRESS AND ACHIEVEMENT
ET does not currently have an MIS based on individual student records as had been envisioned in 
the original BEST planning documents. It will be necessary for ET to develop a student tracking 
system based on individual student records. This is required in order to generate solid data on 
student achievement and to empirically examine the critical issue as to how far students progress 
through the system, i.e. proportions completing only one grade, two grades, etc. 
2. MAINTAINING AND IMPROVING INSTRUCTIONAL QUALITY
ET has made important progress in developing systems to maintain and improve instructional 
quality by its well-designed “train the trainer” efforts targeted to departmental coordinators. This 
effort now needs to be extended to developing the skills of facilitators. Even though facilitators 
are not always trained teachers (although, in fact, some are), they can, as non-specialists, play a 
greater and more important role in facilitating student achievement than they do currently.
In addition to current efforts to improve facilitators' mastery of basic education content (e.g. 
mathematics) it would be desirable to provide training regarding at least the following: a) 
techniques to stimulate interaction among participants, b) strategies, exercises and activities to 
extend learning beyond the “classroom”, c) techniques for informally assessing students’ 
competencies and how to extend these techniques to tracking student progress and providing 
tutorial assistance as needed, and d) ways to use cooperative learning techniques including peer 
tutoring to improve student performance. A very useful resource for initiating efforts to assure 
uniform instructional quality and to generally enhance facilitators’ ability to move students ahead 
as rapidly as possible would be instructors’ guides to accompany the workbooks and 
textbooks for each grade level.
3. UPGRADING AND UPDATING CURRICULUM MATERIALS
ET must re-examine the conceptual basis of its curriculum and the radio and print materials based 
on that curriculum. It should then begin to revise materials in an orderly, systematic fashion in 
response to the needs of educational stakeholders—learners themselves, Honduran employers, 
and civil society. We recommend beginning with the materials for grade levels 4-6—as it is at this 
point in the basic skills curriculum where the current lack of stimulating, adult material is most 
noticeable and where the economic payback for a first-rate curriculum would be greatest. 
Ultimately, ET should adopt a process for constantly assessing curriculum and materials in light of 
the mix of students who enroll, and skills demands they face, and findings from formative 
evaluation research; this process should then provide the basis for ongoing revisions and 
refinements to the curriculum and instructional materials based on it.
An important first step in upgrading and updating ET materials is for the project to review the 
substantial literature on basic skills development in relation to workplace skills demands. There is 
a good deal of evidence that working to develop such skills is also a critical step in preparing 
youth and adults to participate meaningfully in the civic life of their communities, manage the 
challenges of family life in a social environment which is increasingly stressful (particularly when 
rural families migrate to urban areas). Perhaps most critical in this inventory of “foundation” 
skills is preparing students to become self-directed learners—using multiple sources of 
information and information resources to advance their own skills development 34
. 
ET materials will need to better support active learning and authentic task completion. The 
content and guidance in materials must help facilitators and the learners they work with be not 
simply consumers of information and followers of orders but also proactive information managers 
21
in securing the information they need, evaluating its relevance and accuracy, organizing and 
storing information, and using that information to communicate to others. Active learning must 
also be understood to include self-expression and assertion of one’s own identity—in different 
genres of communication, including oral exposition, writing letters and notes to friends and family, 
in poetry, and in structured communication with co-workers and supervisors.
ET students deserve a first-class basic skills curriculum. Investments in developing such a 
curriculum, an appropriate instructional design, and materials better designed to prepare 
youth and adults in the program for not simply surviving but prevailing in a challenging 
economic and social environment are particularly worthwhile because there is a solid 
infrastructure for service delivery—radio, cassettes, and the project’s extensive and sound 
network of coordinators, promoters, and facilitators.
4. INTEGRATION OF GRADES 1-6 AND GRADES 7-9 CURRICULUM
The current ET Grades 1-6 curriculum is likely to leave students who have completed the 6th
grade level unprepared for middle-school level work in grades 7-9. In the short-run, it would be 
wise to consider development of a transitional/remedial module to prepare ET 6th grade graduates 
for subsequent education or training—in ET grades 7-9, in vocational training, or in regular school 
(ciclo común). In subsequent revision of the ET Grades 1-6 curriculum and post-pilot 
refinements to IEQII’s Grades 7-9 materials it would be useful to seek, if not to integrate, at least 
to harmonize, curriculum and materials. 
5. RESOURCES TO ENHANCE STUDENT LEARNING
Investments in basic equipment to allow facilitators to support students’ learning would be 
judicious. The cost of such support is not high and the probable impact on instructional quality is 
high. These materials include: blackboards, writing tables, and supplemental reading materials.
The availability of reading materials is particularly important because if books, magazines, 
newspapers, instruction manuals, fiction, and poetry books are accessible and students 
are encouraged to try them out, it is possible to engage students in spending more time 
on task building their reading skills . At the same time, the process of reading on their own will 
provide students an important experience base in self-directed learning, finding out what they like 
and what they don’t, getting help with difficult passages, working collaboratively to understand 
what an article or book is about. The availability of different genres of reading material is 
essential because these texts will provide students the models for understanding “voice” in verbal 
and written communication and building their own skill in writing.
A variety of materials should be available at each grade level and promoters should be provided 
satchels of such reading materials to make grade-appropriate materials available to facilitators for 
each of their learning groups. Reading assignments can easily be added into the instructional 
design even before curriculum and text materials are updated. An additional effort might consist 
of rewarding students who complete a grade level with a choice of two or three grade-appropriate 
books; this would make a powerful symbolic statement about the value of literacy and get books 
into homes where there is likely to be no reading materials—for adults or children.35
6. THE MEASUREMENT OF EDUCATIONAL OPPORTUNITIES AND 
OUTCOMES, AND VALID ACHIEVEMENT BENCHMARKING
ET testing practices must insure that the inferences that are made based on student testing are 
defensible in relationship to accepted theory and empirical evidence. This is essential in order to 
22
use assessments to identify faults and strengths in instructional programs related to the educational 
experiences of learners in order to identify potential corrective measures. A necessary condition for 
this is consideration of the instructional and the curricular validity of the tests. ET (and all testing 
under BEST) must face the issue of whether tests measure the objectives of the curriculum, and 
whether instructional programs provide learners with instruction in the skills and knowledge being 
assessed.36 Many issues of curriculum validity 37 that are problematic in tests designed and 
administered under ET and all BEST projects, would be solved if customary procedures were 
followed – as the alignment of measures to the curriculum is part of the procedure of test 
development38
.
Evaluation in ET must also recognize the imperative of characterizing instruction in order to 
explain achievement. At present, ET evaluation does not measure the array of instructional 
experiences that explain the knowledge and skills of students whose abilities they measure. A new 
evaluation design for ET should include measures of Opportunity-to-Learn (OTL)39 and 
educational opportunities to provide the vital link between goals and outcomes that would turn 
mere testing into educational evaluation. Including measures of educational opportunities in 
student assessments provides indicators on the educational process as it unfolds in the various 
program sites, thus providing program managers with vital tools for the evaluation and 
formulation of effective educational programs and in the case of system evaluation – effective 
educational reform policies. Incorporation of techniques for measuring educational opportunities, 
including OTL is vital to turning educational assessment in BEST programs, into authentic 
educational evaluation. 40
.
Improving ET evaluation systems requires addressing two features: 1) testing for program 
advancement of ET participants (pass-fail decisions); and 2) testing to compare the achievement 
of ET students against the achievement of their peers in other programs.
Pass-fail tests that are both simple to administer and score and yet still meet minimum standards 
can be developed. They would be longer, as they would need to contain at least three items for 
each domain, but they can be composed so that students have adequate opportunity to 
demonstrate what they know and so that promoters can correct them. The ET evaluation team 
must follow standard procedures for criterion referenced testing – including domain definition and 
validation by judges; formulation of item development specifications with the corresponding 
validation by judges; and item development and selection according to standard criterion￾referenced measurement techniques. If cost and personnel issues makes it impossible for any one 
other than the promoter (never the facilitator) of each group to correct the tests, each promoter 
can be given an answer key, after they have collected the assessments from each site.
Benchmarking the achievement of ET students against the achievement of students in other 
programs is a more challenging task. The best way to accomplish this simply and validly is to 
administer identical tests under identical conditions to random samples of students from each 
program that is to be compared. Tests for this purpose must be validated across all of the 
programs, not only among ET participants. Every student must receive the test at the same time, 
and the test must be administered and scored under the same conditions. This means for example, 
that ET students must not have their tests scored by ET staff, whereas the students in other 
programs have their tests scored by parties unassociated with their programs. The best procedure 
to guarantee professional evaluation standards, is to commission an independent body -- such as 
the UMCE -- to conduct the entire benchmarking study from test development to writing of the 
final report. The additional financial outlay will be more than compensated for by freeing ET staff 
from this task and by the production of data and reports that could be used in complete 
23
confidence in the formative evaluation of ET. Such a strategy would contribute superior 
resources for the development of timely and sound program enhancements. 
7. FUTURE MANAGEMENT AND IMPLEMENTATION MECHANISMS
The current management and implementation structure appears to be working well for ET. 
Different options deserve to be considered in connection with efforts to assure the project’s 
sustainability. We address these in the strategic planning document.
8. SYSTEMS TO ASSURE ACCOUNTABILITY AND MAXIMIZE IMPACT
In its general outlines, ET is well-prepared to develop accountability systems. However, the 
challenges it faces in this endeavor are technically formidable. Essentially, the need is for ET to 
develop the following sub-systems (which are detailed separately): a) a valid and reliable system 
for assessing and certifying student achievement, b) an MIS which supports individual student 
records, c) management procedures to conduct low-cost applied research on student achievement 
as the basis for initiating corrective action, and d) reporting systems to capture geographic and 
individual variations in performance.
Reliance on empirical data on student performance and staff productivity are important tools for 
initiating corrective action in a variety of forms including: technical assistance and training, 
supervisors’ guidance to staff and/or volunteers, adaptation of instructional techniques, revision 
of curriculum, problem-solving related to logistics, and so on. ET has a good system for initiating 
corrective action; the project’s weakness is in the “diagnostic” subsystems for evaluating student 
performance and deciding what action to take.
E. LINKAGES, PARTNERSHIPS, AND SUSTAINABILITY
ET has a good initial track record of developing partnerships. We believe this experience provides 
a foundation for a concentrated effort to build more and better partnerships as a key strand in 
efforts to assure the sustainability of ET alternative education delivery system. The project 
developed, during the first quarter of 2000, a draft document providing guidance on setting up 
new learning centers—a useful tool for moving forward in responding to new demands for service.
A key issue to address is that the most successful partnerships are those that involve not only 
financial collaboration but, also, joint consultation on collaborative efforts. An important avenue 
to explore will be to consult carefully with partnering organizations--including employers and 
municipal governments--about respective objectives, priorities and so on.. We strongly urge ET 
and USAID to work toward a curriculum development process which is responsive to these 
stakeholders. Such responsiveness will be extremely beneficial both in terms of increasing 
educational quality and relevance but also as a means of securing higher levels of financial and 
political support. 
ET has worked well in consultation with its private sector partners but should now take the next 
step and develop workplace literacy programs which can be “customized” to provide both basic 
skills foundations and, at the same time, directly address the pressing needs of employers (e.g. for 
employees to read safety instructions on dangerous machines accurately). In this context, it is 
important not only to strengthen the Grades 4 -6 curriculum but, also, to complete the Grades 7-9 
curriculum. In proceeding to complete the Grades 7-9 curriculum it will be necessary to provide 
ongoing technical assistance and oversight to assure that instructional design and curriculum 
content conform to stakeholder expectations.
24
There are also opportunities for productive collaboration with other alternative education 
programs. ET’ sharing of textbooks with adult education night classes and with PRALEBAH is 
an example of cooperative efforts. Future efforts might include collaboration with primary 
schools and other programs in getting books and other reading material into rural communities, 
sharing scarce transportation, exploring more ways to work with PROHECO’s community 
schools, and general collaboration to promote learning and literacy.
While ET has used its air time with radio broadcasters very skillfully to promote student 
achievement with spots exhorting students to enroll in classes, to keep on working and not drop 
out, to encourage others to enroll, etc. It has not yet used its access to the airwaves as a means 
for enhancing its own sustainability. Such an effort would be quite worthwhile and is in no sense 
inappropriate. NGO’s throughout the world use broadcast media to solicit support for their 
activities; ET can do the same.41
Sustainability of distance learning delivery systems also requires careful attention to technology 
development. Currently, ET’s identity is defined by the technology it uses to deliver the audio 
component of instruction—broadcast radio. Just as curriculum and instructional design require an 
ongoing process of monitoring and update, technology use requires ongoing consideration of 
newly-emerging technology. Because of the design versatility and addressability of Internet-based 
multi-media educational systems, such program designs form the “gold standard” of distance 
learning. However, even within the context of advanced countries, multimedia materials 
development is costly, suggesting that radio may continue to be the best distribution technology 
for Honduras for years to come—but lower costs for transponder space, for microwave networks, 
mean a variety of emerging technologies deserve to be tracked. 
IV VOCATIONAL TRAINING WITH BASIC EDUCATION
A. OVERVIEW
CADERH consists of a central office, referred to in this document as CADERH, and a network 
of 25 independently operated and financed training centers having an enrolment of 3,000 low￾income youth and young adults, referred to here as the CADERH network. There are five centers 
located in rural areas with an enrollment of 540 students. There are nearly 600 female students in 
the system, most of them in traditional occupations such as garment making and secretarial 
studies. However, one girl this year was certified in electrical installation. CADERH provides 
critical services to the centers in the network which enable them to provide far better quality 
training programs than would be possible without CADERH's assistance. 
From its beginning, the implementation of the vocational training component of the BEST Project 
experienced delays, fostered poor end products, and created conditions not conducive to realizing 
the project purpose and anticipated results. This was due primarily to CADERH's poor 
management. How and why this occurred is discussed later in this section. Unable to endure the 
continued inadequate administration of the project, the Mission advised CADERH in October 
1998 that it must fulfill the following obligations if the project was to continue: (1) hire a new 
Executive Director; (2) improve its accounting system; (3) correct outstanding recommendations 
from recent audits; and (4) complete the construction of four training centers initiated in 1996. In 
early 1999, before CADERH met these requirements, the Mission provided CADERH with 
significant assistance to complete its 1999 Workplan and Budget. By May 1999, the Mission 
requirements had, for the most part, been met. Project management returned to Tegucigalpa from 
San Pedro Sula and the 1999 Workplan was approved. The Workplan provided for constructing 
25
and equipping four more training centers, for providing equipment and training materials to three 
existing centers, and for training 40 vocational education instructors and school directors. 
CADERH is currently completing these tasks prior to the new project completion date of August 
31, 2000.
The difficulties that characterized project implementation have left their mark on the technical 
capabilities of CADERH and on how the Mission and CADERH currently view each other. With 
frequent disruptions in the flow of project resources due to management and accounting 
difficulties as well as the return of the project to Tegucigalpa, CADERH's technical expertise was 
reduced considerably. There is a new group of young technical staff providing field services, but 
they are still at the learning stage. The Mission has major concerns about CADERH ever 
becoming a dynamic leader of private sector training at the same time that it provides technical 
services to centers that train low-income, out-of-school youth. Moreover, the Mission seriously 
doubts that CADERH will ever sustain itself financially. CADERH views USAID/Honduras as 
continually adding to the number of training centers without considering CADERH's financial and 
technical capacity to provide quality services to the CADERH network. CADERH feels 
marginalized from decisions related to the magnitude of its own responsibilities and the 
anticipated resources that it will have to fulfill these responsibilities.
Table 1 below provides a summary of accomplishments in relation to planned objectives.
Table 1
CADERH Accomplishments in Relation to Planned Objectives
Results Anticipated Achieved % Complete
No. of new centers 15 8 53
No. of upgraded centers 12 8 67
No. of students 12,600 3,774 30
No. of female students 3,780 924 24
No. of students certified 12,600 1,500 12
No. of centers offering ICB 27 25 93
No. of students employed 10,800 1,125 10
No. of new training modules 5 9 180
No. of modules distributed 50,000 90,000 180
% of income from production 50% 22%
Retention rates in existing centers 90% 93%
Retention rates in new centers 85% 68%
Maximum cost per student hour $.50 $.37
Centers using ET 27 0
Centers with business training 27 0
Centers with production funds 27 16 59
Difficulties in attaining the anticipated results are discussed in the following sections.
26
B. FINDINGS REGARDING KEY PROJECT COMPONENTS
1. CONSTRUCTION AND EQUIPPING OF CENTERS
The construction process was a study in poor management. USAID/Honduras provided the floor 
plans for the buildings and required that adobe be used as the primary building material. As 
construction began, management problems became evident. Since CADERH could not pay 
workers on a timely basis due to faulty accounting and slow payment procedures, they would 
leave the work sites. The adobe production was not synchronized with construction demands --
adobe blocks that were not sufficiently dried out were in fact used so that construction workers 
were kept busy. Plastic sheets needed to protect the newly applied adobe were not purchased 
when needed. There was only one CADERH engineer providing oversight, which was not 
sufficient and that person had little experience in adobe construction techniques. As a result, 
some of the buildings constructed during this period have serious construction flaws.
The second set of four centers is a product of the 1999 Workplan. Because the Workplan was 
only approved in May 1999, construction could not start until June when the best season for 
initiating construction was over. Since there was a requirement that 70% of the adobe materials 
be made prior to actual construction, many of the earlier construction problems were avoided. 
CADERH had two engineers supervising construction (although there are questions about their 
competence) and internal management had improved considerably, but getting funds to the 
worksites was still very slow. Construction of these centers occurred during the post-Mitch 
building boom when experienced workers and construction materials were in short supply and 
more costly than provided for in the budget. Again, a significant construction related problem was 
noted in the one training center visited from this group of new centers.
Equipment purchases for the first centers ran into problems as well. Orders were placed on the 
basis of the needs for some centers which were not built due to last minute changes in the 
selection of what communities would be served. Equipment arrived late due to poor management. 
Finally, some equipment arrived at centers incomplete because of inadequate inspection upon 
arrival in Honduras. One major mistake was providing the same dollar amount for equipment 
purchases for each center irrespective of the trades being taught, since some trades require more 
expensive equipment than others. Solicitations for equipment purchases under the 1999 Workplan 
were handled by a committee, and all worked well during the solicitation and bid selection 
process. Difficulties began when the equipment arrived, since no one on the committee was 
available to review what was shipped. As a result, a number of wrong items were accepted, some 
items ordered did not arrive, and some equipment was missing parts upon arrival. It is clear that
CADERH did not have the administrative capacity to manage the construction and 
equipping of the training centers without outside assistance.
2. PROVISION OF RELEVANT TRAINING AND THE DEVELOPMENT AND USE 
OF THE BASIC TRAINING MODULES
Relevant training depends primarily on the actions of the CADERH network, which is made up of 
independent entities. The centers, unfortunately, do not have an effective mechanism such 
as participation on the CADERH Board or through an Advisory Committee to have a say 
regarding CADERH policy or operations. CADERH does, however, provide significant 
value added to the quality of their training, basically through the provision of competency based 
training modules. The centers visited all believe that the modules enhance their instructional 
capabilities and are relevant to the trades offered, the students served, and the level of job 
performance in their communities. There were some comments that the latest revisions made 
27
under BEST were not as neatly and professionally done as were earlier upgrades. In addition, one 
school director pointed out that occasional discrepancies appeared within or between modules in a 
given trade which caused students to consult instructors for clarification.
The number of students enrolled in the CADERH network during the project was far lower than 
anticipated. Fewer centers were constructed than programmed and they were completed much 
later than anticipated, which affected enrollment. While some centers were reintegrated into the 
program, they only came back recently. The "Centro Modelo" was projected to have a substantial 
enrollment, but instead fell into near disuse throughout much of the project. Finally, after 
Hurricane Mitch, many students were kept home to assist families recover from their losses. With 
enrollment down, all other student related measures suffered accordingly. The training modules 
provided by CADERH contribute significantly to the quality of instruction throughout the 
CADERH network; failing to attain enrollment ta rgets was related to poor management 
and Hurricane Mitch.
3. PROVISION OF TECHNICAL ASSISTANCE AND STAFF TRAINING TO THE 
CENTERS
A second source of value added by CADERH is through technical assistance and staff training. 
Because of a relatively high turnover of instructors due to opportunities for better paid jobs, 
CADERH fills a critical role by providing training in the use of competency based (ICB) methods 
to new instructors. This is still a relatively new concept in Honduras and, since many instructors 
come from private industry, they are not experienced teachers nor are they familiar with 
competency based instruction. Instructor training should be provided as well for improving 
technical skills but funding has not been available to do so. Staff tra ining is also provided to 
school directors. Some directors noted that staff training and technical services have not been 
consistently made available. Because of problems CADERH has in securing funds, the services 
are provided when funds are there to do so. CADERH is currently revising the way technical 
assistance is made available to the CADERH network. This is described in Appendix E. 
Observations of one CADERH Technical Educational Advisor working with a variety of training 
centers indicated that significant technical assistance is being provided by CADERH. For 
example, a problem of how to respond to a large request for training by a garment assembly plant 
was successfully addressed by the CADERH Advisor, working with the school director and his 
staff. Staff training and technical assistance are important services provided by CADERH 
to the training centers. 
4. ASSISTANCE ON TRADE CERTIFICATION OF THE SKILL LEVELS OF 
STUDENTS
A third source of value added is through the certification of the proficiency levels of training 
center graduates. Certification consists of scoring at least 90% on a written test about theoretical 
knowledge related to the trade and scoring 100% on a demonstration of the ability to perform the 
tasks required by the trade. Two problems were noted: (1) certification is very costly, and (2) 
only one set of written exams is used for each trade so it is possible to remember or report on the 
questions for use in subsequent applications. Centers rely on CADERH to administer the 
certification tests.
5. INCREASING BASIC EDUCATION LEVELS THROUGH ACCESS TO ET AND 
PROVIDING TRAINING IN SMALL BUSINESS DEVELOPMENT
The BEST Project called on CADERH to introduce ET programs throughout the CADERH 
network. This did not happen although it was reported but not confirmed that one training center 
28
was using ET. Since all students entering CADERH programs must have completed the sixth 
grade, CADERH is concerned that ET does not yet provide education beyond that level. Several 
of the centers visited expressed interest in having access to ET when educational programs are 
available at the 7 - 9th grade levels and three CADERH centers have been designated as sites for 
field testing the program. While completing the sixth grade is a prerequisite for entry into 
CADERH programs, it is obvious that not all sixth grade graduates have the basic education 
requirements to implement the ICB based modules. Several schools visited have remedial 
programs in Spanish and math to cover these deficits.
Courses for small business development and management were not developed under the BEST 
Project, although this was a requirement. Several of the training centers visited were assisting 
graduates to begin their own businesses, although a number suggested that small businesses have 
more success when individuals are more mature. Training centers want courses for small business 
development and plan to offer them along with production programs which teach students how to 
deliver goods and services that meet market demand. CADERH did not implement the ET or 
the small business development programs called for in the BEST Project although 
training centers expressed interest in having access to both programs.
6. DATA ON STUDENT SUCCESS IN TRAINING AND IN THE LABOR MARKET 
AND MANAGEMENT SYSTEMS TO MEASURE EFFICIENCY/ENSURE 
ACCOUNTABILITY
CADERH's data on student success in training is very weak. Efforts are being made to bring 
uniformity to the data by reviewing the data collection forms with school directors. Quarterly 
reports are made on the number of students at the beginning and the end of the quarter by sex and 
by area of study. The rate of retention is calculated by comparing the two numbers. Since 
training is on an open-entry/open-exit basis, it is possible to have more students at the end of the 
quarter than at the beginning which calls into question the reported retention rates. Even more 
alarming is that retention measures apply only to the quarter that is reported - not to the duration 
of the training program. A program may be one or two years in duration. A true measure of 
success is the number of students who are certified in their trade compared to the number who 
entered the program. Since certifications are done by CADERH, they are precisely measured and, 
if correctly compared to the proper cohort of school entrants, can provide an excellent measure of 
success. CADERH says that this is the measure they are using. The number of instructional 
hours is also reported. Data on training costs and sources of funding are reported in detail. While 
provided quarterly, they are separated by month and are broken down into many components, 
including the amount of income resulting from production programs and a detailed breakdown of 
costs. Estimates of cost per-hour-of-training should be quite accurate.
Placement data are not well collected. CADERH's estimates are based on infrequent field studies 
which determine a representative percentage of placements of graduates. The data are valid when 
first collected. These ratios, however, are then applied to the number of graduates in subsequent 
years to estimate the number of students placed. Were this information precise, it is questionable 
as to how useful it would be. Students typically leave a CADERH program at 16 to 17 years of 
age -- some even younger. The labor law does not permit young people to enter the labor force 
until they are 18, although it is reported that 47% of 15 - 17 year olds are economically active. 
The initial jobs of CADERH graduates are those of general helpers or low paid, low level workers 
in their trade. The real success of CADERH graduates will be how their discipline and training 
allow them to advance in the first several years after their entry into the labor force. At the end of 
five years, one would hope to find them beginning to attain the maturity and workforce experience 
that allow their training to pay off. The only way to measure this is through sample longitudinal 
29
studies. While CADERH is trying to improve the uniformity of data collection, the data 
gathered do not permit accurate measures of the efficiency of current programs nor the 
success of graduates after entering the labor force.
CADERH has a major weakness in its ability to track its costs and income by program area. In 
addition to its training related services managed by the Education Development Division, 
CADERH has a number of programs which should generate income or at least be self-financing. 
These include a video production service, an English language program, a placement service, a 
division providing services to firms related to human resources and training, and the “Centro 
Modelo” in San Pedro Sula. In response to a recent request by the Board of Directors, CADERH 
was unable to provide timely data on the financial status of each of these entities. It is likely that 
one or more of these units is losing money but the Board has been unable to determine which 
these might be. CADERH must improve its financial management capability to adequately 
track the contributions or losses of each of its separate activities.
C. POLICY ISSUES TO BE ADDRESSED
1. ADEQUACY OF PROJECT DESIGN TO ADDRESS THE SKILL TRAINING 
NEEDS OF LOW INCOME, OUT-OF-SCHOOL YOUTH AND THE NEEDS OF THE 
LABOR MARKET
The project built on the success of an earlier Mission financed activity with CADERH and was 
appropriately designed to provide needed training services to the target population. While the 
activities included in the CA are in line with the anticipated results, one important factor 
contributing to project success was misjudged by USAID/Honduras in designing the project --
CADERH's lack of administrative capacity to carry out project implementation. CADERH 
benefited from significant amounts of USAID/Honduras' assistance through program grants prior
to the BEST Project. The first program, lasting from 1983 until 1986, got CADERH started as a 
private organization to stimulate competency based training in the private sector and to certify 
workers. This was done with considerable Mission involvement -- almost to the point of 
CADERH being the product of USAID's interest in developing an appropriate organization to 
provide skills training to the private sector. In 1986, CADERH received a new grant enabling it 
to build 15 training centers serving low-income people to be operated by NGOs and 
municipalities, to provide private sector in-plant training, and to greatly expand the range of 
training materials and certification services. In both cases, close Mission supervision and plentiful 
technical services were provided. An evaluation in 1992 upon project completion indicated that a 
sound instructional system was established, including 15 privately maintained training centers. It 
also pointed out that CADERH was likely to have difficulty attaining financial sustainability.
After the second project ended, CADERH turned its attention to income generation to keep itself 
alive without Mission support. As a result, between 1992 and late 1995 when BEST Project 
resources began to provide support to CADERH, there was a significant reduction in CADERH's 
ability to keep modules up-to-date, to provide certification services, to organize staff training for 
the affiliated centers, and to maintain the technical advisory groups which oversaw the relevancy 
of the training modules. The number of training centers in the CADERH network dropped to 
eight because of the few services provided them. CADERN did, however, become nearly self￾sustaining. In 1995, the Mission received a proposal from CADERH, for the program included in 
Component II of the BEST Project. Because of the successful final evaluation of the earlier 
project and because of CADERH's near financial self-sufficiency, a Cooperative Agreement (CA) 
was signed. The Mission determined that very little oversight was required for implementation of 
30
the CA and only relatively limited technical assistance was needed. This proved to be a serious 
misjudgment. CADERH direction changed at the beginning of the project. Project management 
was shifted to San Pedro Sula remov ing it from easy oversight by the Mission and the CADERH 
Board and Executive Director. CADERH, in an effort to expedite implementation, hired an ex￾USAID employee to manage the project who had limited experience with training and was 
eventually fired because of questionable management of project resources. A mid-term evaluation 
that should have detected these problems was never done. The bad management that 
characterized the project from its initiation until its compete overhaul and return to 
Tegucigalpa in early 1999 is partially explained by this initial misjudgment about the level 
of Mission supervision and technical assistance required by the project. 
In spite of cautions in the final evaluation of the Mission's earlier project with CADERH about 
the difficulties related to sustainability, the BEST Project made little provision for this important 
item. It is true that CADERH had attained a precarious level of self-financing at the time the 
project was developed, but it had been at the expense of offe ring adequate support services to the 
training centers. The project design should have provided assistance to CADERH for 
attaining financial self-sufficiency.
2. CADERH INCOME SOURCES AND THE PROVISION OF QUALITY PRIVATE 
SECTOR ORIENTED TRAINING SERVICES
Because CADERH's ability to attain long-term financial sustainability was identified as a problem 
before the BEST Project was designed, the project provided for an endowment fund which will 
generate from $55,000 for the year 2000 up to $97,000 in the year 2017. This will provide modest 
support for providing assistance to the training centers. A recently signed agreement with INFOP 
(Instituto Nacional de Formacion Profesional) will also provide CADERH with another $55,000 
of support for the centers. These two sources of income, however, fall short of what CADERH 
needs to provide quality training services over the long-run. A USAID funded program with 
DevTech will cover an upgrading and distribution of training modules and staff training services 
for training centers until December 31, 2001. A detailed discussion of possible additional sources 
of income for CADERH to consider is included in Appendix E. Additional assistance provided 
to CADERH must consider how the organization can attain financial sustainability in the 
short-run.
3. IDENTIFICATION OF CRITICAL TRAINING 
The CADERH network provides skills training in entry level occupations related to the needs of 
their local communities. The output of each center is small and much of the training relates to 
forming discipline and good work habits which are universal labor force requirements. Therefore, 
the identification of training needs is not a critical factor in guiding their operations. Were the 
Mission to finance more training centers in addition to those to be constructed by DevTech, it 
should obtain an overview of the vocational training efforts of all Honduran institutions and 
compare these to projected labor market requirements so that training resources contribute both 
to national development requirements as well as the needs of low income, out-of-school youth. 
INFOP is completing such a survey and the results should be available by January 2001.
D. LINKAGES, PARTNERSHIPS AND SUSTAINABILITY
1. PLANS OF THE Secretaría de Educación AND INFOP TO EXPAND THEIR ACTIVITIES 
IN THE AREA OF VOCATIONAL EDUCATION
31
The current vocational training programs of the SE and INFOP are discussed in Appendix F. 
There are no plans to expand the role of the Ministry in the area of vocational education. The 
major concern is to improve the installations that exist. Ministry representatives believe that it 
would be advantageous to coordinate the activities of the various entities in Honduras which 
provide training. This coordination, in their judgement, should be done informally without any 
one institution being predominate and without legislation mandating that institutions operate 
under the guidance of the Ministry. It was suggested that the Secretariat for Technical 
Cooperation (SETCO) or a representative group of international donors like MERECE could 
initiate discussions between SE, INFOP, CADERH and the universities about how to support 
each others' activities in meeting Honduran labor market requirements.
INFOP is outside the formal education sector although its funds come from a 1% tax on private 
sector payrolls. INFOP strongly supports an initiative that emerged as part of the Education 
Reform process although it was not included in the final document. SINAFOP (Sistema Nacional 
de Formacion Profesional) would be created, which includes INFOP, SE, UNAH and UPN, to 
coordinate and set the standards for all programs which prepare people to enter the labor market. 
The tone of the document describing its functions was very bureaucratic and inflexible which is 
the opposite of what people working in the SE programs suggested. No mention is made of the 
role of private training centers within SINAFOP. Coordination of private and public sector 
activities in the area of vocational training is needed but the very formal coordination 
mechanism represented by SINAFOP should be very carefully studied and vetted prior to 
enactment.
2. PLANS OF OTHER INTERNATIONAL DONORS TO ASSIST IN THE 
DEVELOPMENT OF VOCATIONAL TRAINING PROGRAMS 
There are three international programs providing assistance to SE vocational training institutions. 
The PCOVH (Producion en Colegios Vocacionales en Honduras) Project financed by the Central 
American Development Bank is in its final phase. Assistance was provided to 20 SE Technical 
Institutes in the form of equipment, staff training and training materials. The Government of 
Mexico has provided assistance for the development of training in two trade areas: construction 
and garment making. The European Union is providing assistance to 15 other Technical 
Institutes through the AFOPH Project (Apoyo a la Formacion Professional en Honduras). Their 
four year program has one more year before completion. The program is a pilot project to 
demonstrate how all Technical Institutes might function better. They have trained 360 teachers in 
almost 400 courses with over 34,000 hours of class instruction. Their goal is to enable teachers 
and school directors to offer better programs through improvements that they themselves develop 
and implement. In six of the 15 pilot institutes, they will introduce the concept of production 
centers. In these cases, students simultaneously learn and engage in income generating activities. 
INFOP will receive $5,000,000 from Spain to buy new equipment for all its centers, IDB 
financing for workshops in low income urban neighborhoods, additional equipment from Japan, 
and other assistance to develop small business development programs. Significant assistance is 
being provided vocational training programs; the immediate need is for informal 
coordination of all Honduran labor force related training.
3.SUSTAINABILITY OF VOCATIONAL PROGRAMS, THE ENDOWMENT FUND,
AND USAID'S RELATIONSHIP WITH CADERH 
Responsibility for providing training services under BEST rests, for the most part, with the 
training centers. They will continue to provide training services with or without continued 
USAID support. They have community resources in place to utilize, maintain and generally 
32
benefit from the assistance they receive from the Mission. CADERH, however, significantly adds 
to the quality and efficiency of the outputs of the centers. However, CADERH has been heavily 
dependent on USAID support. The major issue, therefore, is whether CADERH can become 
financially viable without USAID assistance. The BEST created endowment fund, as discussed 
earlier, will only partially cover CADERH's costs for providing needed services to the centers. It 
would be a great loss if CADERH were no longer able to provide this support to the CADERH 
network.
We recommend, therefore, that an Exit Strategy, lasting until December 31, 2001, be 
developed to provide assistance for CADERH to become financially self-sustaining while 
the DevTech Program with CADERH is being implemented. This Strategy would provide 
some complementary operating funds to the assistance provided by DevTech to bridge the period 
until CADERH can raise sufficient funding. The major thrust of the assistance package focuses 
on: (1) reorganization of the Board of Directors to enable it to become a real leader of private 
sector efforts in the area of skills training and a source of funding; (2) rationalization of CADERH 
to eliminate unnecessary functions and organizational components which lose money; (3) 
exploration of potential sources of funding that are common for similar private sector 
organizations in other countries, (4) determination if the “Centro Modelo” can become a source of 
income (with USAID concurrence); and (5) exploration of the INFOP tax base as a source of 
revenue for CADERH. The Exit Strategy should begin as soon as possible and should last no 
longer than the DevTech Program. See Appendix E for a detailed presentation of the Exit 
Strategy.
If during this period, it becomes apparent that CADERH cannot bring about the basic conditions 
to fulfill its commitments under the proposed Strategy, the resources in the program should be 
immediately directed to explore two other possible sources of services for the training centers. 
The first is an association of the training centers to address their own needs, although financial 
self-sufficiency for such an organization will be a problem. The other possibility is an affiliation 
between the training centers and INFOP which has similar relationships with other private sector 
training facilities. It is through this program of "Centros Colaboradores", that INFOP provides 
the funding for the current CADERH-INFOP Agreement, and where such a relationship could be 
established. INFOP is not an organization warmly endorsed by the private sector, although its 
image has improved somewhat under its current leadership. It is heavily oriented to traditional 
vocational training methods and top-down management practices although competency based 
instruction is being introduced for short courses.
V. POLICY REFORM AND PROJECT PROMOTION (PILOT PROJECTS)
A. OVERVIEW
The policy reform and project promotion component of BEST (Component III) was designed to 
promote debate on educational change by supporting educational research and other policy-related 
activities and disseminating the results.. The funds ($300,000) were included in the cooperative 
agreement with CADERH, but were subsequently brought back into the Mission when concerns 
arose regarding CADERH’s management.. Until after Mitch, the main use made of Component 
III was for technical assistance run out of the Mission.
33
1. CURRENT STATE OF IMPLEMENTATION
After the hurricane, Component III funds were re-programmed to support Mission efforts to 
support a major new educational reform effort developed by the GOH education authorities, with 
assistance from USAID and other donors. 
At the same time, USAID initiated a number of pilot projects, designed to test several basic 
education models, which had the potential to make valuable inputs to the reform program, as well 
as to make small, but direct contributions to the post-Mitch emergency recovery effort in 
education. These included EE, TB, and AM. While EE was subsequently terminated, out of 
concern that the SE was not sufficiently interested in it to warrant its continuation, its basic 
elements were continued in a new SE project, Salvemos Primer Grado, without funding from 
USAID. The others are still in operation.
2. ACCOMPLISHMENTS: EDUCATIONAL REFORM
Although Component III activities did not begin until late 1998, after Mitch, a great deal has been 
achieved in the approximately year and a half since the hurricane. The accomplishments fall 
under two headings: Educational Reform and Pilot Projects. Accomplishments of the pilot 
projects are included in the next section, Findings Regarding Key Project Components.
In the immediate aftermath of Mitch, USAID made a critical contribution, together with the 
Minister of Education, senior members of the Ministry, key Honduran educational leaders, and the 
other donors, in setting in motion a comprehensive educational reform effort for Honduras. The 
Mission helped bring the key players together, out of which a consensus developed to use the 
Mitch disaster, and the outpouring of sympathy and assistance which followed, as a springboard 
for change.
Subsequently, a series of joint activities were planned and executed by GOH education leaders 
and a group of lead donors agencies, including USAID. The goals for this effort included: 1) The 
building of a national coalition in support of the reform, through a series of consultative and 
participatory measures, including development and dissemination of a major national reform 
proposal; 2) the progressive accumulation of ideas, suggestions, and other substantive inputs into 
the content of the proposal and the subsequent reform; 3) the creation of appropriate coordination 
mechanisms among the donors and between the donors and the Honduran education authorities; 
and 4) laying the groundwork, once the proposal had been prepared, for institutionalizing and 
implementing the reform.
The first activity was a series of five seminars held around Honduras. A main purpose of the 
seminars was to open up the reform debate to a wide spectrum of institutions and individuals from 
the public and private sectors, political parties, educational institutions, non-governmental 
organizations, churches, communities, parent groups, labor unions, media, and educational experts 
together to debate reform issues and to seek to obtain their support. The seminars were also 
designed to be substantive events, thus assisting in developing a deeper understanding of the 
principal deficiencies of Honduran education and the key problems and needs that would need to 
be addressed.
The seminars generally met their objectives. The various events were well attended, and judging 
from the participant lists, the target groups were largely reached. Discussions were reportedly, on 
the whole, lively, and much useful information was obtained. Total attendance at the five events 
34
was on the order of 1200. Informal contacts by the Team with seminar participants suggest that 
the reactions were generally positive.
A second activity was the undertaking by FONAC of an extensive series of community and local￾level discussions throughout Honduras, designed to give a wide range of ordinary Hondurans of all 
kinds the opportunity to offer their opinions on local educational needs and, hopefully, gain their 
support for the reform. 
Third, a process was begun, led by FONAC, to prepare a comprehensive educational reform 
proposal, drawing on the recommendations emanating from the seminars, the national 
consultation, and individual proposals submitted by a variety of Honduran public and private 
institutions involved in education. The FONAC proposal, which consisted mostly of general 
principles and guidelines, was presented June 21-22 at a special Assembly in Tegucigalpa.
B. FINDINGS REGARDING KEY PROJECT COMPONENTS
1. SUPPORT FOR EDUCATIONAL REFORM
It is clear that USAID has played a key role since the Mitch disaster in promoting, sustaining, and 
participating in the development of a new and promising educational reform effort in Honduras. 
In particular, the Mission's efforts immediately after the hurricane to help gain consensus for the 
reform, design an action plan, conduct a national consultation, and develop a comprehensive 
proposal, were timely and of excellent quality.
Immediately after the presentation of the FONAC proposal, the SE , USAID and other donors 
began meeting to discuss ways to quickly institutionalize and operationalize the reform 
implementation mechanisms and develop a short list of concrete, high impact programs to be 
initiated in the next few months. 
 After Mitch, key members of the donor community in Honduras, including USAID, developed a 
productive set of joint working relationships among themselves and with the GOH education 
leadership. In time, these informal relationships took on structure., culminating in early 2000 with 
the establishment of MERECE. MERECE, while unofficial, became the principal vehicle for 
communication between the donors and the Honduran authorities on educational reform policy 
matters. USAID has played a major, and perhaps critical, role in the process of developing 
and institutionalizing this valuable mechanism.
Subsequently, an unofficial Honduran counterpart group, LA CUMBRE took shape, consisting of 
the Minister of Education and the heads of the three leading Honduran educational institutions –
the National Autonomous University of Honduras (UNAH), the National Pedagogical University 
(UPN), and the National Institute for Professional Training (INFOP). The formation of La 
Cumbre was a further, critical step and,again, the record shows that USAID played a key 
role in making it happen.
MERECE, with a current membership of over twenty organizations, meets at least monthly. Joint 
meetings with LA CUMBRE are arranged as needed. MERECE has formed four working groups 
covering the areas of Coordination of External Assistance; Teacher Training, Curriculum 
Development, and Management, which already have produced a number of useful substantive 
inputs to the process. The leadership of MERECE rotates every six months. John Helwig of 
USAID is the current Coordinator.
35
USAID Honduras should be commended for its work to date in the educational reform 
area, especially the coordination activities. We recommend that work continue along the 
lines already traced, with emphasis on targeted and measured support for the reform 
implementation mechanisms and the development of an integrated package of programs 
to assist in executing key reform components. Our views on the specific program activities 
that we think the Agency should consider funding will be detailed in our strategy paper.
2. PILOT PROJECTS
TELEBÁSICA, a program using educational television and texts to provide all key subjects at 
grades 7, 8 and 9, initiated its pilot activities in 1999. In that year the grade 7 curriculum was 
piloted in 32 sites in 12 departments. This year it is being piloted in 35 sites, in the same 12 
departments. The total initial enrollment for grade 7 in 1999 was 897. This year, the total initial 
enrollment in both seventh and eighth grades is 2,798. 
Team members visited one TB site, interviewed TB staff members, and reviewed and evaluated 
levels 1 and 2 of the TB texts. (See Appendix D for details.) The TV lesson we saw was 
excellent, and we found that the texts ranged from an acceptable to a high level of quality. 
However, we noted the need for further teacher training and ongoing supervision in order 
to achieve quality implementation of this program.
The series is delivered to classrooms on videotapes, which are copied at the ET office in 
Tegucigalpa. It is unreasonable to expect that a single video cassette re corder could meet the 
needs for cassettes in this program, taping daily broadcasts for 36 sites, and, in fact, we observed 
classroom difficulties caused by failure of equipment. Obviously, provision must be made for 
adequate equipment and for its replacement on a regular basis.
It is unclear whether TB should be reserved for use in the Basic Education program where it is 
currently being piloted, or whether it would also be useful in alternative delivery systems for 
grades 7 -9. The TB program is in the process of being evaluated by IEQ-II. That evaluation 
should yield a great deal of interesting and useful information to assist in further decisions about 
the TB pilot. However, given the pending evaluation, as well as the plethora of grade 7-9 
curricular frameworks currently available or under consideration, we feel it would be premature to 
make a definite recommendation regarding TB's future use at this time. Therefore, we 
recommend that TB continue as an interim program in Basic Education until evaluation 
results are received and final decisions are made in the SE regarding a standard grade 7-9 
curriculum for Honduras.
ESCUELAS CON ÉXITO was piloted during the 1998 school year, after which USAID funding 
was discontinued. The SE has adopted the program, with some revisions, calling it Salvemos 
Primer Grado (Let’s Save First Grade). Under the direction of a central unit in the MOE, the 
program is being implemented by all departments in the entire country. Orientation and training 
has been provided to educational personnel in each department. The program was adopted at the 
beginning of the school year, and has no operating budget within the SE this year, although it will 
be proposing a budget for school year 2001. Implementation at this time varies widely from 
department to department. 
The EE pilot in 1998 was undertaken in the first and second grades of 39 schools in the 
departments of Francisco del Morazán and Intibucá. Its objective was to reduce rates of repetition 
36
in grades 1 and 2, by improving teaching and student learning. See Appendix D for discussion of 
the problem of repetition. 
The EE project proposed three main interventions: 
First, it put an emphasis on training teachers in active, participative teaching methodology. 
“Maestros Distinguidos” (master teachers) were chosen to visit classrooms of participating 
teachers to provide coaching and support. Each master teacher was responsible for a group of 23 
classrooms. 
Second, staff worked with a group of teachers to “calendarize the curriculum”. The idea behind 
“calendarization” was to remedy the problem that many teachers never completed the first and 
second grade curriculum with their students, and had not identified which portions of the 
curriculum were of highest priority. The working group selected those portions of the curriculum 
deemed to be the most important, and actually recommended the days and weeks when those 
items should be taught. 
Third, teachers were asked to give periodic informal tests to check student learning on these key 
topics, and to keep the results on a master chart. Using these data, they were expected to provide 
extra assistance to children who had not mastered the indicated skills, and they were also 
encouraged to communicate regularly with parents, to show them the data on their child’s 
learning, and to enlist their support.
The project design focused on several key factors known to be contributors to students’ failure to 
learn. The focus on active, participative methodology is an important one, since it encourages 
students to become involved in and to feel responsible for their own learning. This kind of 
teaching also engages students’ interest, so that learning is not perceived as a passive task of 
memorizing and copying. The approach taken of identifying local skilled teachers is an excellent 
one. Their participation in the project would tend to further strengthen their abilities, thus 
strengthening local capacity to solve instructional problems. Furthermore, this design would tend 
to foster local support networks of teachers working together to discuss student learning and to 
develop solutions to problems particular to their own environment.
The “calendarization” seems likely to reduce the opportunity for teacher initiative. Especially in 
reading and language arts, there is no proof that individual skills must be taught in a certain 
sequence. A greater, underlying problem is that this model was developed from an outdated 
curriculum which needs revision. A review of the Rendimientos Básicos on which this system is 
based reveals no mention of reading comprehension at all in the first grade. The evaluation 
activities suggested require only factual answers, and they repeatedly emphasize that responses 
must be “exact”. The grid used for “calendarization” in Spanish, in fact, lists a specific word to be 
learned in each time period, rather than mentioning more fundamental competencies. 
The evaluation42 of the pilot reported that first grade repetition rates were reduced from 22% in 
1997 to 10% in 1998, and that the second grade rate of repetition was reduced from 19% in 1997 
to 8% in 1998. Unfortunately, team members noted flaws in the research design of the study 
which make it difficult to evaluate the success of the pilot. (See Appendix D for discussion of 
the pilot evaluation.) 
Nevertheless, even given these problems, the proposed system has several strengths, and it 
addresses a key problem noted not only in Honduran schools, but throughout Latin America: the 
37
failure of teachers to find out what their students are learning, and their consequent failure to 
design and select from a variety of instructional strategies to meet students’ needs. The model for 
ongoing formative evaluation in classrooms proposed by EE is one that requires teachers to use 
informal assessments of learning on a regular basis and to work again with those students who 
need special help. An additional strength of the model is the emphasis on communication with 
parents to enlist their support.
Therefore, we conclude that this pilot presented a valuable package of essential and well-focused 
interventions designed to improve student learning in the first and second grades. The model 
could be greatly strengthened if it were based on a revised, up to date curriculum that 
includes higher level thinking skills, and if the “calendarization” model were made 
somewhat more flexible. 
The fact that the Ministry has officially incorporated this program into its first grades countrywide 
bodes well for its sustainability in the longer run. The Ministry has requested financial assistance 
for the implementation of Salvemos Primer Grado. This will be an issue for further discussion in 
our forthcoming strategy document. However, it is likely that we will be recommending that 
improving the Salvemos Primer Grado be among the options to be considered for possible 
funding by USAID in the next five years.
APRENDAMOS MATEMÁTICA is a supplemental mathematics program intended for the first 
three grades using IRI. The program is delivered by means of a one-half hour radio broadcast or 
cassette. Its pilot activities began in the spring of 1999, when the first grade program was 
introduced in 10 departments, delivering the program to a total first grade enrollment in those 
departments of 202,000 children. In 1999 the project reports having trained 6,000 first grade 
teachers, as well as the departmental directors and specialists in the relevant departments. The 
first grade program has now been introduced nationwide, and piloting of the second grade program 
is taking place in the same 10 departments. 
Team members visited AM classes in several schools, in the departments of Cortés and Santa 
Bárbara as well as in the greater Tegucigalpa area. ( See Appendix D for details of our 
observations.)
The lessons are fast paced, with frequent changes of task and are interspersed with physical 
activities and songs. The children almost uniformly were listening and responding with 
considerable enthusiasm. Teachers varied in their level of participation, some actually leaving the 
room during the broadcast, while others participated actively and used a variety of strategies to 
assist children having difficulty. We observed that in schools where the director was actively 
involved and enthusiastic about AM, the teachers tended to follow suit. We recommend 
continued training and ongoing supervision of teachers to ensure maximum impact of the 
AP program.
The program is not intended to stand alone, but should be a supplement to the regular math 
instruction. To maximize program effectiveness it would be desirable that the teacher provide 
follow-up activities in areas in which the children seem to need more reinforcement. However, in 
classrooms we saw, the lessons ended abruptly and children never continued with any related 
math activity. It would be helpful to include in the ongoing teacher training some additional 
strategies to use to deepen and extend some of the concepts introduced in the 20 minute radio 
broadcast.
38
We noted logistical difficulties regarding timing of the radio broadcasts and availability of 
sufficient radios and/or tape recorders. In particular, in some locations the programs for 1st and 
2
nd grade are broadcast at the same time, but some schools only have one radio. AP staff 
suggested that a viable solution for providing radios might be to request that local parent 
organizations, rather than individual teachers, buy them. The radios could then be shared with 
other groups needing them in the local area, and ownership and responsibility for the radios would 
be clearly defined. Another problem, which will require more creative solutions at the local level, 
is that many teachers in rural areas have two or three grades in the same classroom. This 
situation, difficult already, will become even more complicated next year when broadcasts will be 
provided for first, second and third grades.
A summative evaluation of AM conducted in 1999 indicated that math scores of children were 
higher in classes participating in the program than in control groups. However, the evaluation was 
based upon the Rendimientos Básicos developed under the PEEP Project, which presents 2 
important problems:
•The AM program was evaluated against criteria (the Rendimientos Básicos) that played no role 
in its program design. (It is based on a curriculum that has been used in a number of Central 
American countries, most recently in El Salvador).
•PEEP’s Rendimientos Básicos contain no ‘ejes transversales’ – the one element of Honduran 
curriculum that was actually inserted into AM’s program. Thus, the impact of their inclusion 
is largely unknown since they were not included in the evaluation design.
The above factors would suggest that the tests might not have indicated the full extent of the 
children’s learning under the AM program. Unfortunately, the data treatment and presentation did 
not follow proper procedures, particularly in regard to reporting appropriate statistical significance 
tests. Therefore, it is impossible to determine whether the contrasts reported are significantly 
different from differences that can be expected by chance alone. 
Regarding sustainability, it seems likely that the project can gradually work its way towards self￾sufficiency, as it gradually receives more support from NGO’s and local businesses. It has the 
advantage of operating within the school system, so that ongoing supervision and support is 
already expected from the Ministry and its decentralized education units.
In spite of the technical and implementation problems mentioned, AM is a program that is 
assisting children in their learning of mathematical concepts and is a valuable addition to 
the primary grades mathematics curriculum. It can be made better by the inclusion of the 
improvements we have suggested. A redesigned evaluation focusing on issues related to teachers’ 
classroom use of this resource might provide still more guidance about ways to further enhance 
the program. 
C. PROJECT ISSUES TO BE ADDRESSED
The objectives of BEST Component III, at the outset, were not well defined or articulated. But, 
inasmuch as the Component de facto was not used for intended program activities until after the 
Mitch disaster, the team has not gone more deeply into the reasons for the apparent deficiencies in 
the original design. In the aftermath of the hurri cane, the objectives were redefined and became, 
in effect, support of the reform process and funding of pilot projects to lay the groundwork for 
reform. In our view, the new focus of the actions that ensued were on target, and the 
results so far, have been positive.
39
1. SUPPORT OF EDUCATIONAL REFORM
The joint effort to develop a national educational reform plan via a broad and open national 
consultative process responded, in part, to the desire to avoid the fate of several earlier reform 
plans, that are today gathering dust on various shelves. USAID and others felt that if enough 
sectors of society, particularly the political parties and the pre -candidates for the 2001 elections, 
could be engaged and converted into active participants and supporters of the reform, it would 
increase the chances that the incoming government, whether Liberal or Nationalist, would 
continue its implementation. 
While it is impossible to predict what will happen when the new government takes over, there are 
some well-founded reasons to be cautiously optimistic that the basic elements of the reform will 
continue. In other words, it is a reasonable expectation that the incoming Administration, while it 
surely will want to put its own stamp on the reform program, will do so based largely on the plans 
developed over the last year and a half, including continuation of ongoing projects, rather than 
discarding everything and starting over.
One reason, and arguably the most important, is that through their involvement in the preparatory
period, the major political parties and presidential candidates have publicly declared their support 
and have committed themselves, should they come to power, to continue the reform. Similarly, 
the major actors and institutions in the education sector, the private sector, key elements of civil 
society, and even the teacher unions, though probably with greater reservations, are committed to 
supporting and many of them are likely, should the reform come under attack, to help defend it
There also is increasing evidence that the seriousness of Honduras’ situation in the emerging 
global economy has hit home. This is felt most strongly, perhaps, in the private sector, but 
globalization forces affect almost every major institution in the country. There is wide and 
growing consensus, as well, that the key to unlocking the Gordian knot in Honduran 
development, which is abysmally low productivity, is human resource development, especially 
education, health, and nutrition.
Another factor in the equation is the fact that Honduras badly needs additional debt relief to help 
reduce its fiscal deficit and has recently been approved for $900 milllion in debt relief under the 
Highly Indebted Poor Country (HIPC) Program. Honduras has prepared a Interim Poverty 
Reduction Strategy that puts forth a number of initiatives and specifically cites the proposed 
education reform as among the steps Honduras will take to meet IMF/lender requirements.
Finally, given Honduras’ high level of dependence on international aid and investment, the 
bilateral and multilateral donor community in Honduras, acting in concert through MERECE, is 
in an unusually strong position to help assure continuity in the reform process.
Good groundwork has been laid, via the preparatory actions taken over the last eighteen months, 
for a sound institutional structure to oversee the reform. At the time of writing (July, 2000), 
efforts to create the necessary mechanisms are underway. Discussion of what would be the most 
appropriate legal structure for the reform is also taking place.
The Team believes that a top immediate priority is to help the SE get a short list of 
feasible, high-impact, and visible educational improvement activities quickly underway to 
demonstrate that the long-awaited reform is not just words but deeds. What specifically 
should be done and by whom will need to be worked out between the SE and the donors through 
MERECE. One possibility that should be seriously considered is technical assistance to the new 
40
unit that the SE is setting up to coordinate implementation of the reform in the areas falling under 
the Ministry’s jurisdiction.
The proposal to subsequently organize a further, highly visible conference or other event to 
publicize the fact that reforms are actually happening strikes us as appropriate, assuming, of 
course, that there are real achievements to talk about. 
2. PILOT PROJECTS 
We believe that the pilot projects, including AM, which was funded with ESF funds, have been 
generally helpful in helping prepare the ground for reform. 
At present, USAID Honduras is actively engaged in reassessing its education strategy to develop a 
set of programs to support the proposed education reforms in ways appropriate to AID’s policies, 
capacities, and resources and the roles and interests of other donors. The results to date of the 
pilot projects are inputs to that process.
3. MANAGEMENT STRUCTURE AND IMPLEMENTATION METHODS
There are two points to be made about the adequacy of the current USAID management structure. 
From our perspective, it is leanly staffed for its many current responsibilities. The post-Mitch 
emergency reconstruction program will gradually wind down, of course, but, the demands of 
supporting the new educational reform are already creating new and different needs for Mission 
staff support, and these needs will only become more compelling over coming months. The Team 
urges the Mission and AID/W to carefully analyze future reform-related requirements to assure 
that staff numbers and skills keep pace with program changes. The Team believes that, in general, 
USAID should define its role as overseeing, coordinating, and continuously evaluating the new 
educational reform programs, as well as assuring that projects receive the technical, financial, and 
other resources they need in a timely fashion. 
Every project should be designed to provide within it at least a critical minimum of USAID 
funded technical support. The ways this will be done will vary. A technical assistance contract or 
task order under an IQC probably is the preferred vehicle for major projects, but the relatively 
high cost of these mechanisms necessarily will limit their use. In most cases, a programmed series 
of visits by consultants working with local counterparts will be sufficient. We believe that the 
scope of work of a consultant making regular visits can be structured to cover more than one 
activity and include staff training in his/her area of responsibility, as well as ongoing technical 
assistance. While evaluation should be continuous and formative, the Team believes that formal 
evaluations of all activities should occur at two to two and a half year intervals.
VI. CONCLUSION
In evaluating the several components of the generally successful BEST Project, we have noted a 
consistent tendency to emphasize both quantity and cost saving at the expense of quality. In fact, 
project objectives have often been stated in ways that would encourage program staff to push 
large numbers of students through the program as quickly as possible rather than to focus on the 
quality of their learning experience. Although it should clearly be a high priority both for USAID 
and for the Government of Honduras to deliver services in the most cost efficient manner, we 
believe it should be an even higher priority to develop and deliver programs that demonstrate a 
commitment to high quality education. High quality education requires excellent curriculum and 
41
materials, excellent teaching, thoughtful and consistent supervision, careful evaluation, and well￾organized management and logistics. USAID has a commendable record of assistance to 
Honduran education. We believe that the improvements we have suggested in the BEST project 
components will assist USAID in its continuing efforts to help Hondurans achieve success in their 
impre ssive educational reform effort.
42
Notes
 
1
 However, the evaluation team finds no reference to findings from piloting of these materials, suggesting that piloting, if actually accomplished, was somewhat 
informal. 
2
 We also observed children attending classes with their parents, a valuable way for them to learn but, more importantly, so come to appreciate and value a 
culture of learning.
3 Van Steenwyk, Ned, USAID report, Las Mujeres en el Programa Educatodos, 1998; HIID, Girls' and Women's Education Activity, 
Component III, Appendix I, Draft Review of Honduras Baseline Data, 1998.
4
 There are some inconsistencies in project documents as to whether the service objective was intended to specify enrollments or “graduates”, i.e. students 
who have successfully completed one grade level.
5
 The initial project proposal focused on immediate economic returns; we believe a more appropriate outcome measure is cumulative economic and non￾economic returns, discounted over an appropriate payback period.
6
 The project design does not specify volunteer term/level of effort. We assume that the intended objective was to specify one volunteer-facilitator per 100 
lesson semester-long course, i.e. 15,000 volunteer-semesters of effort. Thus, the 7,500 volunteer-years is 100% of the planned objective .
7
 Interested students who want to join a class before the semester is too far advanced are allowed to join based on their apparent skill level and motivation 
and, in some cases, facilitators may help them catch up by covering the lessons they missed in the textbook.
8
 Even if it the EDUCATODOS testing system had reliably measured post-program competencies, it is not possible to estimate the “value-added” 
contribution of program participation because only some of the enrollees were actually given a pre-test. The research literature on adult learning is clear that 
competencies acquired through experience and self-study often bring adults to functional levels higher than those they had when they left school. Nonetheless, 
since many program non-completers may have achieved gains before leaving the program, we estimate that about 70% of enrollees made substantive gains, 
although the completion rate is lower. 
9
 These policy commitments can be found in a variety of documents including the FONAC proposals, the curriculum framework document developed by the 
MERECE group, and others. Se also Jorge Sanguinetti's keynote address to the 1999 ET-sponsored conference.
10 These frameworks are the SCANS framework (Secretary’s Commission for Achieving the Necessary Skills) developed by the U.S. Department of 
Education and the U.S. Department of Labor and the Equipped for the Future framework developed by the National Institute for Literacy. While these were 
developed in a United States sociocultural framework they are, nonetheless appropriate for Honduras in that the “high performance” competencies identified in 
these frameworks are universal features of the 21st century global economy and of increasingly stressful 21st century family and community life.
11 There are deficiencies in the Rendimientos Basicos framework itself which we discuss later in this report but in this section we focus specifically on the 
EDUCATODOS materials.
12 Project staff have attempted to overcome some of these limitations with training designed to support sound instructional methodologies but it is not clear 
how much can be done within the current instructional design.
13 The project has distributed 2,267 sets of cassettes with 50 lessons in each cassette.
14 We estimate radio distribution of ET programming at about 0.10 Lempiras/per instructional hour or slightly over $10 Lempiras 
per 100 hr. Course. Assuming that about 2/3 of students successfully complete a grade, the distribution cost is about $15 lempira or 
about $1 per student grade-level successfully completed.
15 We were particularly impressed, for example, by the initiative of a promotor in Santa Barbara who had set up an ET program in the local prison. We were 
present at a meeting in which he very skillfully resolved some incipient problems—in a very collegial discussion with two inmate-facilitators and a prison 
administrator. We also saw a similarly skillful negotiating meeting in which the promotor secured the local school director’s permission to use the school as a 
site for the upcoming ET pilot site classes for grade 7.
16 Thus, the project does not have access, for example, to financial data on the cost per textbook in order to determine whether the decision to acquire presses 
to print texts in-house was justified or not. Similarly, the overall project budget does not provide detailed breakouts for key cost centers such as: purchased 
radio station time, purchase of cassette recorders and tapes. We are confident that the underlying records are available. Our concern is not fiscal accountability 
but, rather, the availability of such information for making informed management decisions.
17 Ideally, the project manager should have access to monthly reports listing obligations from each fund source, cash received, and expenditures made against 
each fund source.
43
 
18 In reviewing the cost data on radio broadcasting distribution we noted there was excellent documentation of costs and a adequate 
basis for estimating total in-kind donations in the form of discounts from market rates but that the calculations needed to 
systematically assess cost-effectiveness (e.g. cost/hour/station, cost/hour/impression, value of donated promotional time).
19 We understand that the ET program design assumes that pre-program placement tests are given to all students. This was not 
reflected in the accounts we heard from field personnel about their actual operational procedures.
20 We do not believe such a system should be developed, although well-designed research on random samples of sites might provide valuable insights into 
actual patterns of attendance and dropout. The Van Steenwyck report on “Los Desertores de EDUCATODOS” provides this sort of information, although 
we suggest additional tabulations for diagnostic purposes.
21 In Appendix H we detail both our assessment of current efforts to estimate economic benefits and our suggestions for improvements.
22 These items do not take advantage of scaffolding or other techniques to use multiple-choice items for assessment of problem solving or other higher order 
skills.
23 Domain is the area of knowledge and/or skill to be measured. In criterion-referenced testing this includes an operational definition of the “piece” of 
curriculum to be measured and the associated item development specifications.
24 Among the many potential test items, some work better in reliably and validly assessing a student’s skills than others. The idea is to explore the 
performance of a number of potential test items empirically. There are many techniques in criterion-referenced testing for defining domains and for developing 
and choosing items that validly and reliably sample behaviors that serve as indicators of mastery of these domains – in appendix B are cited a number of 
works that put forward well tried methods to accomplish this. The Unidad de Medicion de la Calidad (UMCE) uses a set of appropriate techniques derived 
from the work of Popham, Haladyna and Esquivel. Currently, no such methods are being used in ET. 
25 In fact, the definition of a test in criterion-references measurement is the n number of items necessary to validly and reliably measure a domain. Never is one 
item deemed a valid sample to substantiate inferences regarding what criteria students have attained.
26 There is little conceptual difference between attempting to infer how much students know about adding 2-digit numbers based on their performance on one 
item than in inferring how well Honduran students learn about “operations with whole numbers” from the number of items that one student answers 
correctly.
27 ET evaluation staff also cite concerns with costs – stating that current program constraints make following full development 
procedures for criterion-referenced tests unfeasible. If this is so, it can only be regarded as an important flaw in the project design.
28 These are Standards for Educational Measurement established by the joint commission of the American Psychological Association, the American 
Educational Research Association and the National Council of Measurement in Education (APA/AERA/NCME)
29 In fact, currently the Office of Management and Budget (OMB) of the US Federal Government does not permit publication of evaluation reports under the 
imprimatur of any federal agency without this critical datum, it is deemed so essential in the assessment of claims made on the basis of evaluations. 
30 If pre-program skills levels were validly assessed, not only would such assessment be useful for placement/diagnostic purposes, but it would be possible to 
more reliably generate estimates of the value of program services—since some students have surely entered at lower skill levels than their grade level schooling 
implies and others have entered with higher skill levels.
31 The relevant research publications we have reviewed include Van Steenwyck's work on program dropouts and women's 
experience, as well as Mary Ellen Duke's very interesting field research on the program experience of girls and women.
32 ET may indeed have been doing a good job of decreasing illiteracy in rural areas of Honduras, or perhaps in both rural and urban areas. But whether this is 
the case is now unclear. Whether the strategies used by some departmental coordinators or some promoters worked better than others is also an unknown. 
Whether some types of students served by the project did better than others would have been valuable input for efforts to fine-tune instructional design to 
improve equity. These opportunities have been lost.
33 USAID can potentially play an important role in leveraging attention to quality instruction as a means to improve overall project cost-effectiveness and it is 
likely that it must assume such a role because, without such a focus, investments in basic education will not be able to make a substantive contribution to 
Honduras overall economic development and poverty reduction.
34 It is not necessary to replicate these frameworks but it is necessary to consider them in developing a uniquely Honduran set of curriculum objectives for 
knowledge, information management skills, and development of the competencies needed to be a successful lifelong learner.
44
 
35 Ideally graduating students might be given a choice among several kinds of books—to make the point that literacy includes thinking about what one wants 
to read and choosing the books which one likes.
36 The answer to this problem represents a critical element in determining the policy relevance of assessments and thus lies at the very core of the use of 
assessments in educational policy and educational program evaluation. 
37 For convenience we use the terms curriculum validity and instructional validity as they provide distinctions useful to this exposition. However, they are 
more properly regarded as complementary facets of construct validity.
38 A number of standard works are mentioned in Appendix B that will be helpful in determining appropriate technical options
39 The International Association for the Evaluation of Educational Achievement (IEA) introduced the notion of Opportunity-To-Learn (OTL) in the 1960s as 
a means of ensuring the technical validity of their assessments. Initially, what was done is that teachers were requested to look at a copy of the items on the 
assessments, and report whether or not they had provided their students the instruction necessary to solve these items correctly. The simplicity of these early 
measures belies the enormous conceptual importance of OTL which has consistently proven to be an important explanatory variable when exploring leaning 
determinants. More recent work – cited in Appendix B – advances the original ideas of OTL under a new theory of the measurement of educational 
opportunities.
40 To validate a test is to provide evidence that theory and empirical data support the inferences that the test designer proposes for the test. If a test is 
intended to be used so that scores can be interpreted as indicators of the level of mastery students have achieved over the content and skills that make up the 
goals of their schooling, professional standards as codified by the joint commission of APA/AERA/NCME require that test developers validate them for this 
purpose. Professional standards in this regard are presented in Standards for Psychological and Educational Measurement (1999 edition) – and the very best 
work concerning appropriate validation procedures is presented in a number of important works, cited in Appendix B. 
41 Ideally, “authentic” radio spots featuring successful students talking briefly about their experience would be a useful addition to the current campaign based 
on “professional” literacy promotion messages. Social marketing research shows that high-impact messages are those in which listeners can hear themselves 
and people they care about. We believe such a campaign would also affect potential financial donors and collaborating organizations.
42 van Steenwyck, Ned, Escuelas con Exitos: Los Resultados, September, 1999