MIDTERM EVALUATION
OF THE
LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS
INITIATIVE
By:
Cesar Chelala
Mary Ruth Horner
Submitted by:
LTG Associates, Inc.
and
TvT Associates, Inc.
Submitted to:
The United States Agency for International Development
Under Contract No. HRN-I-00-99-00002-00
May 2000
Midterm Evaluation of the Latin America and the Caribbean Integrated Management of Childhood Illness
Initiative, was prepared under the auspices of the U.S. Agency for International Development (USAID)
under the terms of the Monitoring, Evaluation and Design Support (MEDS) project, Contract No. HRN-I￾00-99-00002-00. The opinions expressed herein are those of the authors and do not necessarily reflect the
views of LTG Associates, TvT Associates, or USAID.
Information about this and other MEDS publications may be obtained from:
Monitoring, Evaluation and Design Support (MEDS) Project
1101 Vermont Avenue, N.W., Suite 900
Washington, DC 20005
Phone: (202) 898-0980
Fax: (202) 898-9397
scallier@ltgassociates.com
ACRONYMS AND FOREIGN TERMS
ACS Agente comunitario de salud (community health agent)
AIEPI Atención Integrada a las Enfermedades Prevalentes de la Infancia (IMCI)
AIN Atención integral al niño (integrated attention to the child)
ARI Acute respiratory infection
BASICS Basic Support for Institutionalizing Child Survival Project
CDD Control of diarrheal diseases
CCH Community and Child Health
COMSAIN Comunicación en Salud Infantil (Communications in Infant Health)
COTALMA Centro de Capacitación para la Lactancia Materna y Nutrición
(Training Center for Breastfeeding and Nutrition)
CTO Cognizant technical officer
EPI Expanded programme on immunization
G/PHN Bureau for Global Programs, Field Support and Research
Center for Population, Health and Nutrition
HIS Health information system
HN Office of Health and Nutrition
IMCI Integrated management of childhood illness
IMR Infant mortality rate
IR Intermediate Result
LAC Bureau for Latin America and the Caribbean
LACI LAC Regional IMCI Initiative
M&E Monitoring and evaluation
MINSA Ministerio de Salud (MOH)
MOH Ministry of Health (recently renamed Secretary of Health in Honduras)
MSH Management Sciences for Health
NGO Nongovernmental organization
PAHO Pan American Health Organization
PCMI Proyecto de Capacitación Materno Infantil
PROCOSI Programa de Coordinación en Salud Integral (Coordination in Integrated Health Program)
PVO Private voluntary organization
R4 Results Review and Resource Request
RPM Rational Pharmaceutical Management
RSD Office of Regional Sustainable Development
SO Strategic Objective
UNICEF United Nations Children’s Fund
USAID United States Agency for International Development
WHO World Health Organization

CONTENTS
ACRONYMS AND FOREIGN TERMS
EXECUTIVE SUMMARY ................................................................................................. i
I. Introduction ..................................................................................................................1
A. Background of the Initiative ..............................................................................1
B. Purposes of the Evaluation.................................................................................3
C. Field Procedures.................................................................................................4
D. Evaluation Team Composition...........................................................................5
II. Project Design Issues....................................................................................................7
A. Role of the IMCI Regional Initiative .................................................................7
B. Overall Project Design Components..................................................................8
C. Appropriateness of the USAID Results Framework..........................................8
III. Management and Administration Findings................................................................13
A. Management Structure ........................................................................................13
B. In-country Coordination of Activities.................................................................15
C. Staffing................................................................................................................17
D. Information Sharing among IMCI Countries......................................................18
E. Conclusion...........................................................................................................19
IV. Implementation Findings............................................................................................21
A. Status of Activities to Date .................................................................................21
B. Development and Use of Tools...........................................................................25
C. Interagency Collaboration...................................................................................26
D. Community Participation ....................................................................................27
E. Barriers to Implementation..................................................................................28
F. Facilitating Factors ..............................................................................................30
V. Special Achievements of the LAC Regional IMCI Initiative ....................................33
A. Advocacy ............................................................................................................33
B. Institutionalization of IMCI ................................................................................33
C. Coherence of IMCI’s Message............................................................................34
D. Training Activities ..............................................................................................34
E. Creation of Tools and Dissemination of Experiences to Other Countries ..........35
F. Ability to Work under Adversity.........................................................................35
G. Sustainability.......................................................................................................36
VI. Conclusions................................................................................................................39
A. Overall.................................................................................................................39
B. Project Design .....................................................................................................40
C. Management and Administration........................................................................41
D. Implementation ...................................................................................................42
E. Sustainability.......................................................................................................43
VII. Recommendations......................................................................................................45
A. Recommendations to USAID/LAC ....................................................................45
B. Recommendations to PAHO and BASICS .........................................................46
C. Recommendations to All Partners (USAID, PAHO, and BASICS) ...................46
D. Priority of Allocating Resources for In-country Activities.................................47
TABLES
Table 1: Aspects of Implementation that are Working Well ............................................ iii
Table 2: Aspects of Implementation that are not Working Well ...................................... iv
ANNEXES
A: IMCI Action Memorandum.........................................................................................51
B: Grant to PAHO ............................................................................................................57
C: USAID Results Framework .........................................................................................67
D: LAC IMCI Schematic Design......................................................................................77
E: Scope of Work .............................................................................................................81
F: Evaluation Schedule.....................................................................................................89
G: Documents Reviewed ..................................................................................................93
H: Interview List...............................................................................................................97
i
EXECUTIVE SUMMARY
In the 1980’s, the World Health Organization (WHO) and the United Nations Children’s
Fund (UNICEF) developed the strategy, integrated management of childhood illness
(IMCI). The objective of IMCI is to decrease morbidity and mortality from common
childhood illnesses through the application of a low-cost strategy that identifies and treats
life-threatening illnesses more quickly than through traditional vertical disease control
programs. The IMCI strategy encompasses a range of interventions for the prevention
and management of major childhood illnesses, both in health facilities and in the home. It
incorporates many elements of diarrheal disease and acute respiratory infection (ARI)
control programs and child-related aspects of malaria control, nutrition, expanded
programme on immunization (EPI), and essential drugs programs. According to WHO,
implementing the strategy requires and facilitates active collaboration among these
programs and requires efforts at different levels of the health system.
In the Americas, the United States Agency for International Development’s (USAID)
Bureau for Latin America and the Caribbean (LAC) developed a five-year, $5 million
regional activity for more effective delivery of child health services in response to the
control of diarrheal disease (CDD), ARI, and malnutrition. This program, the LAC
Regional IMCI Initiative (LACI), has been implemented through two partners, the Pan
American Health Organization (PAHO) and the Basic Support for Institutionalizing Child
Survival Project (BASICS), since its authorization in March 1997. The eight countries
selected for emphasis, based on their high infant and child mortality rates, were Bolivia,
Ecuador, El Salvador, Guatemala, Haiti, Honduras, Nicaragua, and Peru. In the late
1980’s, these countries were designated by LAC for emphasis in child survival
programming.
A midterm evaluation was conducted to review the USAID Results Framework, evaluate
overall progress to date, identify major accomplishments and barriers, and propose
recommendations for midcourse corrections. A 2–person team conducted 1 week of
interviews in Washington, D.C., beginning in early April 2000. Field work then
commenced and involved visits of 1 week each to three of the eight target countries:
Honduras, Bolivia, and Peru.
A. ACTIVITIES FUNDED
LACI took on a supporting role for introducing IMCI in the eight target countries.
PAHO and BASICS managed independent budgets and activities from a joint work plan.
Examples of the use of LACI funds include:
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
ii
ß advocacy to convince countries to adopt IMCI as policy;
ß adaptation of WHO’s generic IMCI materials for use in the different
countries;
ß selected in-country activities that had regional implications (e.g., introduction
of the rational pharmaceutical management tools and manual);
ß regional meetings to address specific issues and for annual information
sharing; and
ß two national evaluations (Bolivia and Peru).
Because of the nature of the activities supported by the LACI, a measure of their
effectiveness is the degree and quality of the implementation of IMCI at the national
levels.
B. FINDINGS
Findings regarding the Results Framework and progress to date include:
1. Strategic Objective level: As presently stated, the two indicators are not
appropriate for the initiative, although significant progress towards achieving
the goal was observed by the evaluation team.
2. IR 1 (IMCI information and policy adoption) and IR 2 (national IMCI
operational plans): Substantial progress has been achieved; seven of eight
countries have adopted IMCI and have developed plans.
3. IR 3 (improved country capacity): The indicators need revision;
considerable progress has been achieved.
4. IR 4 (use of monitoring and evaluation): The indicator needs revision;
progress is being hampered by the low use of appropriate tools.
Based on the findings of this evaluation, the LAC Regional IMCI Initiative has made
substantial progress. A summary of the progress to date is provided in the tables on the
following two pages. Table 1 summarizes aspects of implementation that are working
well, suggests the reason(s) for this, and provides specific examples from the countries
visited. Table 2 describes aspects of implementation that are not working well, suggests
the reason(s) for this, and provides recommendations.
EXECUTIVE SUMMARY
iii
Table 1: Aspects of Implementation that are Working Well
Which aspects are
working well? Why? Examples
1. USAID/LAC’s Role as a Catalyst Activities are well chosen for catalytic role Regional meetings and conferences
stimulate crossfertilization of ideas;
training of facilitators out-of-country
stimulates training activities in-country
2. The Partnership Generic roles for PAHO and BASICS are
appropriate; play to their strengths; annual
meetings (PAHO, all partners)
PAHO is the natural counterpart to the
Ministry of Health (MOH) in-country;
BASICS is the natural connection to
USAID Missions and the Global Bureau*
1. Involvement of the Ministries of
Health
LACI was perfect fit for MOH’s emphasis on
decentralization and need for improved quality
of health services
LACI complements MOH’s own
contributions of human, financial, and
physical resources to IMCI
4. Creation and Translation of Tools Right tool(s) in the right place (Latin America)
at the right time
All 3 countries visited are using materials
translated for Latin America from generic
WHO, PAHO, and BASICS materials
5. Expansion of IMCI Due to early implementation success of IMCI,
the training component has attracted additional
financial support; general drug availability at
most health centers supports expansion
Financial support for expansion is
available in HondurasæUSAID/
Honduras; BoliviaæUSAID/Bolivia, for
community IMCI, World Bank; Peruæ
World Bank, Proyecto 2000;
nongovernmental organizations (NGOs) in
3 countries (e.g., CAREæHonduras, Plan
InternationalæBolivia and Peru)
6. Quality of Clinical Training High-quality and locally adapted training
guides, manuals, and supporting materials;
selected cadre of trained facilitators
Enthusiasm expressed by hospital
directors, facilitators, participants (nurses,
doctors), promoters, mothers
7. Intersectoral Collaboration
(health and education)
Opportunities to reinforce content and channels
of communication
Peru: The health center at Cura Mori,
Piura, has developed a mutually
reinforcing relationship with teachers at
local schools for health education and
promotion (e.g., teaching hygiene,
sanitation)
8. Community Participation Strong consensus on need for working at
community level with promoters, mothers,
extended families about nutrition and danger
signals
Close and mutually respectful partnership
between NGOs and MOH to implement
IMCI at the community level; builds on
NGOs’ proven expertise
*Global Bureau = Bureau for Global Programs, Field Support and Research
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
iv
Table 2: Aspects of Implementation that are not Working Well
Which aspects are not
working well? Why? Recommendations
1. Central (D.C.-based) managerial
structure (the steering
committee) does not meet
expectations
Personnel turnover (USAID/LAC and
BASICS); selective delegation of authority to
steering committee; members’ roles and
responsibilities not sufficiently delineated
from the beginning
More authority to steering committee (along
lines of Health Sector Reform Initiative);
practice team building with an outside
facilitator; prepare a detailed implementation
plan for rest of project, including better
delineation of roles and responsibilities;
appoint a partner with overall coordinating
role for LACI in each country
2. SO indicators are not appropriate
for regional initiative
One donor (LAC) cannot be responsible for
indicators which reflect work of all IMCI
donors and MOH in each country; national￾level indicators for regional project; cannot
be collected without extraordinary resources
Consider adaptations from other LAC
regional initiatives and earlier draft
articulation of indicators for LACI; address
in post-evaluation detailed implementation
plan
3. Overemphasis on clinical
training
Clinical training is the number 1 design
component of WHO’s IMCI; everyone
started here; traditional design approach only
emphasizes actual training activity, not
overall situation with supporting systems
4. Weak health system support
(quality of services, supervision,
referral, integration of IMCI
indicators into health information
system [HIS], monitoring and
evaluation [M&E], cost analysis
of drug use, cost analysis of
IMCI adoption)
WHO’s IMCI design was divided into three
parts; because of the extreme excitement,
energy, and focus put on training, the support
components have almost been ignored;
support components are not as visible nor as
glamorous as training
5. Existing tools are not widely
used (supervision, drug
management, M&E, health
facility assessment, cost
analysis)
Not enough holistic planning for individual
MOH country-level annual plans of
operations
Address these system support components in
preparing detailed implementation plan (in
D.C.) (e.g., training follow up and ongoing
supervision must be included in training
budgets; incorporation of tools and the
human resources required to introduce them
must be included in annual IMCI plans and
budgets)
6. LAC Regional IMCI Initiative
not receiving credit for activities
supported
In-country implementers do not always know
original source of funds and also have
sources of other regional funds; giving credit
where credit is due is a learned skill
Articulate need in concrete terms in revised
roles and responsibilities of partners;
reinforce need with in-country collaborators;
give examples of positive recognition where
it has happened (e.g., placement of agency
logos in Bolivia’s statistical document
launched with Meta 2002 celebration)
C. ACHIEVEMENTS
All three direct partners in LACIæUSAID/LAC, PAHO, and BASICSæhave reason to
be proud of their accomplishments. The Latin American region is serving as the test case
for many aspects of IMCI and in just three years the results obtained are substantial. On
a policy level, the ability to garner the political commitment at the highest levels of the
MOH has been critical to the effective launching of the IMCI Initiative in Latin America.
On a personal level, the dedication, enthusiasm, and optimism of MOH and NGO staff,
EXECUTIVE SUMMARY
v
promoters, and mothers interviewed are proof that IMCI’s success will ultimately be
decided by each individual’s willingness to change her/his own behavior. The evidence
obtained during this evaluation is that IMCI can be successfully implemented and
managed.
Additional specific achievements of the LACI are detailed below.
Advocacy
Seven out of eight countries chosen for the LACI have adopted the IMCI strategy.
Institutionalization of IMCI
In Honduras and Peru, IMCI is incorporated into the MOH’s operational plans. In
Bolivia, IMCI is incorporated into the MOH’s Seguro Básico de Salud.
Coherence of Message
This is a key factor in any new strategy. The IMCI message has been conveyed undiluted
and undistorted to different audiences at different levels.
Training Activities
Although in-country training in itself is not part of the regional initiative, its quality is
related to the quality of the training manuals and the expertise of the trainers. Over 8,000
people were trained with manuals adapted from generic materials through the use of
regional funds. Those interviewed had good knowledge of IMCI and an integral vision of
children’s health. As a result, there has been better counseling of mothers, more
awareness of mothers and community members of their role in children’s health, better
diagnosis of pneumonia, more awareness of risk factors, and better referrals by health
workers. A consequence that has not yet been documented but has been stated in several
instances has been fewer children’s deaths.
Adaptation and Dissemination of Tools In-country and to Other Countries
Examples include:
ß Rational Pharmaceutical Management (RPM) developed by Management
Sciences for Health (MSH) with BASICS and PAHO support was field tested
in Bolivia and Ecuador. This tool collects cost data, assesses the role of the
private sector, determines the availability of drugs at health centers, and
assesses prescribing practices. It should be more widely used and is an activity
that could be supported by regional funds.
ß With LACI support, the Centro de Capacitación para la Lactancia Materna y
Nutrición (Training Center for Breastfeeding and NutritionæCOTALMA) in
Bolivia modified the community IMCI materials that were originally
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
vi
developed by PAHO/UNICEF. With technical assistance from World
Education, COTALMA also adapted materials for auxiliary nurses, called
Complementary IMCI.
ß Because of the success of the implementation of IMCI in Latin America, two
PAHO representatives were invited to present their experience in an upcoming
international meeting in South Africa.
ß Peru and Honduras made their own shortened versions of the clinical IMCI
training course (from 11 to 7 days).
Ability to Work under Adversity
Examples include the following:
ß in Honduras, despite the worst hurricane in the last 25 years (Mitch), the
initiative progressed;
ß in Bolivia, a dedicated facilitator works on training activities on her vacation
time; and
ß in Peru, after work hours, an entrepreneurial young doctor conducted a
thorough study of drug use in a group of clinics.
Sustainability
As of this evaluation, the LACI shows great promise for making a major contribution to
the eventual sustainability of IMCI within nationwide health systems in the target
countries. Currently, there are five different approaches being pursued to promote
sustainability, all with initially promising results, although mostly on a limited and pilot
basis to date:
ß incorporation of IMCI into MOH policies,
ß stimulation of donor support and funding for the expansion of IMCI,
ß incorporation of health issues into activities of the education sector,
ß training of community-based health promoters and mothers in IMCI
messages, and
ß incorporation of IMCI into the curricula of nursing and medical schools.
D. CONCLUSIONS
EXECUTIVE SUMMARY
vii
In summary, USAID’s latest Results Review and Resource Request (R4) for
USAID/LAC (March 22, 2000) states: “The IMCI initiative catalyzed acceptance of
IMCI in the Americas.” These 10 simple words are an incredible understatement of the
enormous amount of effort and significant positive results seen so far. The progress to
date is already remarkable for this stage in a multicountry effort with fairly limited
funding. The recommendations presented herein are practical and viable, given the
resources available, and if implemented as soon as possible, could well lead to
developing some synergy between and among collaborators. It is also possible that if
synergy is developed, the total achievements for LACI will exceed the sum of their
individual parts.
E. RECOMMENDATIONS
The recommendations that follow are directed towards USAID/LAC and the two partners
(PAHO and BASICS), as specified:
Recommendations to USAID/LAC
Funding
Continued funding should be provided to LACI, which has led to many important
achievements in spite of some obstacles. Although USAID/LAC cannot accept funds
from other sources, they can support the fundraising efforts of others (especially those
already underway by PAHO and the ministries of health) to expand and enhance IMCI in
the Americas, particularly through in-country activities.
Resources
Remaining LACI resources should be invested only in those six target countries that have
so far been active participants in the initiative, unless health authorities in Guatemala and
Haiti show greater interest in having the initiative implemented in their respective
countries.
Steering Committee
Although considerable progress has been achieved in terms of improving the
collaboration between BASICS and PAHO, the steering committee could profit from
three distinct but related actions:
ß having each member empowered by her/his institution with a level of
decision-making authority appropriate to the critical managerial role of the
steering committee;
ß reaching an understanding among the partners regarding the different
institutional cultures and the need to address these differences more
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
viii
effectively, such as by conducting teambuilding activities with an outside
facilitator; and
ß redesigning the remaining annual work plans and budgets to include a clear
delineation of roles and responsibilities between both implementing partners.
Indicators
Because of the regional nature of the project, indicators should be adapted to regional
activities. Indicators for the overall Strategic Objective and for Intermediate Results 3
and 4 should be revised and baseline and annual data should be reported.
Recommendations to PAHO and BASICS
Compliance
PAHO and BASICS reports and work plans should be sent to USAID/LAC on a timely
basis. New staff for the initiative should be approved by USAID, according to the terms
of their agreements.
Appropriate Credit
In several instances, the source of financial support for LACI activities was not properly
acknowledged. It is important that appropriate credit be given, according to the terms of
PAHO’s and BASICS’ agreements with USAID.
Recommendations to All Partners (USAID, PAHO, and BASICS)
Communication among the Partners
Flow of communication between USAID/LAC and USAID Missions, as well as between
PAHO and BASICS with USAID Missions, should be increased, which could lead to a
more effective integration of activities at the local level.
IMCI Support Systems
Until now, the emphasis of IMCI at the country level has been on the introduction and
expansion of training activities. However, this should not be done in detriment of
improving other components (supervision, monitoring and evaluation, HIS, and drug
management), which are equally critical. In addition, training lessons were better learned
when follow-up activities were conducted to reinforce the training, showing the
importance of redesigning training plans to make them complete.
Sustainability
EXECUTIVE SUMMARY
ix
Sustaining IMCI should be reinforced by continuing efforts at integration into nursing
and medical schools’ curricula, as well as by obtaining additional sources of financial
support. Also important is the continuation of joint implementation activities with NGOs
(such as Plan International and CARE). In addition, new programs and organizations
should be contacted and made part of this initiative. Ultimately, the sustainability of the
IMCI strategy will be another measure of success of this already worthwhile initiative.
Priority of Allocating Resources for In-country Activities
Given the limited resources of the LACI budget and time, the priority of the following
three activities should be
ß improving IMCI support systems,
ß introducing and expanding the community IMCI component, and
ß undertaking efforts to promote sustainability.
F. SUMMARY
The LAC Regional IMCI Initiative, at its midpoint, is at the right time to make the
necessary changes for improved implementation. So far, the initiative has filled an
important role in terms of supporting a composite of activities difficult to finance through
other means. These activities have acted as a catalyst for a complex set of actions aimed
at expanding the use of IMCI in several countries in the region. It is now critical to take
advantage of the lessons from the initial period, plan and budget strategically, and move
quickly to solidify what in many ways have been remarkable accomplishments.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
x
1
I. INTRODUCTION
A. BACKGROUND OF THE INITIATIVE
According to statistics from the World Health Organization’s (WHO) Information Kit on
the Integrated Management of Childhood Illnesses (IMCI) (the information source for
this section), every year approximately 12 million children in developing countries die
before they reach their fifth birthday, many during the first months of life. In the region
of the Americas, more than 200,000 children under 5 years of age die each year from
illnesses that can be prevented or treated with relatively simple approaches. It is
estimated that 70 percent of those deaths in Latin America and the Caribbean are due to
acute respiratory infections (mostly pneumonia), diarrhea, measles, malaria, and
malnutrition. These diseases and others, such as those caused by intestinal parasites and
vaccine preventable diseases, are the principal reasons for medical consultation and
hospitalization in Latin America and the Caribbean.
Projections based on the global burden of disease indicate that unless greater efforts are
made to control them, these diseases will continue to be major causes of child deaths,
particularly in developing countries. Frequently, children’s deaths are caused by a
combination of these conditions, so that most sick children come for consultation with
signs and symptoms related to one or more of these diseases. In many cases, a single
diagnosis may not be simple or possible. It may also be necessary to combine therapy for
several conditions. An integrated approach to treating the sick child, therefore, is
necessary if child health programs go beyond single diseases and intend to address
children’s overall health.
The IMCI Strategy
Disease-specific control programs have provided valuable lessons in the past 20 years.
One of these important lessons is the need to implement strategies that promote
coordination and greater integration of activities to improve the prevention and
management of childhood illness. It was with these criteria in mind that WHO and the
United Nations Children’s Fund (UNICEF) originally developed the IMCI strategy.
IMCI is an effective, low-cost strategy for improving the quality of child health at the
health facility and community levels. The IMCI strategy encompasses a range of
interventions for the prevention and management of major childhood illnesses, both in
health facilities and in the home. It incorporates many elements of diarrheal disease and
acute respiratory infections (ARI) control programs and child-related aspects of malaria
control, nutrition, expanded programme on immunization (EPI), and essential drugs
programs. According to WHO, implementing the strategy requires and facilitates active
collaboration between these programs, and requires efforts at different levels of the health
system.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
2
Interventions
IMCI consists of a series of public health interventions, preventive measures at home and
in the community, and early diagnosis and treatment. It also includes nutrition
counseling, immunization, and other aspects of child health through the promotion of
three main components:
ß improving case management skills of health professionals through locally
adapted guidelines and training activities,
ß improving the health system for more effective management of childhood
illness and delivery of health care, and
ß improving family and community practices through education and counseling
activities.
Relationship of IMCI with Other Technical Programs
Among the additional activities that IMCI promotes are immunization, nutrition, safe
motherhood (antenatal, postnatal, and delivery care), and drug supply management. IMCI
does not intend to replace these programs but tries to ensure that all activities are well
coordinated and implemented to make IMCI more effective. In this regard, at the country
level, this collaboration with other programs is fundamental for the acceptance and
endorsement of IMCI clinical guidelines, as well as for their promotion and use.
Rationale for the Use of IMCI in the Americas
Although the infant mortality rate (IMR) in the Americas has declined steadily,
particularly in the last few decades, there are still big differences among different
countries, among population groups living in different areas in the same country, or
among people with different ethnic backgrounds. The IMR in some countries of the
region is still 10 times higher than in the most developed countries in the hemisphere.
This difference is largely associated with high mortality from infectious and parasitic
diseases. While in the developed countries, acute respiratory infections, diarrhea, and
malnutrition account for only 6 percent of deaths among children under five, they account
for 40-60 percent of deaths of children in the same age group in the developing
countries.
Considerable resources are invested in the diagnosis and treatment of these and other
child diseases in the developing countries. Some inputs, as is the case of antibiotics, are
often used unnecessarily, since more than 50 percent of children given these drugs do not
need them. In addition, the frequent use of antibiotics is expensive, can lead to
antimicrobial resistance, can provoke drug shortages in the health services, and can
prevent children who need them from receiving these drugs. The application of this
strategy allows a complete assessment of a child’s health status, the detection of other
illnessesæeven when they are not the main reason for consultationæand through better
I. INTRODUCTION
3
diagnosis, savings in the use of such critical drugs as antibiotics. In addition, because of
its integrated approach, which goes beyond children’s health, IMCI can articulate
integrated health management strategies throughout the human life cycle.
Agencies’ Cooperation for the Implementation of IMCI in Latin America
To implement the strategy, the Pan American Health Organization (PAHO)/WHO, and
UNICEF initially joined forces through an interagency agreement, signed in 1996, to
assist national authorities in incorporating the IMCI strategy at the health services and
community levels. The grant to PAHO/WHO from the United States Agency for
International Development (USAID) was signed in May 1997, to assist the countries with
the highest infant mortality levels in the region in the implementation of the IMCI
strategy. Staff from USAID, PAHO, and the Partnership for Child Health Care, Inc. (the
cooperating agency that implements the Basic Support for Institutionalizing the Child
Survival Project [BASICS]), collaborated in the design of the Latin America and the
Caribbean (LAC) Regional IMCI Initiative (LACI). The initiative is a $5 million, five￾year program, sponsored by the United States Agency for International Development
(USAID) and implemented by PAHO and BASICS for the Bureau for Latin America and
the Caribbean (LAC), Office of Regional Sustainable Development (RSD), Center for
Population, Health and Nutrition (PHN). BASICS had been involved in IMCI
implementation prior to this initiative and the adoption of IMCI by PAHO. See annexes
A-D for the background documents of LACI (authorization, grant letter to PAHO,
USAID Results Framework, and the LACI schematic design).
Broad approval for IMCI has helped mobilize several governmental and
nongovernmental resources with a different degree of commitment in each country.
Commitment at the national level, essential for the implementation of the strategy, began
with the meeting at Santa Cruz de la Sierra (Bolivia), convened in 1996 by
PAHO/Control of Diarrheal Diseases (CDD)/ARI. This meeting began by outlining the
IMCI strategy and concluded with the Declaration of Santa Cruz de la Sierra, where 18
countries expressed their firm interest in making every effort to ensure that all children
under five in the hemisphere have access to this strategy. The next step was the official
adoption of the IMCI strategy as a basic health policy approach for securing a reduction
in infant mortality.
In April 2000, USAID/LAC initiated a midterm evaluation of the LACI.
B. PURPOSES OF THE EVALUATION
As detailed in the scope of work (annex E), the purposes of this midterm evaluation are:
1. To determine if the USAID Results Framework is still valid and feasible and
to recommend adjustments to the Results Framework, design, and activities,
as appropriate.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
4
2. To examine progress toward achieving the Strategic Objectives (SO) and the
Intermediate Results (IRs) as planned, evaluating whether the IRs are likely to
be met in a timely and effective manner. To identify specific internal/external
constraints, which may limit their accomplishments or success, and to
recommend adjustments based on findings and conclusions.
3. To assess how the initiative structure and design as configured are working,
including both the specific technical areas and activities and the delegation of
responsibilities to the two partners, as well as the working relationships and
collaboration in the implementation of initiative activities. Adjustments to
implementation based on findings and conclusions should be recommended.
4. To assess the administration of the initiative by USAID and the two
implementing partners, including the status of coordination and
communication within the organizations, specifically between headquarters
and field operations and host country governments with USAID/Washington
and field Missions.
C. FIELD PROCEDURES
During the first week of the evaluation, a 2-day planning meeting was held in
Washington, D.C., and interviews were conducted with USAID (the Bureau for Global
Programs, Field Support and Research and LAC), PAHO, and BASICS officials. Three
weeks were spent in the field, one each in Honduras, Bolivia, and Peru (see annex F for
the evaluation schedule and annex G for documents reviewed). Follow-up interviews
(WHO, UNICEF, and the World Bank) were conducted after the first week in Honduras
and after the field work was finished, in person or by telephone. Discussions were held
with all key stakeholders.
During the field visits, the following individuals were interviewed in each country: the
PAHO representative or acting representative and maternal and child health or IMCI
responsible official at the PAHO country office; Ministry of Health (MOH) central-level
and local-level staff in the localities visited; USAID Mission staff; a BASICS
representative and officials; and officials from other organizations connected to this
initiative (local nongovernmental organizations [NGOs]). In addition, several meetings
were held during the field visits with the persons in charge of applying the strategy and
interviews were held with facilitators and health workers trained in this strategy (annex H
lists all persons interviewed).
The methodology applied during the evaluation was the application of both free and
structured interviews. The main questions included:
ß managerial structure of the initiative,
ß relationships among the different initiative partners,
ß technical questions,
ß key achievements,
I. INTRODUCTION
5
ß main obstacles,
ß managerial strengths and weaknesses,
ß facilitating factors, and
ß lessons learned.
D. EVALUATION TEAM COMPOSITION
The evaluation team was composed of two consultants. The team leader is a physician
with extensive experience in the evaluation of health projects in developing countries,
with good knowledge of PAHO/WHO and other United Nations agencies, and he is
fluent in both English and Spanish. The other team member is an expert in nutrition with
extensive experience both in program implementation and evaluation of health projects in
developing countries, is fluent both in English and Spanish, and has previous experience
with private voluntary organizations (PVOs), USAID, and several international
organizations.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
6
7
II. PROJECT DESIGN ISSUES
This section addresses aspects of conceptual design only; those aspects related to the
design as it was revealed in management and implementation are discussed in sections III
and IV, respectively.
A. ROLE OF THE IMCI REGIONAL INITIATIVE
By definition, a regional initiative is designed to conduct regional activities. The LAC
Regional IMCI Initiative was designed to help make IMCI well known in Latin America.
Two partners (PAHO and BASICS) were chosen to implement the initiative, with special
focus on eight target countries. The budget, $5 million over five years, was not designed
to sustain long-term activities of any given partner in any given country. Rather, the
budget was developed for selected activities that would act as a catalyst for IMCI in the
region.
From a management point of view, a regional initiative is different from a typical project
with its own staff and budget. While the LACI has a central manager, the cognizant
technical officer (CTO) in USAID/LAC, all other professionals working on this project
report to supervisors in other management structures in other organizations. Thus, the
initiative is built on the assumption that the key stakeholders are able to work effectively
as team members even though their compliance is not required through direct supervisory
lines. Discussion of the findings regarding the management of the LACI is found in
section III.
In addition, given no direct management authority over the partners and a budget which
was designed to stimulate, not sustain, in-country activities, the LACI took on
considerable risks that it could effect significant changes in the region, and especially the
eight target countries in the time frame allowed. In short, the initiative took on the role of
a catalyst for introducing IMCI in Latin America. The key features of a catalyst in this
program setting are:
It is a small amount of funds, effort, and input; it sets off a reaction which would
not otherwise be happening; the substance loses its identity in the reaction; the
end product is considerably different from any of the original separate ingredients
(contributions by USAID, PAHO, BASICS, and other collaborators); and the
catalyst cannot be recovered once the reaction has taken place.
The way in which the LAC Regional IMCI Initiative has functioned as a catalyst, joining
efforts of other agencies and governments, and the results produced are described in
subsequent chapters of this report.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
8
B. OVERALL PROJECT DESIGN COMPONENTS
The LACI is implemented through two partners: PAHO, through a grant, and BASICS,
through field support to the contract that BASICS has with USAID/G/PHN. Compared
with other USAID/LAC regional initiatives, the IMCI effort has fewer partners and one
of them, PAHO, is also a partner in other initiatives. Nevertheless, the findings of this
evaluation suggest that the absolute number of partners may not be as important to
collaborative harmony as the willingness to collaborate and the teambuilding skills (e.g.,
effective two-way communication, compromising, giving and receiving constructive
criticism, planning, mutual accountability) of the specific partners.
The original design contemplated change throughout the region and targets eight
countries: Guatemala, Haiti, El Salvador, Nicaragua, Honduras, Ecuador, Bolivia, and
Peru. However, since the partners are also involved in other IMCI commitments (PAHO
in other countries throughout the Caribbean, Central and South America, and BASICS in
other countries in Africa and Asia), trying to focus the LACI on these eight specific
countries creates difficulties in setting priorities for one’s efforts. One way these
difficulties are expressed is in the steering committee, as described in section III.
C. APPROPRIATENESS OF THE USAID RESULTS FRAMEWORK
The original schematic design of the LACI (see annex D) shows three phases of the
initiative: design, implementation, and monitoring and evaluation (M&E). Although the
presentation of these phases as separate components may be useful as an organizational
aid, the separation of M&E as a distinct phase can lead to difficulty if viewed as
chronologically distinctæby weeks, months, or even yearsæfrom implementation. Plans
for M&E, including indicators, frequency of collection and analysis, the baseline survey,
and resources for M&E, should all be determined before the first day of implementation
and then incorporated as an ongoing implementation activity. Further discussion of the
role of M&E as a support function in IR 3—building country capacity to implement
IMCI—is discussed in section IV.
Strategic Objective (SO) and SO Indicators
The Strategic Objective for the LACI is, “More effective delivery of child health services
in response to CDD, ARI, and malnutrition.”
The SO indicators are the number of target countries with
1. 90 percent of facilities in pilot districts delivering IMCI services, and
2. 10 percent of facilities delivering IMCI services.
The original SO indicator 1 was: “Number of target countries with 100 percent of
facilities in pilot districts delivering IMCI services.”
II. PROJECT DESIGN ISSUES
9
These two indicators represent appropriate targets for IMCI in general, for example, for
all donors and the MOH to work on in any given country. However, for just one donor,
they are inappropriate because they reflect the efforts of the entire country (or region or
district) and therefore there is little possibility of connecting one donor’s inputs to the
outcomes (referred to as “plausible attribution” by USAID). In addition, it is too difficult
for PAHO and BASICS to report on these indicators because they are out of the control
of the supervision scope of the two partners. Finally, even if the participating countries
had the means to and agreed to help provide the data necessary to calculate these
indicators, it would be too expensive an undertakingærelative to the total budget of the
LACIæto gather the data in a comprehensive, timely, and accurate manner.
These indicators, or other related ones developed to suit the unique conditions of each
country, are more appropriate for each country to collect, using resources available from
all partners contributing to IMCI. In addition, these indicators would need to be used in a
timed fashion, such that each country would set up appropriate annual goals in their quest
for fulfilling the indicators.
The LACI partners are in agreement over their concern about the indicators and their
desire for changes to be made. Even though discussions on this issue have already taken
place with USAID/LAC, the parties involved are still in disagreement about the viability
of these indicators. Since useful data are not being collected to calculate them, all parties
need to return to discussing the issue. One potentially rich source of ideas for revised
indicators comes from earlier discussions among the partners where indicators were
developed but then not used.
Other USAID/LAC regional initiatives can provide examples of potential indicators that
could be adapted for the LACI. For example, the following two indicators are taken from
the latest USAID/LAC Results Review and Resource Request (R4) report (March 22,
2000) and seem useful as points of departure for IMCI regional indicators:
1. “For the third year running, 100 percent of the methodologies and tools
introduced by the health sector reform initiative were used by 50 percent of
the countries where introduced.” (Key Results, p. 25) This indicator could be
adapted for IMCI.
2. “Target countries with reform processes that substantially integrate lessons
learned from this initiative (methods and tools, information, monitoring and
exchanges).” (Health Sector Reform section)
The LAC Regional IMCI Initiative can play an important role in advising the
participating countries about potentially useful indicators they can adopt for monitoring
the implementation and results of IMCI. However, the partners in the regional initiative
itself must be in agreement about their own indicators first. (See the recommendations
for additional details regarding the suggested process for developing new indicators.)
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
10
Intermediate Results (IRs)
The four IRs and their respective indicators are listed below.
IR 1: Country leaders have information for IMCI adoption.
ß Indicator 1: Number of target countries where country Ministry of Health
leaders have adequate information regarding IMCI
ß Indicator 2: Number of target countries with official decision to adopt IMCI
IR 2: Country plans and strategies in place for introduction and implementation of IMCI
ß Indicator: Number of target countries with national operational plans in place
The original indicator for IR 2 was: “Target countries with IMCI plans and strategies
adopted including identification of resource requirements.”
IRs 1 and 2 and their indicators are adequate as written and no revisions are
recommended.
IR 3: Improved country capacity to implement IMCI
ß Indicator 1: Number of target countries with more than 10 percent of
ambulatory health facilities in designated areas which have IMCI norms,
health service providers trained in IMCI, and IMCI essential drugs available.
The original Indicator 1 included “… IMCI essential drugs available at least 75 percent
of the time; and district plans that include IMCI at this level.”
ß Indicator 2: Number of target countries with 80 percent of designated districts
with plans for IMCI implementation developed.
The original Indicator 2 included “… countries with 100 percent of pilot districts when
have: IMCI norms available; service providers trained in IMCI; IMCI essential drugs
available at least 75 percent of the time; and district plans that include IMCI at this
level.”
Comments and recommendations regarding the viability of indicators 1 and 2 for IR 3 are
exactly the same as those made above for the SO indicators, given the similarity of these
two sets of indicators.
IR 4: Monitoring and evaluation used to adjust IMCI program plans
ß Indicator: Number of target countries that have incorporated monitoring and
evaluation findings into annual IMCI district plans.
II. PROJECT DESIGN ISSUES
11
The original indicator for IR 4 was “…countries with IMCI annual plans that reflect
findings from monitoring and evaluation.”
Dedicating an IR specifically to monitoring and evaluation functions is commendable.
However, as currently written, this indicator can be too easily checked off when
“completed” even just once, in one district, in one year. In reality, this indicator needs to
better capture the evolving (and, it is hoped, ever improving) process of monitoring and
evaluation as it is introduced, used, and integrated into the fabric of a district’s annual
operating health plan.
In general, the four IRs above read more like steps in implementation as opposed to
reflections of overall results and accomplishments necessary to lead to IMCI being used
in the region. The indicators for IRs 1, 2, and 4 do not necessarily reflect the quality,
extent (coverage), or results of the implementation of activities supported by LACI. In
contrast, the indicators for IR 3 go well beyond the scope of LACI.
The proposal (January 7, 1997, p. 3) includes in the narrative a number of expected
conditions that seem to reflect what the partners expect as a result of implementing the
LACI. (The words in italics are examples of the “promises” of IMCI and are potentially
useful characteristics of positive IMCI results that could be the points of departure for
developing revised indicators.)
ß “…integrated management can mean greater efficiency in training,
supervision, developing policies and norms, and monitoring and evaluation
from the community level to the health care facility…”
ß “…over time, resources required may be reduced…”
ß “…IMCI approach can avoid the duplication of efforts…in training and
implementing parallel vertical control programs in acute respiratory infections
(ARI), control of diarrheal diseases (CDD), nutrition and vector borne
disease…”
ß “…positive complementarity of proposed activities with country-level child
health care priorities…”
ß “…strengthening regional and subregional technical cooperation in child
survival among the participating countries...”
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
12
13
III. MANAGEMENT AND ADMINISTRATION FINDINGS
A. MANAGEMENT STRUCTURE
The LAC Regional IMCI Initiative is implemented through two partners: PAHO and
BASICS. The original idea was that each partner would bring its own unique and yet
complementary assets to the initiative. PAHO would bring its role as the natural
counterpart for the ministries of health in the region, its technical expertise, its previous
experience with different components that comprise IMCI (e.g., acute respiratory
infections, diarrheal diseases), its presence in every country, and its long-term experience
working with different partners on regional health initiatives. BASICS would bring its
technical expertise, its experience in IMCI in other regions, particularly Africa, and its
prior experience as a USAID contractor, which facilitates communications with USAID
Missions. However, BASICS’ contribution is conditioned by USAID Missions’ interest
that Global Bureau projects work in close partnership with their in-country bilateral
activities and their willingness to support national IMCI implementation. Also, since
BASICS is a contractor of the Global Bureau, the LAC IMCI Regional Initiative requires
considerable collaboration between USAID/LAC and the Global Bureau.
There is presently a steering committee made up of representatives of USAID/LAC, the
Office of Health and Nutrition (USAID/G/PHN/HN), PAHO, and BASICS. Currently,
this group meets regularly for information exchange, joint work planning, and progress
reports. Although the local USAID Missions initially participated in the initiative’s
design, they are not active partners in its day-to-day management.
Strengths of the Management Structure
Decentralization of decision-making (through PAHO and BASICS and then to their
respective country offices) allows for innovation and responsiveness to local conditions.
Flexibility offered by USAID/LAC in the use of regional funds within the parameters of
the proposal allows funding of regional activities not funded normally through other
agencies or governments.
Weaknesses of the Management Structure
Insufficient Decision-making Power
As presently organized, the steering committee does not have sufficient decision-making
power. For example, the PAHO representative at the steering committee is not authorized
to make management decisions, thus diminishing the effectiveness of the steering
committee.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
14
Legal Agreements with USAID/LAC’s Two Partners
The legal agreement with BASICS is a contract and the agreement with PAHO is a grant.
The degree of USAID’s substantial involvement is greater in the former than in the latter,
thus setting the stage for disagreements and differing expectations. Some expected
deliverables planned in the original proposal have not been met according to schedule;
thus, new terms have to be negotiated with specific consequences for noncompliance.
Definition of Roles and Responsibilities between PAHO and BASICS
From the very beginning, roles and responsibilities were not clearly delineated, which led
to frequent misunderstandings. The proposal states, “In terms of the activities envisioned
under this proposal, PAHO and BASICS will share implementation responsibility,
according to each partner’s technical and organizational strengths” (p. 7). Even though
the proposal was prepared jointly by PAHO and BASICS, it naturally did not include the
specificity in practical terms regarding day-to-day roles and responsibilities.
Unfortunately, subsequent documents, such as the annual work plans, lacked clear
definition on this point.
Concept of Lead Partner
The original proposal stated, “Because of its permanent intergovernmental nature in the
region, PAHO will assume the overall coordinating role for this LACI.” (p. 7)
Subsequently, a decision was made whereby PAHO and BASICS took the lead partner
role in four different countries each. For various reasons, this concept did not work out in
practice; however, the need remains for one partner in each country to be the country￾level coordinator for the LAC Regional IMCI Initiative. Since neither partner is now
designated as the lead partner, coordination of their joint activities is left to chance.
While in Honduras this situation has hampered progress, in Bolivia it has not disrupted
the long-term effective working relationship between PAHO and BASICS. In Peru, there
is no BASICS presence. The designation of a lead partner in each country should be
based on criteria developed by the steering committee and need not result in an even
division between the partners.
Transition of BASICS I to BASICS II
BASICS I came to an end near the end of the second year of the regional initiative. This
prevented long-term planning from BASICS for the five-year initiative. In addition, the
transition from BASICS I to BASICS II was unusually protracted, thus creating a gap in
the field while usurping management attention among the stakeholders in Washington.
Multiplicity and Parallelism of Management Structures
Almost all persons involved in the LAC IMCI Regional Initiative report to different
supervisors within different institutional structures. This inhibits the implementation of
decisions. Furthermore, for these same persons, the LAC Regional Initiative is usually
III. MANAGEMENT AND ADMINISTRATION FINDINGS
15
less than 50 percent of their total scope of responsibilities, thus also limiting their
engagement in the process. Although these LACI management features are an inherent
part of the initiative and probably cannot be modified, they need to be considered as plans
and expectations for the remainder of the initiative are developed.
Lack of Communication
The USAID/LAC is being perceived as “overworked and understaffed” and therefore
unable to communicate about IMCI as well as it might otherwise. This situation is
manifested in little direct communication between USAID/LAC and the participating
Missions.
B. IN–COUNTRY COORDINATION OF ACTIVITIES
Honduras
The coordination of activities at the country level had different characteristics in the
countries visited. In Honduras, there is a need for strong leadership, particularly from the
Secretary of Health. In addition, the PAHO representative has recently arrived in
Honduras and is only now in a situation of closely following project activities. Presently,
there is not a functioning interagency group in Honduras. However, representatives of the
Secretary of Health, bilateral agencies, and NGOs have formed a group called
Comunicación en Salud Infantil (COMSAIN), one of whose main functions is the
elaboration and unification of educational materials related to children’s health. With
strong leadership from the Secretary of Health, such a group could become an effective
coordinator of IMCI activities in the country. Although there is a good personal
relationship between the persons at BASICS and the PAHO staff responsible for IMCI,
there is no joint planning or monitoring of activities related to IMCI between them. Joint
planning, if done effectively, can increase the probability for more efficient use of
resources, creative problem solving, and ultimately, synergy in programming.
CARE coordinates its activities with the Secretary of Health, which trains CARE
personnel. UNICEF has financed several training activities, and has participated in the
adaptation of educational materials, particularly those related to the area of
communication in child health. Under consideration is an American Red Cross grant to
PAHO for US $6.7 million for IMCI training activities in several countries, including
Honduras.
Bolivia
In Bolivia, there is a very effective interagency committee devoted to children’s health
care issues, which existed before the LACI. The activities of this committee comprise
review and support of the MOH’s annual operating plan; exchange of information;
sharing of experiences in monitoring and evaluation; provision of technical assistance
and financial contributions; and review of documents. The clear and effective leadership
of the MOH is a critical factor. NGOs can relate to this committee either directly or
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
16
through their membership in Programa de Coordinación en Salud Integral (PROCOSI),
an NGO network. There is strong leadership from the Ministry of Health and an excellent
working relationship between the PAHO and BASICS staff in charge of the
implementation of IMCI at the country level, as well as between them and the USAID
country office. All of these conditions help foster a very fertile environment in-country
for making optimum use of the inputs provided through various LACI activities and
channels.
In what could be a useful contribution to LACI efforts, the USAID country office, with
the collaboration of the Global Bureau, has developed an IMCI interactive training course
in Spanish, which was first tried in Uganda in its English version. That course was a
useful tool for IMCI and has been approved both by WHO and Bolivia’s Ministry of
Health. Through the initiative, this course could be introduced in the other participating
countries.
Because of administrative problems, UNICEF Bolivia is not closely involved in the
implementation of IMCI in the country. UNICEF has financed some training activities
and the production of educational materials and has played a critical role in the
incorporation of the IMCI strategy in the Seguro Básico de Salud, a health insurance
system for children and pregnant women. This has helped institutionalize IMCI and has
greatly contributed to the rapid expansion of the strategy at the country level. In addition,
UNICEF has financed the inclusion of IMCI in the curriculum at the medical school of
University of San Andrés in La Paz.
Peru
In Peru, IMCI activities are coordinated by the Ministry of Health with the cooperation of
PAHO. There is an effective interagency committee convened by PAHO’s representative
in the country. At the Ministry of Health, there is an IMCI national coordinating
committee (Comité Coordinador Nacional Atención Integrada a las Enfermedades
Prevalentes de la Infancia [AIEPI, or IMCI]-Ministerio de Salud [MINSA, or MOH])
that was established in 1997 and which coordinates IMCI activities in the country.
The local USAID office is directly supporting IMCI, mainly through some training
activities funded through the bilateral Proyecto 2000. Funds from this project are used for
maternal and child health training activities in 12 subregions in the country, for training
community health workers, and for a project on the rational use of medicines (uso
racional de medicamentos). LACI funds for IMCI advocacy, materials adaptation, and
limited implementation activities are channeled through USAID’s grant to PAHO, as
there is no BASICS presence in Peru. There is no joint planning of activities or other
collaboration between USAID/LAC and the local USAID office for the implementation
of IMCI in the country. There was limited support of USAID/Peru to BASICS in 1998 to
organize IMCI training courses. The World Bank is providing financial support for
strengthening the health infrastructure in several provinces, and Plan International is
conducting an intensive set of training activities in community IMCI as part of pilot
projects in Piura and Sullana.
III. MANAGEMENT AND ADMINISTRATION FINDINGS
17
C. STAFFING
PAHO
At PAHO headquarters, the AIEPI (IMCI) technical and management unit comprises six
staff members, all of whom have extensive experience in international public health and
interagency technical cooperation in Latin America. There is a regional advisor for IMCI
who is responsible for the coordination and supervision of technical and administrative
issues relevant to IMCI within PAHO, including management and coordination of the
LAC Regional IMCI Initiative. The regional technical officer is responsible for
supporting IMCI planning, monitoring, and evaluation; he regularly attends the steering
committee meetings in Washington, D.C. Both of these positions are funded 100 percent
by PAHO as a counterpart contribution to the USAID grant. There also is a regional
IMCI specialist in charge of administrative issues, special presentations, documentation
review, proposal writing, and training needed in the LAC Region. Funded from PAHO’s
own budget as a counterpart to the USAID grant, there are six PAHO national IMCI
consultants responsible for specific IMCI activities (to which they dedicate
approximately 50 percent of their time) in Honduras, Bolivia, Peru, Ecuador, Haiti, and
El Salvador. In the other target countries, PAHO staff carries out IMCI duties. In
addition, PAHO has been providing additional technical support to the countries by
sending consultants on an ad hoc basis. Also paid by the Initiative are three short-term
consultants: a subregional consultant and two consultants who serve as administrative
staff.
BASICS
At the headquarters level, BASICS’ senior management group delegates program
management responsibility to regional and country clusters composed of operations and
technical division staff, as well as to technical working groups, which coordinate their
actions with USAID’s Global Bureau. During BASICS I, there was a LAC operations
officer in charge of overall planning, management, and coordination with PAHO and
USAID’s Global and LAC Bureaus, and a senior technical officer in charge of managing
all technical aspects of IMCI implementation, in close coordination with PAHO,
particularly at the beginning of the project. Both of these positions were part time to this
initiative. The operations officer is not connected to this initiative any longer, and the
senior technical officer is now in charge of IMCI implementation at the global level. For
BASICS II, there is a full-time technical officer who has been closely following the
implementation of regional and subregional clinical training courses and other technical
workshops in coordination with his PAHO counterpart. They are both members of the
initiative steering committee. An operations coordinator is in charge of tracking work
plans with USAID’s Global Bureau and preparing technical directives and clearance
requests.
Honduras
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
18
In Honduras, at the Secretary of Health, IMCI activities are coordinated by the chief of
the Department of Maternal and Child Health and by the officer in charge of AIEPI, who
is directly responsible for IMCI activities. The PAHO officer presently in charge of the
initiative has only recently been given this responsibility and cannot devote all his time to
it since he has other responsibilities. The BASICS representative is in charge of following
the implementation of the initiative in the country.
Bolivia
In Bolivia, the chief of the National Program for People’s Care (Programa Nacional de
Atención a las Personas) of the Ministry of Health is an effective coordinator of IMCI
activities in the country. Because of his previous role as president of the Bolivian
Pediatrics Society, he has been able to quickly move the strategy forward. He coordinates
an active interagency committee that includes bilateral agencies and NGOs working in
the country. A consultant in child and perinatal health from PAHO is in charge of IMCI
activities, which are not her sole responsibility. On the BASICS side, both the BASICS
representative in the country and a technical officer are in charge of the implementation
of the initiative.
Peru
In Peru, the MOH is in charge of implementing the IMCI strategy in the country. At the
Ministry of Health, there is a Comisión Nacional Coordinadora (National Coordinating
Commission), with representatives from the MOH, PAHO, UNICEF, USAID, and the
MOH program directors. This group is responsible for the preparation of operational
plans and monitoring activities. In Peru, the World Bank is supporting a health sector
reform project that will lead to the integration of several programs. At the ministry level,
however, there are still 21 different vertical programs. The infecciones respiratorias
agudas (acute respiratory infections [ARI]) program coordinates the institutional IMCI
and the enfermedades diarreicas agudas (control of diarrheal diseases [CDD]) program
coordinates the community IMCI activities. Both types of activities are under the
supervision of the executive director of Women and Children’s Health Programs and the
Programa Ampliado de Inmunizaciones (Expanded programme on immunization [EPI]).
Because there is no BASICS counterpart in the country, PAHO is the main technical
support for the Ministry of Health, a function that is effectively carried out by the officer
in charge of following AIEPI for PAHO.
D. INFORMATION SHARING AMONG IMCI COUNTRIES
Information sharing about IMCI activities in the countries involved normally occurs
through two mechanisms. One mechanism is yearly meetings for PAHO personnel with
the participation of technical officers involved in the initiative, where progress and
problems that hinder the implementation of IMCI in the region are discussed. During
these meetings, which have a strong element of self-criticism, PAHO’s regional activities
and plans are discussed and organized to make them coincide with the plans of the MOH,
so as to make these plans both possible and sustainable. In addition, two yearly meetings
III. MANAGEMENT AND ADMINISTRATION FINDINGS
19
(Evaluation and Planning Meetings for the USAID/LAC–PAHO-BASICS Initiative)
took place, one in Quito, in 1998, and one in El Salvador, in 1999, with the next one
planned to take place this year in Honduras. During these meetings, progress in IMCI
implementation at the country level is discussed and several workshops are conducted,
covering such topics as how to expand activities without losing quality, how to improve
monitoring and evaluation, coordination, and mobilization of resources to support IMCI
strategy implementation, IMCI, and nutrition. These meetings are attended by officers
from financing and implementing organizations, as well as from the Ministries of Health
and NGOs focused on by the initiative.
E. CONCLUSION
As presently organized, the initiative does not have a strong central managerial structure,
a role that could be fulfilled by the steering committee. One of the functions of this
committee should be the formulation of detailed implementation plans, which would
clearly define the roles of both PAHO and BASICS. One of these responsibilities, already
stated in the original proposal, was for PAHO to have overall coordination responsibility,
a role which changed with the decision to give both partners a leading role in each of four
countries. When that plan did not work out, no replacement plan was set in place, with
the result that the coordination of activities between both partners in each country is
practically left to chance. A stronger steering committee could fill the vacuum in the
presently weak managerial structure of this initiative and become a key factor for
improving implementation.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
20
21
IV. IMPLEMENTATION FINDINGS
Through its contractual agreements with PAHO and BASICS, USAID/LAC is funding a
series of activities in the form of the LAC Regional IMCI Initiative. The actual channels
for funding these activities are complex as they involve two implementing partners who
execute LACI activities in Washington, D.C., and in specific countries which serve as the
host for a regional or subregional event or which are the recipients of funding for a
specific in-country activity (e.g., a national IMCI evaluation). In the three countries
visited, the evaluation team found that it was often not possible for the PAHO, BASICS,
or MOH interviewees to easily identify the exact source(s) (or breakdown among
sources) of funding for any given activity without detailed research into the financial
records.
For example, PAHO has a variety of funding sources to support its IMCI activities in the
region, of which LACI is one. PAHO maintains LACI funds to be expended in
Washington, D.C., on behalf of the eight LACI target countries and also remits funds to
these countries for LACI-sponsored activities. Each country also has PAHO funds from
other sources (via Washington, D.C.) to support the country program which includes
IMCI. Thus, any given activity that focuses on IMCI could have a variety of logical
funding sources. The situation with USAID is similar in that BASICS in-country has
several potential funding sources, inter alia, bilateral funds from the local USAID country
Mission, country-specific funds managed by BASICS in Washington, and core funds
from BASICS. Finally, given that IMCI is supported by donors other than USAID/LAC
and PAHO, any given in-country activity that is contemplated in the LACI annual work
plan and budget could eventually be cofunded by other donors.
To the extent possible, the evaluation team tried to identify the source(s) of funds for the
activities discussed in this report. (In this regard, any error of attribution is the
responsibility of the evaluation team.) The fact that it was difficult to trace the path of
LACI funds directly as various IMCI activities were explored in the three countries
visited does not suggest that these funds have not and are not being appropriately
programmed and recorded.
A. STATUS OF ACTIVITIES TO DATE
Strategic Objective: More Effective Delivery of Child Health Services in Response
to CDD, ARI, and Malnutrition
Originally, PAHO developed an indicator-based worksheet which was sent to the
countries to try and collect information on the SO and IR indicators. This activity
included working collaboratively with WHO and BASICS to define the indicators and
develop the format for PAHO to analyze the results. However, the indicators for the SO
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
22
and IRs 3 and 4 in use at the time of the evaluation were not functioning for the overall
monitoring of LACI.
Although neither PAHO nor BASICS is presently collecting information on the two SO
indicators, based on data gathered and interviews with health officials both at the central
(MOH) and peripheral levels (hospitals, clinics, and health centers), significant progress
has been achieved towards this goal.
Intermediate Results
IR 1: Country Leaders have Information for IMCI Adoption
By 1999, all eight countries included in the LACI had information about this initiative.
IR 2: Country Plans and Strategies in Place for Introduction and Implementation of IMCI
Services
Progress in Haiti has been exceedingly slow. As of the time of the evaluation, only
Guatemala was still reluctant to implement IMCI. A change in government is most likely
necessary before the Guatemalan MOH would be willing to consider IMCI.
IR 3: Improved Country Capacity to Implement IMCI
The two indicators for this IR did not seem to be reported on (as of 1999) because they
are country-level indicators requiring detailed information not available yet in the
participating countries visited.
Training to Date
According to PAHO (1999 Report to USAID/LAC), the following results have been
obtained regarding training activities that have used manuals and other materials
developed and/or adapted through LACI-funded activities:
ß Number of facilitators: A critical mass has been established for replicating
IMCI at the district and local levels.
ß Number of courses: Over 200 national and operational IMCI clinical courses
have been conducted; in addition, PAHO has developed three community￾level training courses.
ß Number of people trained: Over 8,000 health workers have been trained in
the IMCI clinical course in the 8 Latin American countries involved in the
initiative.
IV. IMPLEMENTATION FINDINGS
23
Training Effectiveness
Although the LACI provided funds directly for developing the national training teams,
information was obtained regarding the subsequent ripple effect that these national teams
have had on training effectiveness in the three countries visited. Effective training
depends on several factors, including quality of the trainers; adequacy of the training
materials to the trainees’ levels of understanding; appropriate length of the training event;
appropriate selection of trainees with similar levels of training needs (i.e., not mixing
auxiliary nurses, licensed nurses, and doctors); appropriate number of trainees; adequate
ratio of facilitators to trainees; and timely and high-quality follow up. Among these
factors, one of the most consistent weaknesses reported was the lack of timely follow up
to the IMCI institutional training. In addition, in those facilities where the nurses are
trained and the doctors who supervise them are not, IMCI cannot be effectively
implemented.
Presently, the number of facilitators available for training activities cannot meet the
demand as training spreads geographically and at different levels of health personnel (i.e.,
institutionalælicensed nurses and doctors, auxiliary nurses, and community health
workers). Although a systematic evaluation was not conducted of the IMCI training for
health professionals (licensed nurses and doctors), auxiliary nurses, and community
promoters, based on the interviews conducted with different levels of health workers, it
appears that the quality of training has been adequate. However, in the case of Bolivia,
the MOH decision to shorten the IMCI institutional course from 11 days to 5 has serious
implications; the decision has been met with almost universal criticism, principally
because of the negative effect on the quality of the training (and hence the wise use of
resources for that training).
In Bolivia, due to the low literacy level of the auxiliary nurses, a complementary course
(with funding from BASICS) was developed by the Centro de Capacitación para la
Lactancia Materna y Nutrición (Training Center for Breastfeeding and Nutrition
[COTALMA]). The course has less technical content and more graphic materials than the
standard IMCI course to facilitate comprehension. Technical assistance from World
Education helped the facilitators develop nonformal adult education methods for use with
the materials adapted for this group of trainees.
Overemphasis on Clinical Training and Underemphasis on Health Systems
Support Components
Although IR 2 contemplates that plans and strategies will be in place for IMCI
implementation, it appears that the clinical training component has received the vast
majority of attention. In the three countries visited, IMCI training activities are expanding
too fast, to the detriment of other system components. Reasons for this imbalance include
ß greater donor interest in clinical training to introduce a new technique (IMCI)
than in the less attractive and more complex systems needed to support the
trainees once they return to their jobs,
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
24
ß MOH vulnerability to accepting donor funds for training without challenging
the conditions,
ß country IMCI leaders did not develop a detailed series of preconditions for
initiating clinical training, and
ß emphasis by WHO to implement clinical training, as the manuals for this
component were the first tools developed.
This major emphasis on training (initially clinical and now community IMCI) has
prevented significant attention from being given to the supporting components. IMCI’s
second component comprises quality of services, supervision, and integration of IMCI
indicators into HIS; monitoring and evaluation; management and cost analysis of IMCI
drugs; health facility assessment; and overall cost analysis of adopting IMCI. If this
trend continues, the potential for systematic monitoring and evaluation will be seriously
impaired and thus the ability to realize and document IMCI’s impact on reducing costs,
morbidity, and mortality.
In most regions visited, the specific health system component that interviewees were
concerned about was supervision. Results from a study in Kenya showed that new
knowledge about IMCI, when not reinforced, has a tendency to be lost within one year.
This finding underlines the need for regular supervision, an activity which is not being
carried out in the three countries visited during this evaluation (and which was never
expected to be funded by LACI). Although lack of financial resources was often cited as
a major impediment to undertaking supervision, in the long run, it is more cost-effective
to finance supervision than to risk investing major resources in training without
addressing this critical component.
The LACI partners—PAHO and especially BASICS—have helped develop a long list of
useful tools for IMCI, including many of the training materials and tools for addressing
the systems support functions, such as supervision. In sum, these latter tools are
underused in the countries visited and may be throughout the region if the Rational
Pharmaceutical Management (RPM) experience is indicative of a wider phenomenon (see
discussion in the section on development and use of tools).
IR 4: Monitoring and Evaluation is Used To Adjust IMCI Program Plans
In Honduras, a baseline study was conducted which addressed the availability of human
resources, drugs, and quality of care. The next step is to refine the instrument and apply
it in regions II and III in that country. In May 2000, BASICS plans to develop an
instrument for evaluating IMCI in the countries implementing the initiative. To the
extent that supervision is taking place, there may well be adjustments made on a case-by￾case and visit-by-visit basis. At the community level, BASICS is supporting a multiyear
study to assess the ultimate changes in nutritional status of children in the areas where
atención integral al niño (integrated attention to the child [AIN]) has been introduced.
IV. IMPLEMENTATION FINDINGS
25
In Bolivia, there was a formal baseline study of the three early intervention areas
(May-July 1998) and a subsequent evaluation in April 1999. This was the first IMCI
evaluation of its type to take place in the world. According to the coordinator of the
training center at the Children’s Hospital in Santa Cruz, the evaluation findings were used
to
ß modify those training aspects that were found to be weak, such as the proper
diagnosis and treatment of pneumonia and anemia, counseling to mothers, and
immunization practices; and
ß improve the inputs for IMCI through Seguro Básico de Salud, such as the
provision of drugs to health facilities.
In Peru, an IMCI evaluation funded by LACI was carried out in 1999. Although
substantial data have been collected, the results are still being analyzed.
Based on the three countries visited, the following items still need improvement:
ß Many districts still have irregular supplies of the key diagnostic IMCI form
used when interviewing the mother and examining the child, and
ß There is a lack of specific training follow up (within several weeks and then
after approximately 6 months) and no systematic supervision of health
personnel trained in IMCI to reinforce new learning, resolve problems, and
share experiences.
Both of these aspects could be addressed at regional meetings to make national officials
aware of their importance for the best implementation of the initiative.
B. DEVELOPMENT AND USE OF TOOLS
Development of Tools
In Washington, BASICS has developed a set of tools for IMCI; PAHO has translated and
developed many WHO materials for IMCI, for example, a set of materials geared to the
training of the agentes comunitarios de salud (community health workers) that has served
as a basis for country adaptation.
An RPM tool was developed by Management Sciences for Health and financed by
BASICS, with the collaboration of PAHO, which collects cost data, assesses the role of
the private sector, determines availability of drugs at health centers, and looks at
prescribing practices. This tool is now being used in Africa and has been field tested in
Bolivia and Ecuador. The tool has raised awareness about the drug situation, has created
the conditions for fluid dialogue between IMCI staff and the central drug officials in the
countries, has provided information on costs so as to quantify savings through IMCI
implementation, and has created baseline information to measure improvement in drug
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
26
use. It needs additional funding, however, for effective advocacy to promote wide
application.
To support the community AIN in Honduras, BASICS has
ß developed a set of counseling sheets (láminas de consejería) for use when
talking to mothers, as well as a facilitator’s guide and a manual for monitoring
AIN;
ß adapted the IMCI flow chart to include supplementing all preschool children
with iron and vitamin A;
ß incorporated criteria for the classification of dengue fever; and
ß defined criteria for the proper diagnosis of streptococcal laryngitis so as to
avoid the excessive use of antibiotics.
An evaluation instrument for IMCI is to be developed in May 2000 for use at the regional
level.
A set of 10 materials was translated and adapted for Bolivia and was supported by
PAHO, BASICS, MOH, Community and Child Health (CCH), the Bolivian Pediatrics
Society, and UNICEF. Also in Bolivia, AIEPI comunitario materials are in the final
stages of development by COTALMA. An instrument for follow up (seguimiento) of the
trainees has also been developed.
Use of Tools
In Bolivia, in order to expand the training in AIEPI complementario, the MOH is
developing intermediary training centers in rural areas to facilitate access of auxiliary
nurses to the training process. The first major pilot test of the community materials
adapted by COTALMA is scheduled for May 2000. BASICS will be supporting PLAN
International’s work with community health workers in the Aroma District of the
Altiplano.
The only two countries to use the RPM tool to any significant degree are Bolivia and
Ecuador. In Bolivia, it has been incorporated into the MOH’s Seguro Básico de Salud,
which may lead to the sustainability of this critical IMCI component. The cost of
introducing this tool may be the major impediment to widespread use.
C. INTERAGENCY COLLABORATION
Ever since IMCI was developed by WHO and UNICEF in the early 1980’s, the strategy
has been adopted and supported by a growing number of public and private institutions:
multilateral (UNICEF and the World Bank), bilateral (governments of Canada, Germany,
the Netherlands, Japan, Spain, and the United States), national (MOHs, pediatric
IV. IMPLEMENTATION FINDINGS
27
societies, and nursing and medical schools), NGOs, and even individual doctors in their
private practices.
Since 1998, UNICEF has been coordinating the worldwide IMCI Interagency Group,
stressing the community aspects of IMCI, and has helped in the development of several
training and health facility assessment tools. UNICEF has also been instrumental in the
incorporation of IMCI into de Seguro Básico de Salud in Bolivia.
Interagency collaboration can be manifested in at least three different, simultaneous, and
complementary ways: by the agencies and governments collaborating through financial
contributions (World Bank), by providing technical assistance (BASICS and PAHO), and
by implementing various levels of operations (NGOs and the MOH).
D. COMMUNITY PARTICIPATION
WHO’s original generic IMCI materials specify three main building blocks of IMCI:
trained human resources, systems support, and the community role. To date, the
emphasis has been on the first two components, with heavy weighting on training.
Traditionally, the major stakeholders in community activities have been the NGOs and
those institutions that finance them (e.g., UNICEF, USAID). While UNICEF was
initially expected to take a major worldwide role in the development of the community
aspect of IMCI, its emphasis to date has been in five countries in Africa and one
(Ecuador) in Latin America. UNICEF is presently developing a district-based
communications strategy to actively involve families and communities in children’s
health. Part of this strategy will be a manual that can be used throughout Latin America.
Aspects of community participation in the three countries visited are described below.
Honduras
Even before IMCI began in Honduras, the health authorities in the country had already
been working in developing and implementing a strategy geared to children under 5, and
based in the active participation of the community. The point of entrance for children’s
care under this strategyæAINæis the growth and development of the child.
USAID/Honduras and BASICS were and continue to be major supporters of this
approach.
In contrast, PAHO’s support for the institutional component of IMCI naturally led to the
development of an approach to and materials supporting a different community-based
component. The emphasis of this approach is more focused on identifying and treating
sick children than on promoting health and preventing disease. Thus, these two differing
approaches have stimulated significant debate on their relative merits and weaknesses.
As a result, overall progress in IMCI has slowed down and probably now will be able to
regain its initial momentum following the recent Secretary of Health decision to apply
institutional IMCI at the health services’ level and AIN Comunitario at the community
level.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
28
There is increased interest in developing the aspects of community IMCI that focus on
promoting a child’s health on a day-to-day basis versus intervening only when the child is
sick. In this regard, the community health promoters and mothers are not the only
community-based individuals who can play a role in developing these aspects. For
example, members of the extended family, traditional midwives, schoolteachers, and
religious leaders are potentially important allies and thus potential trainees in community
IMCI.
Bolivia
In Bolivia, the major stakeholders in IMCI all agree on the need to develop and
implement the community-based component. The materials supporting this intervention
(facilitator’s guide, training manual, hand-held flip chart, the modified IMCI protocol,
and the technical manual) are aimed at the community health workers (ACSs) and were
pilot tested in Calamarca in March 2000. They have been revised and are now in the
final stages of production. The next step is for BASICS and PAHO to support PLAN
International’s introduction of these materials with their ACSs in selected areas of the
Altiplano.
Peru
In Peru, PLAN International has been working very intensely on community participation
aspects of IMCI with the collaboration of the MOH, PAHO, and UNICEF, particularly in
areas north of Lima, such as Piura. After some pilot tests conducted in 1999, community
IMCI will also be implemented in Chiclayo and adjacent areas.
E. BARRIERS TO IMPLEMENTATION
An initiative of this complexity will naturally encounter a series of barriers that impede
its successful implementation. A number of these barriers have financial underpinnings;
however, they are not listed under a separate category dedicated only to financial issues.
Institutional
Cultures
The partners represent very different institutional culturesæin their worldview, in their
history, and in their way of operating. Although these differences were embraced in the
original design as being supportive, they were exerting a net negative effect by the time
of this evaluation.
Resistance to Integration of IMCI as an MOH Initiative
As of the time of this evaluation, the MOH of Guatemala had not officially adopted the
IMCI strategy.
IV. IMPLEMENTATION FINDINGS
29
Indicators
Disagreements about the existing indicators and lack of agreement about new alternative
ones have been a constant source of friction among the partners. Given that reporting on
key indicators is not taking place, there is a great need for developing alternative
indicators.
Appropriate Recognition
In Honduras, lack of recognition of the role of USAID as a funding agency for LACI
activities hinders the maintenance of a good working relationship among the partners.
Collaboration and Communication
The initial proposal from PAHO and BASICS, “LAC Regional Results Package:
Integrated Management of Childhood Illnesses (IMCI)æ1997-2001” stated, “…close
collaboration with USAID’s LAC and Global Bureaus, as well as USAID Field Missions,
PAHO Country Offices and other agencies interested in IMCIæNGOs among them, will
be sought through implementation.” (p. 2) However, as the evaluation team had the
opportunity to observe, there was frequently insufficient, poor, or no communication
among the different partners in this initiative.
Disagreement about the Community Component of IMCI in Honduras
While PAHO is promoting the use of the PAHO-UNICEF community IMCI approach
and materials, BASICS and the World Bank are promoting the AIN program, which had
previously been adopted by the MOH. Disagreements regarding these two different
approaches are a major source of friction and have led to a weakening of collaborative
efforts in Honduras.
Addressing the Nutrition Aspects of IMCI
Reportedly, health professionals, particularly medical doctors, have difficulties in
effectively addressing nutrition problems identified through the IMCI exam. Identified as
a weakness in the formal evaluation of IMCI in Bolivia, this issue should be worked out
in modifying the training.
Supervision
In all three countries visited, supervision was noted as one of the weakest components to
effective IMCI implementation. In some cases, as in Peru, the emphasis in supervision is
more on administrative aspects than on technical and educational aspects.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
30
Staffing
At all levels of partner institutions in IMCI, most of the key persons involved have other
significant responsibilities. For those few persons for whom the regional IMCI initiative
is their major job responsibility, the scope of this responsibility is too much for just one
person or a small team.
Personal
Weaknesses in interpersonal communication skills among specific key individuals serve
to magnify the management and communication problems experienced at the institutional
level.
External
External factors, outside the control of any given partner, that have limited IMCI progress
to date, include
ß the downsizing of USAID worldwide and increased focus on Africa,
ß Hurricanes Mitch and George,
ß the predominance of political concerns over sound technical approaches that
led to situations (such as in Bolivia) where the training course was shortened
from 11 to 5 days in spite of the experts’ opinion that it would negatively
affect the quality of the training, and
ß the frequent transfers or change of health personnel both at the central (MOH)
and peripheral levels (districts, hospitals, and clinics), as well as in the
implementing partners’ structures.
F. FACILITATING FACTORS
Over the course of this initiative, different factors have facilitated its implementation,
including political, institutional, financial, personal, and external.
Political
The success of an initiative of this nature is integrally connected with the ability to obtain
approval and continual support of the key health and political authorities in the countries.
Having obtained this high-level support at an early stage in seven out of the eight
countries involved has allowed the process to move forward at a reasonable pace.
IV. IMPLEMENTATION FINDINGS
31
Institutional
As a regional initiative, the LACI initiated, stimulated, and funded activities that
countries could not (tools) or would not (workshops and international meetings)
necessarily support on their own. It did so through a wise choice of partners: PAHO had
the reach and stability of a regional institution; BASICS had the program skills as an
international development organization (versus the medical approach of PAHO), the
specific technical skills, and had been more involved than PAHO in early stages of
developing IMCI (e.g. the first adaptation in Zambia); and USAID provided the funding
and management oversight. When PAHO and BASICS could go into a country together,
they could talk to both the MOH and the USAID Mission. The strategic coordination of
institutional strengths helped open up the IMCI dialogue in-country.
Obtaining the commitment of important national institutions, such as universities
(medical and nursing schools) and pediatric societies, has been critical to the success of
this initiative. Credit for this achievement is shared among the partners and their in￾country colleagues and varies in degree from country to country. A case in point is the
early support of the pediatric society in Bolivia, which contributed to the widespread
acceptance of IMCI. In addition, the USAID/Bolivia bilateral project (CCH), which was
implemented before IMCI, gave the LACI a head startæCCH had been operating in 11
districts, so IMCI started there even before the LACI. CCH paid for the first courses.
The good working relationship among PAHO, BASICS, USAID/Bolivia, and the MOH
existed from the time that project was implemented.
Financial
IMCI is so appealing as an effective health strategy that it has attracted complementary
funding for the various IMCI components (e.g., Japanese funding for drugs in Honduras
and American Red Cross funding for community IMCI in Honduras and other countries).
Personal
The overwhelming consensus of the persons interviewed is that IMCI is an effective
strategy for identifying and treating sick children and thus preventing more serious
medical sequelae or even death. It is possible to verify the positive results of IMCI for
individual patients. Thus, individuals trained in IMCI who implement it correctly develop
a strong commitment to the cause, a commitment that was repeatedly evident in the
interviews conducted during this evaluation. In Bolivia, the head of the MOH’s
Department for Client Services (Departmento de Atención a las Personas, where Seguro
Básico de Salud and therefore IMCI are housed) was the former president of the Bolivian
Society of Pediatrics (thus well respected and connected). He himself was trained as an
IMCI facilitator, which explains in part his commitment to IMCI.
High personal commitment to IMCI by key individuals throughout the systemæfrom
high-ranking administrators and managers to the health practitioners directly involved in
treating patientsæcreates a positive climate for change and the effort needed to bring
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
32
about change in one’s own behavior and that of one’s colleagues and supervisees.
Evidence obtained in the three countries visited shows that the LAC Regional IMCI
Initiative is acting as an instrument of change in their respective health care systems.
External
In the three countries visited, the collaboration of other governments/international
organizations has been critical for the implementation of the strategy and for obtaining
additional funds to support IMCI activities throughout the country.
33
V. SPECIAL ACHIEVEMENTS OF THE LAC
REGIONAL IMCI INITIATIVE
Based on interviews, observations, and visits in three different countries, IMCI has made
substantial progress in the areas of advocacy, institutionalization of IMCI, coherence of
IMCI’s message, training activities, creation of tools and dissemination of experiences to
other countries, ability to work under adversity, and sustainability.
A. ADVOCACY
Effective advocacy by PAHO, BASICS, and USAID/LAC Missions to MOHs, NGOs,
and bilateral and multilateral agencies is manifested by seven out of eight countries
having adopted the strategy and by the amount of funding obtained from other
governments and agencies (USAID Missions, the World Bank, the Inter-American
Development Bank, and PVOs) to carry out in-country IMCI activities. Because of its
special relationship with governments and ministries of health, PAHO has been
particularly effective in its advocacy role. As a result, there has been a strong political
commitment by health authorities at national, regional, and local levels to this initiative.
B. INSTITUTIONALIZATION OF IMCI
In Honduras, the Secretary of Health has officially adopted the IMCI strategy.
In Bolivia, IMCI has strong support from the Ministry of Health, as well as from
academic institutions in the country, such as the Sociedad Boliviana de Pediatría. The
IMCI protocol has been incorporated into the MOH’s overall protocol for health services
for children under five (Seguro Básico de Salud). The document which manifests this
integration (Guías Técnicas para la Atención del Seguro Básico de Salud) was a product
of the efforts of many institutions in Bolivia, among them the MOH, UNICEF, BASICS,
and PAHO.
In Peru, the Ministry of Health has officially adopted IMCI. In December of 1997, a
Comisión Nacional Coordinadora AIEPI was created in the Ministry and has been
functioning actively since then. IMCI activities complement the strategy called Proyecto
de Capacitación Materno Infantil (PCMI), which is financed by USAID as part of
Proyecto 2000 and also with funds from the Peruvian government. In addition, both
Bolivia and Peru have very active interagency committees that provide effective support
to IMCI activities in the country.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
34
C. COHERENCE OF IMCI’S MESSAGE
One key factor in introducing a new strategy, for example, IMCI, is to maintain the
integrity of the message, regardless of the persons conveying that message, and
regardless of the audiences receiving it. Thus far, it appears that the IMCI message has
been maintained over space and time in the three countries visited. This message has
been effectively conveyed in different countries, by many different collaborators, under
very different political and cultural situations, and yet the message has not been distorted
nor diluted. Through the interviews and field visits to three countries, a clear and
consistent message was communicated regarding the challenges and promises of IMCI.
D. TRAINING ACTIVITIES
Although in-country training in itself is not part of the regional initiative, its quality is
related to the quality of the training manuals and the expertise of the trainers. The
initiative’s activities have had a direct bearing on these latter two components. As a result
of the effectiveness of IMCI training activities:
ß more than 8,000 people have been trained in this strategy, including
physicians, nurses, and auxiliary nurses;
ß the trainees interviewed have shown sound knowledge of the IMCI strategy;
ß there is a growing tendency to look at the health of children from a holistic
point of view as opposed to having specific diseases to be managed by vertical
programs;
ß there is a clear awareness by mothers, promoters, and other community
leaders of their role in keeping children healthy;
ß mothers are being counseled more effectively at health centers;
ß health workers manage drugs more appropriately;
ß pneumonia is diagnosed more quickly and accurately in children;
ß health workers have improved their referral capacity, particularly for
pneumonia and diarrheal diseases; and
ß there is the perception (not yet quantified) that IMCI has diminished
children’s deaths by pneumonia and diarrheal diseases.
In Peru, the initial 11–day IMCI training course was adapted and shortened to 7 days by
the MOH, keeping the main components of the original course. Peru’s example has been
enthusiastically adopted and implemented in Honduras, which has led to its wide
V. SPECIAL ACHIEVEMENTS OF THE LAC REGIONAL IMCI INITIATIVE
35
application throughout the country. Other countries in the region have shown interest in
the 7-day version.
E. CREATION OF TOOLS AND DISSEMINATION OF EXPERIENCES TO OTHER
COUNTRIES
An RPM tool was developed by Management Sciences for Health, which was financed
by USAID through BASICS, with the collaboration of PAHO. This tool has been
successfully field tested in Bolivia and Ecuador and the results indicate that this activity
could profit from expansion to other countries with regional funds support.
In Bolivia, LACI funds have supported COTALMA in developing materials for
community IMCI activities and a related package of materials (Complementary IMCI)
for training auxiliary nurses. Other materials to support community IMCI efforts were
developed in Honduras prior to the introduction of IMCI itself. These materials, which
are part of the program called AIN Comunitario, have now been exported to Ecuador and
Nicaragua and eventually will be sent to other Central American countries with World
Bank support. AIN is also being considered by PAHO for inclusion among IMCI training
materials.
In addition to the tools, the regional meetings sponsored by the initiative are effective for
the exchange of information and experiences among the different countries. PAHO’s
regional advisers actively participate in international meetings connected with the
application of the IMCI strategy in Latin America (e.g., Durban, South Africa, meeting
on IMCI, where WHO invited IMCI’s regional advisors to participate and share their
experience). In that regard, many among those interviewed expressed their belief that of
all the regions, Latin America was the most advanced in the implementation of the
strategy. PAHO’s Meta 2002 strategy (Goal 2002ætheir regionwide vision for reducing
infant and child mortality over the next three years) has proved to be an effective
marketing strategy for IMCI and will likely help mobilize additional resources.
F. ABILITY TO WORK UNDER ADVERSITY
One of the remarkable results of introducing IMCI has been the stimulation of active and
dedicated participation of health staffs and communities, sometimes at considerable
personal cost. The examples below are related to the personal commitment characteristic
that was discussed earlier as a facilitating factor. The high quality of inputs from LACI
(e.g., materials, development of national training teams, technical assistance) has
contributed to positive change in key role models (individuals as well as institutions) in
the three countries visited. Their commitment to IMCI serves as a powerful motivating
force for their colleagues.
Although Honduras was devastated by Hurricane Mitch, interviewees did not mention
this as a major obstacle (or did not mention it at all), which suggests that they have
extraordinary commitment and enthusiasm to overcome this obvious tremendous
obstacle. Health workers are not using Mitch as an excuse for being behind or for
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
36
resources having been re-directed to deal with the hurricane’s aftermath. Perhaps part of
the reason for their sustained enthusiasm is that they see that IMCI works in their own
health centers. The MOH and the facilitators remain enthused because the course
maintains its standard level of quality and they are rewarded by the motivation of the
trainees.
In Bolivia, one of the facilitators gives up her vacation days in order to work as a
facilitator because she cannot take the time off officially from her regular job. In Peru,
one doctor, the head of a health center, undertook a study of the change in the use of
antibiotics in 7 rural and urban health centers, analyzing 800 cases before and after the
implementation of IMCI. He did this in his spare time, after the normal working hours of
his own center and those he visited.
G. SUSTAINABILITY
Several ways of working towards the sustainability of the IMCI strategy have been
implemented in the countries visited. Although they are not directly connected to the use
of regional funds, they show a commitment from those involved for making long-lasting
changes through IMCI. Examples of progress toward sustainability of IMCI include its
incorporation into the policies of the ministries of health, expanding its base of financial
support through other agencies and governments’ participation, and the intersectoral
collaboration with the education sector. For example, in Peru, one motivated medical
doctor in charge of a periurban health center (Cura Mori, Piura) has developed a strong
cooperative relationship with the primary school. Activities undertaken include training
teachers and school health agents to assist with de-worming of students, detection of
anemia and guidance for chlorinating household drinking water, and training students to
serve as health advocates among their peers and their own families.
The use of both mothers and promoters as effective channels for the delivery of IMCI
messages within their communities is another example. Due to the complementary role of
providers of health services (e.g., MOH) and consumers (mothers and children), the
potential for services to be mutually acceptable increases when community members
speak the same language and understand the same messages as do the health providers.
Thus, training in the community component of IMCI is critical to increasing the overall
impact of the strategy.
IMCI has been incorporated into the curriculum of nursing and medical schools. In the
three countries visited, IMCI has been introduced, to different degrees, into the curricula
of nursing and medical schools. In Honduras, the nursing school in Choluteca,
CERARH-Sur, has integrated IMCI into the students’ study plans. In Bolivia, IMCI has
been integrated into two medical schools, San Simon (Cochabamba), and the University
of San Andres (La Paz), and in the Nur School for auxiliary nurses in Santa Cruz. In
Peru, IMCI is integrated into the medical school, Universidad Nacional Pedro Ruiz Gallo
(Chiclayo). In this way, future health providers are being trained through normal school
channels, which will eventually eliminate the need for separate (and expensive) IMCI
initial training activities.
V. SPECIAL ACHIEVEMENTS OF THE LAC REGIONAL IMCI INITIATIVE
37
The impetus for the above initiatives related to curriculum changes derives in large part
from members of the original national training teams. As they proceeded with replicating
IMCI training in-country, they realized what a tremendous task lay ahead in training all
the current MOH health providers. They also realized that it is easier to introduce a new
concept (IMCI) to a student undergoing her/his initial professional training than to try to
convince a seasoned practitioner to change ingrained approaches and behaviors.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
38
39
VI. CONCLUSIONS
A. OVERALL
Progress to Date
Based on interviews, observations, and visits in three countries, the LACI has made
substantial progress towards the Strategic Objective of providing more effective health
services to children in response to CDD, ARI, and malnutrition. As expected, the degree
of implementation of the various components of IMCI varies by country and within
country: in some areas, activities are in the pilot testing phase (e.g., community IMCI in
Peru and Bolivia) and other areas have reached a point of moderate expansion (e.g.,
intensive expansion of the clinical training by the MOH in Honduras). Even though the
momentum of the LACI slowed down somewhat in the past 12 months, as reported in
USAID’s LAC Regional R4 report (March 22, 2000), this has not been a major detriment
to overall progress.
Key representatives of all stakeholders in Washington and in the three countries visited
expressed their keen interest in the findings of this evaluation. This positive attitude
bodes well for regaining any lost momentum, making appropriate midcourse corrections,
and achieving the overall goals by the end of the five-year period.
The most significant obstacles to progress (summarized in following sections) stem from
a combination of issues related to project design, management, and an overemphasis in￾country on clinical training, to the detriment of improving critical health support systems.
If these obstacles can be addressed effectively and in a timely manner, it is possible that
LACI will accomplish the majority of its five-year goals in those countries that have
participated actively (i.e., in possibly five or six of the eight target countries) and
stimulate the more effective application of the strategy in other countries in the region.
It is the people, as individuals and in teams, whose dedication to the IMCI philosophy
and practice makes it possible to carry forth and rise above all of the problems that they
face in implementing IMCI within their own spheres of influence. These people have
participated, either directly or indirectly, in activities supported by the LAC Regional
IMCI Initiative. The high quality of the initiative’s inputs and activities has contributed
significantly to creating a positive and supportive environment for change.
Role of LAC Regional IMCI Initiative as a Catalyst
The LAC IMCI Regional Initiative has functioned as a true catalyst, albeit not the only
one, for IMCI in the region. It joined the efforts of other agencies and governments to
accelerate the adoption of IMCI, such that the changes introduced have real potential for
becoming part of the daily routine of health facilities, including hospitals, health centers,
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
40
and health posts in rural areas. Midway into the five-year initiative, concrete evidence of
interest for working at the community level was found, and a set of actions are planned in
that regard in the countries visited. As a catalyst, USAID/LAC is taking some risks, in the
sense that it cannot predict nor control some of the outcomes of the activities it is
supporting in a group of very different countries. The fact that the outcomes have been
so positive to date is a real credit to USAID/LAC and the partners it chose for this effort.
Unique Contribution
Given a state of almost tabula rasa of IMCI in Latin America at the beginning of LACI,
USAID/LAC and its partners had to gamble that the activities they chose for funding
would be viable, practical, economical, complementary, and that they would fulfill the
desired purpose. The findings of the evaluation suggest that these activities have been
strategicæin their content, timing, budgetary support, and location. Many of these
activities are ones that other IMCI supporters, especially local ministries of health, could
never have financed themselves (e.g., adaptation of manuals, creation of new materials, a
national evaluation) or would not have been considered priorities within their limited
budgets (e.g., participation in international workshops to address specific IMCI issues).
In this regard, the LACI contribution has been unique and absolutely critical to the IMCI
adoption process.
LACI–supported activities are leading to some benefits beyond those contemplated for
IMCI itself. In all countries visited, IMCI is already showing promise as an organizing
paradigm for other non–IMCI services provided in the clinic. IMCI introduced the
technique of triaging for danger signs and immediately referring emergency cases to the
most qualified staff person available; this system of triaging is now accepted as a logical
way to identify any emergency case that arrives. IMCI also introduced a very specific
concern for quality of care as reflected in waiting time at the clinic, teaching health
providers how to communicate respectfully with patients, making sure caregivers
understand the instructions for home care. This concern for client satisfaction has easily
been applied to the whole clinic clientele, not only IMCI patients.
Window of Opportunity
There is great interest in IMCI right now, across the breadth of persons interviewed and
the depth of locations visited. The window of opportunity for IMCI currently is open, but
it is finite and probably will be shutting in the not-too-distant future as interest (and donor
support) increases in maternal and perinatal mortality and/or is shifted due to an
unforeseen emergency.
B. PROJECT DESIGN
Overall
In general, the overall design of the LACI is sound; the major problems have been
experienced in the implementation of the design.
VI. CONCLUSIONS
41
Indicators
The indicators for the SO and IRs 3 and 4 are not adequate to capture the progress of
LACI activities.
Choice of Countries
The reluctance of Guatemala to participate in IMCI has been a distraction and has used
valuable resources with little result. Other countries could make better use of the
resources planned for Guatemala and any other country, such as Haiti, which has not
actively engaged in LACI as of the time of this evaluation.
C. MANAGEMENT AND ADMINISTRATION
Steering Committee
The steering committee is not functioning as satisfactorily as it could (e.g., compared
with the steering committee of the LAC Health Sector Reform Initiative) nor as it should
in order to make the most of the human, financial, advocacy, and networking resources
available to it. The LACI is designed with great expectations and great potential for
synergy in planning, implementation, and coordination among the different partners.
Organizational and personality conflicts are preventing this synergy from happening to
the extent possible.
Given the turnover in the steering committee and the availability of the evaluation results,
now would be a good time to regroup, reflect, and recommit to the initiative. In the
words of one interviewee, “Partnerships need to be coordinated; they don’t grow up
naturally.”
In–country Coordination
In most target countries for the LAC Regional IMCI Initiative, both partners (PAHO and
BASICS) have in-country representation. There needs to be a focal point for LACI
communications in each country for the purpose of efficient and effective communication
within the country and between the countries and USAID/LAC.
Receiving Credit
IMCI is owned by everyone; thus, USAID is one of many donors participating in this
global effort. The LAC Regional IMCI Initiative is not receiving the credit it is due,
especially within countries where funds are dispersed through several channels and lose
their identity.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
42
D. IMPLEMENTATION
Facilitating Factors
The quick adoption of IMCI by seven of the eight participating countries is indicative of
the political support given to this strategy. The LACI design, choice of experienced
partners, advocacy with other collaborators, and local USAID support have all come
together to help effect the positive results noted by the evaluation team. In addition,
expansion of IMCI is being conducted in many countries through coordination with the
Child Survival Resources and Collaboration group of PVOs.
Obstacles
This project has had its share of normal obstacles (e.g., turnovers in key personnel, late
reports, difficulty in reporting on indicators). However, the one thing that seems to hold
it back the most is the difficulty in working together (between and among the partner
agencies).
Overemphasis on Clinical Training and Underemphasis on Health Systems Support
Components
In the three countries visited, IMCI training activities are expanding too fast, to the
detriment of the other system components (i.e., attention to quality of services,
supervision, and integration of the IMCI indicators into the HIS; monitoring and
evaluation; management and cost analysis of IMCI drugs; health facility assessment;
overall cost analysis of adopting IMCI). Many tools already exist for helping countries
incorporate IMCI into their health management and administration systems. Developed
by the LACI partners, these tools are not sufficiently promoted nor have they been
adopted in the areas visited in Honduras, Bolivia, and Peru. If more attention is not given
to these IMCI system support components very soon, LACI will not be able to adequately
capture the true depth nor breadth of its results at the end of the five-year project.
Community Component of IMCI–Honduras
If the conflict between the two approaches to community IMCI is not adequately resolved
soon in Honduras, the country runs the risk of not only slowing down its IMCI
implementation, but even losing progress gained so far. An agreement needs to be
reached—and compliance promised—for the more effective implementation of AIN and
AIEPI Comunitario where they will play complementary and not competing roles. This
agreement will need a strong advocacy role from the MOH and PAHO and could have a
significant contribution from COMSAIN. PAHO, BASICS, and USAIDæin the forum of
their LACI steering committee and in the forum of their respective representatives in
Hondurasæneed to reach mutual consensus on how to proceed with community IMCI in
Honduras. The way this disagreement is solved in Honduras can set an example for
similar approaches to problem solving at the regional level.
VI. CONCLUSIONS
43
E. SUSTAINABILITY
As of the time of this evaluation, the LACI shows great promise for making a major
contribution to the eventual sustainability of IMCI within nationwide health systems in
the participating countries. Currently, there are five different approaches being pursued
to promote sustainability, all with initially promising results, although mostly on a
limited and pilot basis to date:
ß incorporation of IMCI into MOH policies,
ß stimulation of donor support and funding for the expansion of IMCI,
ß incorporation of health issues into activities of the education sector,
ß training community-based health promoters and mothers in IMCI messages,
and
ß incorporation of IMCI into the curricula of nursing and medical schools.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
44
45
VII. RECOMMENDATIONS
Through the IMCI Initiative, USAID/LAC has made a unique contribution to the
implementation of this strategy in Latin America. It has done so through a well-designed
and flexible framework and approach to providing financial support for regional activities
not usually financed either by country USAID Missions, governments, or international
bilateral agencies. These efforts should continue to be supported to improve the health
status of children throughout the hemisphere. The recommendations that follow are
directed towards USAID/LAC and the two partners (PAHO and BASICS), as specified.
A. RECOMMENDATIONS TO USAID/LAC
Funding
Continued funding should be provided to LACI, which has led to many important
achievements in spite of some obstacles. Although USAID/LAC cannot accept funds
from other sources, they can support the fundraising efforts of others (especially those
already underway by PAHO and the ministries of health) to expand and enhance IMCI in
the Americas, particularly through in-country activities.
Resources
Remaining LACI resources should be invested only in those six target countries that have
so far been active participants in the initiative, unless health authorities in Guatemala and
Haiti show greater interest in having the initiative implemented in their respective
countries.
Steering Committee
Although considerable progress has been achieved in terms of improving the
collaboration between BASICS and PAHO, the steering committee could profit from
three distinct but related actions:
ß having each member empowered by her/his institution with a level of
decision-making authority appropriate to the critical managerial role of the
steering committee;
ß reaching an understanding among the partners regarding the different
institutional cultures and the need to address these differences more
effectively, such as by conducting teambuilding activities with an outside
facilitator; and
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
46
ß redesigning the remaining annual work plans and budgets to include a clear
delineation of roles and responsibilities between both implementing partners.
Indicators
Because of the regional nature of the project, indicators should be adapted to regional
activities. Indicators for the overall Strategic Objective and for Intermediate Results 3
and 4 should be revised and baseline and annual data should be reported.
B. RECOMMENDATIONS TO PAHO AND BASICS
Compliance
PAHO and BASICS reports and work plans should be sent to USAID/LAC on a timely
basis. New staff for the initiative should be approved by USAID, according to the terms
of their agreements.
Appropriate Credit
In several instances, the source of financial support for LACI activities was not properly
acknowledged. It is important that appropriate credit be given, according to the terms of
PAHO’s and BASICS’ agreements with USAID.
C. RECOMMENDATIONS TO ALL PARTNERS (USAID, PAHO, AND
BASICS)
Communication among the Partners
Flow of communication between USAID/LAC and the USAID country Missions, as well
as between PAHO and BASICS with USAID Missions, should be increased, which could
lead to a more effective integration of activities at the local level.
IMCI Support Systems
Until now, the emphasis of IMCI at the country level has been on the introduction and
expansion of training activities. However, this should not be done in detriment of
improving other components (supervision, monitoring and evaluation, HIS, and drug
management), which are equally critical. In addition, training lessons were better learned
when follow-up activities were conducted to reinforce the training, showing the
importance of redesigning training plans to make them complete.
Sustainability
Sustaining IMCI should be reinforced by continuing efforts at integration into nursing
and medical schools’ curricula, as well as by obtaining additional sources of financial
support. Also important is the continuation of joint implementation activities with NGOs
VII. RECOMMENDATIONS
47
(such as Plan International and CARE). In addition, new programs and organizations
should be contacted and made part of this initiative. Ultimately, the sustainability of the
IMCI strategy will be another measure of success of this already worthwhile initiative.
D. PRIORITY OF ALLOCATING RESOURCES FOR IN-COUNTRY ACTIVITIES
Given the limited resources of the LACI budget and time, the priority of the following
three activities should be
ß improving IMCI support systems,
ß introducing and expanding the community IMCI component, and
ß undertaking efforts to promote sustainability.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
48
ANNEXES
A: IMCI Action Memorandum.........................................................................................51
B: Grant to PAHO ............................................................................................................57
C: USAID Results Framework .........................................................................................67
D: LAC IMCI Schematic Design......................................................................................77
E: Scope of Work .............................................................................................................81
F: Evaluation Schedule.....................................................................................................89
G: Documents Reviewed ..................................................................................................93
H: Interview List...............................................................................................................97

ANNEX A
IMCI ACTION MEMORANDUM
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
52
ANNEXES
53
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
54
ANNEXES
55
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
56
ANNEX B
GRANT TO PAHO
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
58
ANNEXES
59
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
60
ANNEXES
61
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
62
ANNEXES
63
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
64
ANNEXES
65
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
66
ANNEX C
USAID RESULTS FRAMEWORK
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
68
IMCI Initiative
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and
policy interventions
APPROVED: 3/97; IMCI Results Package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: IMCI Results package: More effective delivery of child health services in
response to DD, ARI, and malnutrition.
SO 3 INDICATOR #2: Target countries with 10% of health facilities* delivering IMCI
services
UNIT OF MEASURE: Number
SOURCE: PAHO and BASICS
INDICATOR DESCRIPTION:
a) *Health facilities defined as ambulatory health
facilities, health centers and health posts.
b) Delivering services means at least one person trained
in IMCI and continuous availability of IMCI drugs
COMMENTS: Target countries for the IMCI Results
package are: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua, and Haiti
YEAR PLANNED ACTUAL
1996 -- 0 (B)
1997 0 0
1998 3 No data
1999 5 No data
2000 7
2001
(T)
8
ANNEXES
69
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and
policy interventions.
APPROVED: 3/97; IMCI Results Package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: IMCI Results package: More effective delivery of child health services in
response to DD, ARI, and malnutrition.
SO 3 INDICATOR #3: Target countries with 90% of health facilities in early use areas
delivering IMCI services
UNIT OF MEASURE: Number
SOURCE: PAHO and BASICS
INDICATOR DESCRIPTION:
 a) Health facilities defined as ambulatory health
facilities, health centers and health posts.
 b) Designated districts will be identified early in 1998
in collaboration with MOH/PAHO/BASICS
 c) Delivering services means at least one person trained
in IMCI and continuous availability of IMCI drugs
COMMENTS: Target countries for the IMCI Results
package are: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua, and Haiti
YEAR PLANNED ACTUAL
1996 -- 0 (B)
1997 0 0
1998 3
3
1999 5
3
2000 7
2001 8 (T)
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
70
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and
policy interventions.
APPROVED: 3/97; IMCI Results Package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: Intermediate Result 3.2.1; IMCI Results package: Country health leaders
have information for IMCI adoption.
INDICATOR 3.2.1a: Target counties where Ministry of Health leaders have adequate
information regarding IMCI.
UNIT OF MEASURE: Number
SOURCE: PAHO and BASICS Reports.
INDICATOR DESCRIPTION: Adequate information
is defined as MOH have been briefed on the IMCI
strategy, country requirements in order to implement
(e.g. policies, human resources, drug management
system, etc).
COMMENTS: Target countries for the IMCI Results
package are: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua and Haiti.
Year PLANNED ACTUAL
1996 -- 0 (B)
1997 2 7
1998 8 8
1999 8 8
2000 8
2001
(T)
8
ANNEXES
71
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and
policy interventions.
APPROVED: 3/97; IMCI Results Package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: Intermediate Result 3.2.1; IMCI Results package: Country health leaders have
information for IMCI adoption.
INDICATOR 3.2.1b: Target counties with official decision to adopt IMCI.
UNIT OF MEASURE: Number
SOURCE: PAHO and BASICS Reports
INDICATOR DESCRIPTION:
COMMENTS: Target countries for the IMCI Results
package include: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua and Haiti. Although
Guatemala MOH officials had indicated that they would
adopt IMCI, no actions have been taken since, so we do
not count them as having made this decision.
Year PLANNED ACTUAL
1996 -- 0 (B)
1997 2
7
1998 5 7
1999 8 7
2000 8
2001
(T)
8
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
72
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and policy
interventions
APPROVED: 3/97; IMCI Results Package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: Intermediate Result 3.2.2; IMCI Results package: Country plans and strategies
in place for introduction and implementation of IMCI
INDICATOR 3.2.2: Target countries with IMCI plans and strategies adopted including
identification of resource requirements
UNIT OF MEASURE: Number
SOURCE: PAHO and BASICS Reports
INDICATOR DESCRIPTION:
COMMENTS: Target countries for the IMCI Results
package are: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua and Haiti.
*Until and unless Guatemala adopts IMCI, this cannot
happen. No change since the mid-1999 election.
YEAR PLANNED ACTUAL
1996 (B) 0
1997 2 7
1998 4 7
1999 7 7
2000 8*
2001 (T) 8
ANNEXES
73
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and policy
interventions
APPROVED: 3/97; IMCI Results Package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: Intermediate Result 3.2.3; IMCI Results package: Improved country capacity to
implement IMCI
INDICATOR 3.2.3a: Target countries where 80% of districts have developed plans for IMCI
implementation.
UNIT OF MEASURE: Number/ Proportion
SOURCE: PAHO and BASICS Reports
INDICATOR DESCRIPTION:
COMMENTS: Target countries for the IMCI Results
package are: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua and Haiti.
YEAR PLANNED ACTUAL
1996 -- 0 (B)
1997 0
0
1998 2 No data
1999 6
4
2000 8
2001 (T) 8
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
74
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and policy
interventions
APPROVED: 3/97: IMCI Results Package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: Intermediate Result 3.2.3; IMCI Results package: Improved country capacity to
implement IMCI
INDICATOR 3.2.3b: Target countries with more than 10% of ambulatory health facilities in early
use areas which have IMCI norms, trained health service providers, and IMCI essential drugs
available.
UNIT OF MEASURE: Number
SOURCE: PAHO and BASICS Reports
INDICATOR DESCRIPTION:
a) IMCI norms and trained health service providers;
b) IMCI essential drugs available
COMMENTS: Target countries for the IMCI Results
package include: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua and Haiti.
YEAR PLANNED
a/b
ACTUAL
a/b
1996 -- 0 (B)
1997 0/0 0/0
1998 3/1 No data
1999 6/3 No data
2000 8/5
2001 (T) 8/8
ANNEXES
75
OBJECTIVE: Strategic Objective #3: More effective delivery of selected health services and policy
interventions
APPROVED: 3/97; IMCI Results package
COUNTRY/ORGANIZATION: LAC/RSD-PHN
RESULT NAME: Intermediate Result 3.2.4: Monitoring and evaluation used to adjust IMCI
program plans.
INDICATOR 3.2.4: Target countries which have incorporated monitoring and evaluation findings
into annual IMCI national plans
UNIT OF MEASURE: Number
SOURCE: PAHO and BASICS Reports
INDICATOR DESCRIPTION:
COMMENTS: Target countries for the IMCI Results
package are: Bolivia, Ecuador, Peru, El Salvador,
Guatemala, Honduras, Nicaragua and Haiti.
YEAR PLANNED ACTUAL
1996 -- 0 (B)
1997 0
0
1998 3
0
1999 5
3
2000 8
2001 (T) 8
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
76
ANNEX D
LAC IMCI SCHEMATIC DESIGN
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
78
ANNEXES
79
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
80
ANNEX E
SCOPE OF WORK
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
82
Scope of Work
Midterm Evaluation
Health Priorities Project, 598-0825
IMCI Initiative
30-August-99
Background
Worldwide some 12 million children die each year before they reach their fifth birthday.
Approximately 565,000 of those deaths occur in Latin America and to a large extent are
preventable. The majority of these deaths are attributable to diarrhea, pneumonia,
measles, malaria, and malnutrition, or a combination of these conditions. Global burden
of disease estimates have projected that these same illnesses will remain the most
important causes of child mortality continuing into the year 2020 without increasing
attention to address them.
Building on past experience and lessons-learned from disease-specific control programs,
the World Health Organization (WHO), in collaboration with UNICEF, developed an
integrated approach to the assessment, classification, treatment, and counseling of sick
children and their caretakers. This approach is known as Integrated Management of
Childhood Illness (IMCI) and forms the basis for new treatment guidelines at first-line
health facilities. IMCI provides for integrated case management of the most common
childhood illnesses (acute respiratory infections, diarrhea, malaria, measles, and
malnutrition) which in the past were generally considered and treated separately. PAHO
estimates, that these conditions are responsible for the majority of child deaths in Latin
America. Research demonstrates that most ill chi1dren suffer from more than one
condition and the integrated approach to their diagnosis and treatment ensures that all
their symptoms receive treatment while combining therapy is some cases, therefore
providing optimal care. More recently the IMCI approach has evolved into three
components: improvements in the case management skills of health staff through the
provision of locally adapted guidelines on integrated management of childhood illness
and activities to promote their use; improvements in the health system required for
effective management of childhood illness; and improvements in family and community
practices.
In response to the child survival challenge in Latin America and the Caribbean, staff from
USAID, the Pan American Health Organization (PAHO) and the Partnership for Child
Health Care Inc., the Cooperating Agency which implements the Basic Support for
Institutionalizing Child Survival Project (BASICS) collaborated on thc design of a five
year LAC Regional IMCI strategy. This strategy was planned to improve child health
services thereby reducing, the impact of the main causes of child mortality and morbidity
in the LAC region.
The LAC Initiative focuses on this integrated approach primarily to reduce child
morbidity and mortality but also to promote greater efficiency. IMCI promotes an
integrated approach to sick childcare by advocating an integrated assessment of the child.
ANNEXES
83
Inappropriate management of childhood illnesses wastes scarce resources. Although
increased investment will be needed initially, eventually additional resources will not be
required and the IMCI strategy will result in cost savings. During a time of
decentralization and sector reform in health throughout the region, this integrated
approach seems to work well.
The IMCI Initiative focuses on eight countries in the region where infant and under-five
mortality rates remain high, among the countries in LAC where USAID, has PHN
programs. Target countries of the IMCI results package include; Bolivia, Ecuador, El
Salvador, Guatemala, Haiti, Honduras, Nicaragua, and Peru.
The five year Initiative was authorized on March 5, 1997 for a LOP of $5 million. The
planned allocation of funds was PAHO receiving approximately $2,810,281 (plus up to
an additional $2,000,000 in potential add-ons from USAID country missions for related
activities) and the BASICS I and II projects $2,187,190.
Current plans are to increase the LOP to 55,722,000; no allocation between PAHO and
BASICS has been determined.
The IMCI Initiative activities are grouped into phases: Design, Implementation, and
Monitoring and Evaluation. Each target country will begin participation at various points
within the framework depending on the commitment and capacity of the country to
implement the IMCI strategy.
As a regional activity, the intent is that all activities be regional or sub-regional in scope.
Funds allocated for regional activities in this proposal were not intended to be used for
country-specific IMCI activities, in order not to impinge on mission prerogatives to fund
and design country-specific activities
Implementing Roles
Because of its permanent intergovernmental nature in the region, PAHO assumes the
overall coordinating role for the LAC Regional IMCI Initiative. PAHO and BASICS
share implementation responsibility according to each partner's technical and
organizational strengths. A lead agency has been agreed on by PAHO and BASICS for
each proposed activity or group of activities. PAHO/BASICS regional activities are
targeted at strengthening country-level IMCI capacity.
Results and Performance Indicators
The LAC Regional Strategic Objective of which IMCI is a component is More Effective
Delivery of selected Health Services and Policy. In this case the selected health
intervention is IMCI and the Strategic Objective for the IMCI Initiative is More Effective
Delivery of Child Health Services in Response to CDD, ARI, and Malnutrition.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
84
Thc Indicators to be used to measure progress in achieving the Strategic Objective for
IMCI are the number of target countries—
· with 90% percent of Facilities in pilot districts delivering IMCI services; and
· with 10% of facilities delivering IMCI services.
Intermediate Results
IR 1: Country Health Leaders Have Information for IMCI Adoption;
IR 2: Country Plans and Strategies in Place for Introduction and Implementation of
IMCI;
IR 3: Improved Country Capacity to Implement IMCI; and
IR 4: Monitoring and Evaluation Used to Adjust IMCI Program Plans.
Evaluation Objectives:
1. To determine if the results framework is still valid and feasible and to recommend
adjustments to the results framework, design, and activities as appropriate.
2. To examine progress toward achieving the SO and IRs as planned, evaluating whether
the results/intermediate results will be met in a timely and effective manner. To
identify specific internal/external constraints which may limit their accomplishments
or success and to recommend adjustments based on findings and conclusions
3. To assess how the initiative structure and design as configured is working including
both the specific technical areas and activities and the delegation of responsibilities to
the two partners as well as the working relationships and collaboration in the
implementation of initiative activities. The team should recommend adjustments to
implementation based on findings and conclusions
4. To assess the administration of the initiative by USAID and the two implementing
partners including the status of coordination and communication within the
organizations specifically between headquarters and field operations and host country
governments and with USAID Washington and field missions.
Key questions to be answered:
The evaluation team should answer the following questions and make recommendations
as to how USAID can improve and or strengthen the activity to increase the likelihood of
achieving objectives/results as defined in these questions. Any changes to the questions
listed below will be made within 2 days of signature of the amendment to the Task Order.
ANNEXES
85
Results Framework
Does the Initiative's design and results framework (intermediate results or indicators)
need to be changed or adjusted in any way? What recommendations does the evaluation
team have about changes.
Are the human and material resources available appropriate to the size of the task? Are
there too many target countries for adequate implementation? Are there too few target
countries for regional impact? What is the most strategic use of the available resources to
further the goal of reducing infant and child mortality through IMCI?
As a regional activity, the intent is that all activities be regional or sub-regional in scope.
Country implementation is not included, in order not to impinge on mission prerogatives
to fund and design country-specific activities. What has actually occurred and does the
Initiative need to be modified in any way as a result?
Progress and Achievements
Has the initiative made good progress toward achievement of the IRs and results by
successfully accomplishing specific activities as described in key project documents?
What progress to date has been made in influencing countries to adopt IMCI?
Have strategic plans been developed for beginning IMCI introduction and later
implementation in countries? What is the quality of these plans and are they owned by
host country governments?
Are there alternative models of IMCI introduction? How have these worked and what is
the status in each country?
What is the status of implementation at the country level? Is there evidence that country
training implementation has occurred where TOTs have been conducted? How has the
quality of second and third generation training been effected? Have there been training
impact follow-up studies? If so, how have the findings and results of these been used to
modify training plans and other factors?
Have systems issues been addressed before training scale-up has occurred as articulated
by WHO? What is the status of systems strengthening in each of the emphasis countries
with respect to drug supply and to supervision? How has systems strengthening been
addressed in each of the emphasis countries?
What is the value-added of this Initiative? What will it add to the world experience thus
far in IMCI implementation?
Have any innovate approaches or tools been developed that have been adopted by others
for use?
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
86
To date what is the status of monitoring and evaluation as a tool to improve
implementation of IMCI by the country programs?
Initiative Structure
What are the strengths and weaknesses of the management structure as implemented?
How responsive have the partners been to each other during the course of Initiative
implementation?
To what extent are related structures functioning adequately? Have the IMCI national
working groups been established in each country? Has the work of these groups moved
the initiative forward in its achievement of the IRs?
Administration
As this project was planned to supplement county implementation through regional and
sub-regional activities close collaboration with the USAID/LAC and Global Bureaus as
well as field Missions was deemed essential in the Initiative's design. Has this
collaboration with USAID at all levels occurred and if so has it been successful? Has the
project leveraged additional resources for IMCI through this coordination?
How effective has the management and oversight of Initiative been by USAID/LAC? By
G/ PHN/HN?
Is there good communication between headquarters and the field staff for both partner
organizations? Is there good communication between partner staff and host country
officials?
Have reports, work-plans, and other products been of adequate quality and submitted to
USAID on time with appropriate levels of information?
Use of Resources
Have the project resources been used appropriately and effectively e.g. staff and other
inputs? Has all planned staff been hired and have staff and job descriptions been
approved by USAID as called for in the LAC/W grant to PAHO? In the task order/core
contract with BASICS?
Materials and Procedures
1. Data Sources
The IMCI Initiative evaluation team will review relevant documentation including but
not limited to the following: The IMCI Results package, Authorization, Grant to PAHO,
R4s, Indicators paper, Memorandum of Understanding, Trip Reports, Six-month and
Annual reports, Quarterly Financial Reports, Committee Reports (IMCI Working
ANNEXES
87
Groups), IMCI Country-level Implementation plans, Training plans, Health Facilities
Survey, Reports on Application of Health Facilities Survey Tool, Bolivia Training
evaluation, Drug Assessment Tool, Reports of Donor Assessment tool application, and
other project documents and publications. Interviews will be conducted with staff from
USAID/LAC, USAID/G, PAHO, BASICS II, USAID field Missions, PAHO field
Offices, PVOs involved with the initiative and the MOH
2. Methods of Data Collection
This will be primarily a process evaluation. The data will be col1ected through document
review, key informant interviews, e-mail surveys, telephone interviews, field site visits
and group discussions, The team will travel to three countries in the LAC region and
interview field staff, MOH counterparts, USAID Mission and others involved with IMCI
implementation.
3. Duration and Timing of the Evaluation and LOE
The IMCI Initiative evaluation will begin in November 30, 1999 or earlier if possible.
Consultants should be approved to work Saturdays. The approximate time line envisioned
is as follows:
Background documents distributed
Team Planning Meeting and Washington briefings
(Interviews USAID, PAHO, BASICS)
Team travels to Country 1
Team travels to Country 2
Team travels to Country 3
Washington, DC follow-up discussions, framing principal findings, and
recommendations Brief USAID on principal findings and recommendations
Brief PAHO on principal findings and recommendations
Brief BASICS II on principal findings and recommendations
Draft report to USAID
USAID and partners comments on draft
Final report distributed
Team Composition
The evaluation team will consist of two members. The skills and experience listed below
are necessary but may be fulfilled in a different team configuration than listed below:
1. A child survival specialist/Team Leader with knowledge of IMCI, policy,
management systems, and strategic planning of child health programs in public sector
institutions with, 10 years experience in design, implementation, and evaluation of
USAID health programs. The person should have both field experience in Latin
America and have an excellent knowledge of USAID/PHN strategic objectives.
Experience with UN organizations and WHO or PAHO would also be helpful The
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
88
candidate should be fluent in Spanish (F.S. 3+-4) ant have advanced writing skills in
English.
2. A clinical and community training specialist with experience in national health
training systems and evaluation in Latin America that is familiar with quality
assurance and in fluent in Spanish (F.S.3+ -4) experience with USAID and UN,
WHO or PAHO would also be helpful.
Funding and Logica1 Support
All funding and logistical support for the assessment will be provided through this SO
using a TASK Order to the MEDS project (Monitoring, Evaluation, and Design Services
activity). Activities that will be covered include: recruitment of the team members,
distribution of background documentation provided by USAID, arranging for travel and
logistical support; secretarial and office assistance as needed, support for all personnel
and business expenses related to the evaluation, assistance as required with editing,
printing, and distributing the final report.
ANNEX F
EVALUATION SCHEDULE
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
90
WASHINGTON, D.C.
April 3–4 MEDS Office Team Planning Meeting
April 5–7 Various Sites Meetings (or telephone interviews)
with MEDS and U.S.–based
stakeholders: USAID (LAC, Global
Bureau, Jamaica, former staff),
PAHO, BASICS, RPM
April 8 Travel to Honduras
HONDURAS
April 9 Tegucigalpa Document review and individual
work
April 10 Tegucigalpa Meetings with Dr. Miguel Dávila,
PAHO/Honduras; Barry Smith,
American Red Cross;
CARE/Honduras, Proyecto Hogasa
staff; Dr. Carlos Samayoa, PAHO
Representative
April 11 Field Visit Health Region 2–Director at MOH
Regional Hospital, Comayagua;
IMCI facilitator at MOH Area
Hospital, La Paz; director and nurses
at MOH Health Centers, Taulabé and
Siguatepeque
April 12 Tegucigalpa Meetings with Minister of Health;
MOH IMCI team; Dr. Carlos
Villabos, USAID/Honduras; Vicky
Alvarado, BASICS; Dr. John
Rogosch and Dr. Pinto,
USAID/Honduras
April 13 Field Visit Health Region 4–Choluteca;
Director and teaching staff, CEDAR
Nursing School, Choluteca; Director
of MOH Health Centers,
Comayagua; Sra. Amalia Sierra,
Health Promoter, Pueblo Nueva
Unión
April 14 Tegucigalpa Development of notes from week in
Honduras
April 15 Travel to the United States
ANNEXES
91
UNITED STATES
April 17–28 New York Telephone interviews with U.S.–
based stakeholders: PAHO, World
Bank, BASICS, USAID/LAC,
USAID/Honduras, CORE group
April 19 New York Meeting with Vincent Orinda,
UNICEF
BOLIVIA
April 30 Travel to Bolivia
May 1 Rest
May 2 Meetings with Dr. Jaime Tellería and
MOH IMCI team; Dr. Oscar
Gonzalez, MOH; Dra. Ana Maria
Aguilar and Dr. Dilberth Cordero,
BASICS; Dra. Marta Mejía, PAHO;
Dr. Fernando Lavadenz and Lic. Ma.
Luisa Salinas, Unidad de Reforma de
Salud; Dra. Remy Zumarán, IMCI
Facilitator
May 3 Field Visit Aroma District; directors and nurses
at MOH Health Centers, Calamarca
and Patacamaya; director and
supervisor at MOH District Office
May 4 La Paz Meetings with USAID/Bolivia; Dr.
José Ignacio Carreño, PROCOSI; Dr.
Andrés Bartos and staff, COTALMA
May 5 Telephone interview with Dra.
Lillian Braun, Santa Cruz; meetings
with Lic. Magaly de Yaly, UNICEF
May 6 Travel to Peru
PERU
May 7 Lima Development of notes from week in
Bolivia
May 8 Lima Meetings with Dr. Luis Seminario,
USAID/Peru; Dr. Miguel Dávila,
PAHO; MOH National IMCI
Coordinating Commission
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
92
May 9 Field Visit Chiclayo: director and IMCI staff of
MOH Regional Hospital, Chiclayo;
directors and nurses at MOH Health
Centers–El Bosque, Atusparia,
Cerropan, Fernando Ortiz; mothers
and health promoters at Health
Center Fernando Ortiz
May 10 Field Visit Piura: Director and IMCI staff of
MOH Regional Hospital; director,
nurses, mothers, and promoters at
MOH Health Center Cura Mori
May 11 Field Visit Sullana: Director and IMCI staff of
MOH Regional Hospital; director
and nurses at MOH Health Centers,
Tambo Grande and Querecotillo;
promoters at Locuto
May 12 Lima Debriefing with MOH, PAHO,
USAID/Honduras, UNICEF;
development of notes from week in
Peru
May 13–14 Travel to the United States
ANNEX G
DOCUMENTS REVIEWED
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
94
Basic Support for Institutionalizing Child Survival Project (BASICS). 1999 Report to the
USAID Bureau for Latin America and the Caribbean for the Reporting Period of January
1–December 31, 1999.
______. Tools Expo Catalog. A Summary of Tools for Improving Child Health.
(undated)
______. LAC IMCI Initiative. Regional Achievement Summary. June 16, 1999.
______. FY1997 Midyear Program Review—LAC Regional IMCI Initiative. June 12,
1997.
BASICS and CORE Group. Reaching Communities for Child Health: Partnering with
PVOs in Integrated Management of Childhood Illness (IMCI). Proceedings of a
workshop held at the Pan American Health Organization, Washington, D.C., February
24–26, 1999.
“IMCI pilot schemes successful in Kazakhstan,” The Lancet, Vol. 355, April 1, 2000.
Integrated Management of Childhood Illness (IMCI) in Countries of Latin America.
Report of the meeting. Quito, Ecuador, January 27–29, 1998.
Lambrechts, T. and Salgado, R. Meeting Report of the BASICS II and WHO/CAH
Coordination Meeting in Arlington, VA, January 24, 2000.
Letter from USAID to PAHO approving the grant, Integrated Management of Childhood
Illness Component. May 20, 1997.
“Management of childhood illness in Africa,” BMJ, Vol. 320 :594-5, 2000.
Ministerio de Salud y Previsión Social. Evaluación a Servicios de Salud en la Atención
Integrada a las Enfermedades Prevalentes de la Infancia (AIEPI). Primera Prueba
Mundial, Bolivia, 12–30 de Abril de 1999.
______. Una Evaluación con Base en Indicadores. Evaluación de la Disponibilidad y
Uso de Medicamentos para la Estrategia AIEPI. Línea de Base. Bolivia. Mayo–Julio,
1998.
______. Atención Integrada a las Enfermedades Prevalentes de la Infancia (AIEPI).
Plan Nacional Bolivia, 1998. Año II de la Implementación, Fase de Expansión.
Our Joint Vision: Results of a Joint LAC/PAHO/BASICS Strategic Planning Workshop.
November 1998.
ANNEXES
95
Pan American Health Organization. PAHO/WHO’s IMCI Technical Unit Report for the
Project LAC Regional Results Package: Integrated Management of Childhood Illness,
1997–2001. January–December 1997.
Pan American Health Organization and Basic Support for Institutionalizing Child
Survival. LAC Regional Results Package Integrated Management of Childhood Illness
(IMCI) 1997–2001. A proposal to the Bureau for Latin America and the Caribbean,
United States Agency for International Development. January 7, 1997.
______. Memorandum of Cooperation. 1997.
______. LAC Integrated Management of Childhood Illness Results Package. Semi￾Annual Joint Progress Report to the USAID Bureau for Latin America and the Caribbean
for the Reporting Period of January 1–July 30, 1998. (undated)
______. LAC Integrated Management of Childhood Illness Results Package. 1998
Report to the USAID Bureau for Latin America and the Caribbean for the Reporting
Period of January 1–December 31, 1998. December 6, 1999.
______. LAC Integrated Management of Childhood Illness Results Package. Semi￾Annual Joint Progress Report to the USAID Bureau for Latin America and the Caribbean
for the Reporting Period of January 1–July 30, 1999. (undated)
______. LAC Integrated Management of Childhood Illness Results Package. 1999
Report to the USAID Bureau for Latin America and the Caribbean for the Reporting
Period of January 1–December 31, 1999.
______. USAID/LAC Regional PAHO/BASICS IMCI Project Workplan and Budget,
October 1, 1998 to December 31, 1999. May 5, 1999.
______. USAID/LAC Regional PAHO/BASICS IMCI Project Workplan and Budget,
October 1, 1998 to December 31, 2000. December 1, 1999.
______. LAC Regional IMCI Initiative. Strategic Objective Performance and
Cumulative Indicators.
United States Agency for International Development. Results Review R2a. LAC
Regional Program. FY 1999–2001. (undated)
______. Results Review R2a. LAC Regional Program. FY 2000–2002. March 22,
2000.
World Health Organization. IMCI Information Kit. Geneva, September 1997.
______. Towards a Global Evaluation of the IMCI Strategy. 1998.
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
96
ANNEX H
INTERVIEW LIST
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
98
Washington, D.C.
USAID
Carol Dabbs Team Leader, LAC Regional Health Priorities
Strategic
Marguerite Farrell Senior Technical Adviser, LAC/RSD-PHN
Ellyn Ogden CTO, Polio Eradication and Immunization Support
Sheila Lutjens Jamaica USAID Representative
Al Bartlett CTO of BASICS Project
Karen Cavanaugh Health Systems Adviser, LAC/RSD-PHN
Melody Trott Former USAID Technical Officer
PAHO
Dr. Gabriel Schmuni HCT Program Coordinator
Dr. Yehuda Benguigui Regional Advisor IMCI
Christopher Drasbek Regional Technical Adviser IMCI
Dr. José Antonio Solís HPP Director
Management Sciences for Health
Douglas Keene Deputy Director, RPM Project
BASICS
David McCarthy Technical Officer
Richard Nelson LAC Operations Officer
Kim Cervantes Technical Officer
Dr. Rene Salgado Technical Officer
World Bank
Dr. Judy McGuire Senior Nutritionist, LAC Region
Dr. Mariam Claeson Principal Public Health Specialist
New York
Dr. Vincent Orinda Senior Adviser, Child Health
WHO Geneva
Dr. Thierry Lambrechts Medical Officer, Monitoring and Evaluation
ANNEXES
99
HONDURAS
USAID
John Rogosh Director, Human Resources Development Office
Dr. Ernesto Pinto Foreign Service National
Meri Sinitt HPN Officer
Secretary of Health
Dr. Plutarco Castellanos Secretary of Health
Dr. Jorge Humberto Meléndez Head, Maternal and Child Health
Dra. María Elena Guevara Officer in charge of IMCI activities
Leticia Castillo Collaborates in IMCI activities
Region 2
Dr. Arturo Gutierrez Regional Deputy Director
Hospital Roberto Suazo Cordoba
Dr. Dignora Lizano Pediatrician working with IMCI
Centro de Salud Taulabé, Comayagua
Dr. María Isabel Degrand Director
Region 4, Escuela de Enfermería CERARH-Sur
Lic. Yolanda Echenique Director
Lic. Ana Ester Vivas Teacher
Centro de Salud La Providencia, Choluteca
Dra. Ana Silvia Muril Director
Pueblo Nueva Unión
Amalia Sierra Health Promoter
PAHO
Dr. Carlos Samayoa Country Representative
Dr. Miguel Davila Officer in charge of IMCI activities in the country
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
100
BASICS
Lic. Victoria Alvarado Country Representative
American Red Cross
Barry Smith Country Representative
CARE
Carmen Aleida Hernandez Manager, Project Hogasa
Francisco Almendares Deputy Manager, Project Hogasa
BOLIVIA
USAID
Susan Brems Chief, Health Section
Dr. Charles Oliver Deputy, Health Section
Dr. Oscar Gonzalez Former Manager, CCH Health Project
Ministry of Health
Dr. Jaime Tellerí Jefe, Unidad Nacional de Atención a las Personas
Dr. Fernando Lavadenz General Manager, Health Reform Unit
Dr. Remy Zubaran Pediatrician, Children’s Hospital
Dr. Lilian Brunn Coordinator, Training Center
Children’s Hospital, Santa Cruz
Dr. Andrés Bartos Director, COTALMA
Dr. Roxana Sauneno Pediatrician
Lic. Fatima Mariscal Nutritionist
Lic. Luz de la Fuente Nurse
Lic. Juana Siñani Nurse
PAHO
Dr. José Antonio Pagés Country Representative
Dr. Roberto Kriskovich Health Promoter
Dr. Martha Mejía Consultant, Child and Perinatal Health
BASICS
Dr. Ana María Aguilar Liendo Country Representative
Dr. Dilberth Cordero Consultant
ANNEXES
101
Dr. Alberto Tenorio Consultant
Dr. Adalid Zamora Consultant
UNICEF
Magali de Yaly Nutrition Officer
Centro de Salud Calamarca, distrito Aroma
Dra. Dunia Arakaki Director
Felipa Calle Auxiliary Nurse
Centro de Salud de Patacamaya
Dr. Mónica Reyes Director
Maruja López Auxiliary Nurse
Maxima Zamora Auxiliary Nurse
Sonia Laime Auxiliary Nurse
Distrito Aroma, Department of La Paz
Dr. Mireya Elías Director
Lic. Estela Vela Health Services Supervisor
PROCOSI
Dr. José Ignacio Carreño Director
PERU
USAID
Dr. Luis Seminario Director, Project 2000
Ministry of Health
Dr. Jesus Toledo Tipo Director, People’s Health Unit
Dr. Fernando Cerna Director, PAI and MCH Divisions
Chiclayo
Dr. Victor Linares Baca Regional Director, Chiclayo
Dr. Luis Desa Director, People’s Health Unit
Llully Chumacero Nurse, Clinical IMCI
Mirta Alvarado Nurse, Community IMCI
Fredesbinda Diaz Nurse, El Bosque Health Center
MIDTERM EVALUATION OF THE LATIN AMERICA AND THE CARIBBEAN
INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS INITIATIVE
102
Dr. Victor Gonzalez Director, Atusparia Health Center
Dr. Jorge Ortiz Diaz Director, Cerropon Health Center
Dr. Jorge Postigo IMCI Facilitator
Dr. William Guerrero Director, Belen Hospital
Piura
Dr. Jorge Calderon Marti General Director
Natalie Mendoza Faro Nurse in charge of IMCI
Violeta Romero Nurse in charge of PAI
Dr. Max Rios Director, Cura Mori’s Health Center
Plan International
Maria Espinoza Montenegro Manager
Lilian Cabrera Villar Coordinator
Javier Riofrio Coordinator
Sullama
Gloria Ordoñez Nurse
Dr. Ehunice Vilchez Paredes Director, Tambo Grande Health Center
Jose Prieto Rivas School Volunteer
Estela Arroyo Health Promoter
Juana Calle Health Promoter
Dr. Carlos Rivas Jaramil Director, Querecotillo Health Center
Ruth Villena Nurse
PAHO
Dr. Sonia Tavares Acting Country Representative
Dr. Miguel Dávila Dávila Officer in charge of IMCI Implementation
UNICEF
Dr. Mario Tavera Salazar Health Technical Officer