CARE-MOZAMBIQUE
Final Evaluation
CHILD SURVIVAL XII
IMPROVING MATERNAL AND CHILD HEALTH
IN RURAL COMMUNITIES OF NAMPULA PROVINCE
September 30, 1996 to September 30, 2000
FAO-A-00-96-90040-00
October 2000
Prepared by:
Sunny Wijesinghe, MPH, RN
External Consultant
CARE USA Contact:
Judiann McNulty, Deputy Director
CARE USA
Atlanta, GA 30303
CARE – MOZAMBIQUE Final Evaluation Page i
TABLE OF CONTENTS
ACRONYMS
A. SUMMARY.............................................................................................................................. 1
B. PROJECT RESULTS AND IMPACT .................................................................................. 4
1. Summary Chart.................................................................................................................... 4
2. Technical Approach ............................................................................................................ 5
3. Cross-cutting Approaches ................................................................................................. 13
a. Community Mobilization ............................................................................................ 13
b. Communications for Behavior Change ....................................................................... 15
c. Capacity Building Approach ....................................................................................... 17
d. Sustainability Strategy................................................................................................. 19
C. PROJECT MANAGEMENT ............................................................................................... 21
1. Planning............................................................................................................................. 21
2. Staff Training .................................................................................................................... 21
3. Supervision of Project Staff .............................................................................................. 23
4. Human Resources and Staff Management ........................................................................ 23
5. Financial Management ...................................................................................................... 24
6. Logistics ............................................................................................................................ 24
7. Information Management .................................................................................................. 25
8. Technical and Administrative Support.............................................................................. 26
9. Management Lessons Learned .......................................................................................... 27
D. OTHER ISSUES.................................................................................................................... 29
E. CONCLUSIONS AND RECOMMENDATIONS.............................................................. 29
F. RESULTS HIGHLIGHT...................................................................................................... 33
G. ANNEXES
A. Maps
B. Activities in Intervention Areas
C. Formative Research and Special Studies
D. IEC Strategy
CARE – MOZAMBIQUE Final Evaluation Page ii
E. List of Material and Equipment
G. List of Training Activities
H. Results Highlight
I. Evaluation Team Members
J. Assessment Methodology and Schedule
K. Questionnaires used in the Evaluation
H. REPORT OF THE FINAL SURVEY
CARE – MOZAMBIQUE Final Evaluation Page iii
ACRONYMS & DEFINITIONS
ACD Assistant Country Director
ACNM American College of Nurse Midwives
APE Agente Polivalente Elementar - a community health worker trained by the MOH 
to provide basic preventive and curative health services
ARI acute respiratory infection
CC Community Health Council
CS Child Survival
DDS district-level MOH
DIP Detailed Implementation Plan
DPS provincial-level MOH
EPI Expanded Programme of Immunization
HIS health information system
HQ headquarters
HPU Health & Population Unit
IEC Information, Education and Communication
IMCI Integrated Management of Childhood Illnesses
JSI John Snow International
KPC knowledge, practice and coverage
MCH maternal child health
MIS management information system
MOH Ministry of Health
MTE Mid-Term Evaluation
NGO non-governmental organization
ORS oral rehydration solution
ORT oral rehydration therapy
PHC primary health care
PLA Participatory Learning Assessment
PSI Population Services International
RTA Regional Technical Adivsor
TBA traditional birth attendant
TOT training of trainers
USAID United States Agency for International Development
VIDA CARE Mozambique’s agriculture project
WHO World Health Organization
1 CARE Mozambique Final Evaluation
A. SUMMARY
A.1.a) Project overview
The goals of the four-year Child Survival-XII project were to reduce maternal, infant and child
morbidity and mortality from selected preventable and treatable diseases, and to improve selected
practices of caretakers, community health providers, and Ministry of Health personnel in the
districts of Malema, Mecuburi and Ribaue, in Nampula province (see map, Annex A).
A.1.b) Project objectives
1) Increase the number of complicated obstetric cases managed in the health care system to 10%
of all cases.
2) Increase from 32% to 50% women who give birth with a qualified attendant.
3) Increase from 41% to 60%, the proportion of women who receive presumptive treatment of
anaemia during prenatal care.
4) Increase from 40% to 60% the proportion of caretakers who correctly manage their child’s
diarrhea episode with the usual or increased amounts of food, fluids and breast-milk.
5) Eighty percent of all cases of pneumonia for 0-5 children seen in the health posts will have
correct classification, diagnosis and treatment.
6) Eighty percent of all uncomplicated cases of malaria for children 0-5 and pregnant women
seen in both health posts and in selected communities will have correct treatment of malaria,
according to national protocol.
7) Eighty percent of families in three communities participate in the bed-net pilot project.
The project strategy developed along a two-pronged approach. The first involved training and
supervision activities designed to improve the quality of service delivery within the formal health
care system. This involved work with health center and rural health post staff as well as adjunct
non-formal personnel (community health workers and traditional birth attendants). The second
involved empowering communities and households to assess their own overall health status,
evaluate their capacity to respond to health problems and acquire resources to address those
problems. Both approaches were based on initial qualitative and quantitative research.
A.2) Project achievements
By the end of September 2000, the following achievements had resulted from the project:
• Ninety-one TBAs trained and equipped with birthing kits;
• Sixty-seven MOH staff and 46 APEs trained in sequential management of childhood illnesses
• Fifty-seven APEs received refresher courses (refer to Annex F) and equipment (Annex E);
• Establishment of a supervision system to monitor technical competencies of health facility
personnel as well as APEs and TBAs;
• Structural and material support to rehabilitate the health centers and posts as well as providing
basic obstetric equipment;
• Fifty-one community councils (637 individuals) trained and active in delivering key health
messages employing a comprehensive IEC strategy;
2 CARE Mozambique Final Evaluation
• Establishment of a community referral system to identify early danger signs of childhood
illness and pregnancy;
• Promotion and sales of mosquito nets;
• Development of an emergency transport system including SOS alert flags, bicycle ambulances
and plans for medical emergencies; and
• Organization of two mobile theater groups to transmit messages.
A.3) Highlights of the KPC results
A quantitative Knowledge, Practice and Coverage (KPC) survey took place in August 2000. The
resulting data, when compared to the baseline survey data collected in February 1997 showed that
project had achieved positive behavior changes in all its intervention areas:
A.3.a) Diarrhea case management
More mothers in the project communities gave more breast milk, more liquid and more solid food
when their children had a bout of diarrhea, and more mothers could recognize danger signs of
dehydration.
A.3.b) Case management of malaria
 More mothers knew that a mosquito net could prevent malaria, and despite the cost, more
families owned a mosquito net. An even larger percentage of families planned to purchase nets
when they obtained sufficient funds.
A.3.c) Case management of acute respiratory infections (ARI)
More mothers in project communities could recognize danger signs of ARI, and an increased
percentage of mothers sought medical help when their children had these symptoms.
A.3.d) Maternal health
More mothers sought prenatal care and possessed a maternal health card. Increases were seen in
the numbers of mothers who had received two tetanus shots, the percentage of pregnant women
receiving iron supplements, and the number of births that were attended by a trained attendant
(TBA, nurse-midwife). In addition, more pregnant women in project communities reported
having a birth plan and having purchased a new razor blade for severing the umbilical cord. Some
had even managed to put money aside in case of a birth-related emergency.
The final evaluation (September 2000) concluded that:
• The CS-XII project made major strides toward meeting its objectives, despite a lag in
community-level achievement during the first two years. An opportunity to continue project’s
accelerated efforts could bring greater benefits to project communities.
• A variety of factors, related to health infrastructure, the socio-economic and political
environment in Mozambique, the level of MOH cooperation and difficulties at the project
management level, influenced the attainment of project objectives. Given more time, the
project could have overcome more of these challenges. Sustainability of behavior change
improvements depends largely on the continued application of knowledge and skills acquired
by the health personnel and community members. The cooperation of the MOH could greatly
enhance efforts to sustain the project’s benefits by providing reinforcement.
3 CARE Mozambique Final Evaluation
B. PROJECT RESULTS AND IMPACT
B.1) Summary Chart
Baseline
KPC
(1997)
Final KPC 2000
Project
communities
Other
communities
Objectives Corresponding KPC
Indicators
N % N % N %
Mothers who had 2 or more doses of TT NA NA 50 20.1 23 9.2
Mothers who sought pre-natal care (one or
more times) during pregnancy
161 50.9 204 81.6 169 67.6
Mothers who received iron supplements
during pregnancy
181 56.9 168 67.2 121 48.4
Mothers who had a birth plan NA NA 122 48.8 73 29.2
Mothers who saved money for a birth-related
emergency
NA NA 88 35.2 43 17.2
Births attended by trained personnel 102 32.1 106 42.6 84 33.9
Maternal Health Care
1) Increase the number of complicated
obstetric cases managed in the health care
system to 10% of all cases, by Sep 2000*
2) Increase from 32% to 50% women who
give birth with a qualified attendant, by Sep
2000
3) Increase from 41% to 60%, the proportion
of women who receive presumptive
treatment of anemia during prenatal care by
Sep 2000 Mothers who had a new razor-blade for
severing the umbilical cord
NA NA 91 36.5 37 14.8
Prevalence of ARI during the last 2 weeks 96 29.6 39 15.6 54 21.6
Mothers who sought medical help in 1./2./3.
place when child had ARI during the last
two weeks
23
4
0
42.6
7.4
0.0
11
6
7
47.8
26.1
30.4
17
8
3
45.9
21.6
8.1
Mothers/c child with ARI treated with
antibiotics during the last two weeks
29 30.9 8 21.6 8 14.3
Mothers who know rapid respiration as a
danger sign for pneumonia
168 51.9 205 82.0 163 65.2
Management of ARI
80% of all cases of pneumonia for 0-5 year
old children seen in the health posts will
have correct classification, diagnosis, and
treatment by Sep 2000*
Mothers who did not know any signs 49 15.1 17 6.8 37 14.8
Incidence of malaria during the last 2 weeks 219 67.6 63 25.2 77 30.8
Mothers with child with malaria who sought
medical help in 1./2./3. place during the last
two weeks
165
43
0
50.9
13.3
0.0
23
21
19
36.5
33.3
30.2
21
27
14
27.3
35.1
18.2
Families who owned a mosquito net NA NA 25 10.0 9 3.6
Families who knew a mosquito net could
prevent malaria
3 6.5 149 59.6 44 17.6
Malaria
1) 80% of all uncomplicated cases of malaria
for children 0-5 and pregnant women seen in
both health posts and in selected
communities will have correct treatment,
according to the national protocol. *
2) 80% of families in three communities
participate in malaria bed net pilot project,
by Sep 2000 Families who planned to buy a mosquito net NA NA 159 71.0 115 47.9
Prevalence of diarrhea during last two weeks 207 63.9 66 26.4 93 37.2
Mothers who breastfed more during child’s
diarrhea during the last two weeks
42 20.3 23 35.4 13 14.0
Mothers who gave more liquids during
child’s diarrhea during the last two weeks
40 19.3 16 29.1 12 17.4
Mothers who gave more solid food during
child’s diarrhea during the last two weeks
33 15.9 13 23.6 7 10.1
Mothers who gave home-made ORS during
child’s diarrhea during the last two weeks
11 7.4 10 18.2 4 5.8
Control of diarrheal diseases
Increase from 40% to 60% the proportion of
caretakers who correctly manage their
child’s diarrhea episode with usual or
increased amounts of food, fluids and breast
milk, by Sep 2000
Mothers who gave ORS from packet during
child’s diarrhea during the last two weeks
59 39.9 23 41.8 26 37.7
NA = not available
* Due to loss of loss of project database just prior to the final evaluation, these objectives could not be measured
4 CARE Mozambique Final Evaluation
B.2) Results: Technical approach
B.2.a) Project Approach
The Child Survival-XII project was one of two health projects for CARE Mozambique. The
complementary Reproductive Health project worked in the same districts with a different
approach. CS-XII provided technical assistance, and material, logistical and human resources to
the MOH at the Provincial and District levels, contributing to existing government programs
targeting health problems that affect women and children 0-5 years.
Project Intermediate Objectives
1) Increase the number of complicated obstetric cases managed in the health care system to 10%
of all cases.
2) Increase from 32% to 50% women who give birth with a qualified attendant.
3) Increase from 41% to 60%, the proportion of women who receive presumptive treatment of
anaemia during prenatal care.
4) Increase from 40% to 60% the proportion of caretakers who correctly manage their child’s
diarrhea episode with the usual or increased amounts of food, fluids and breast-milk.
5) Eighty percent of all cases of pneumonia for 0-5 children seen in the health posts will have
correct classification, diagnosis and treatment.
6) Eighty percent of all uncomplicated cases of malaria for children 0-5 and pregnant women
seen in both health posts and in selected communities will have correct treatment of malaria,
according to national protocol.
7) Eighty percent of families in three communities participate in the bed net pilot project.
Location
CS-XII operated in Nampula Province in northern Mozambique. The project was active in the
districts of Mecuburi, Ribaue and Malema, reaching a total of 51 communities and 26 health
facilities with their entire catchment areas.
Intervention Mix
The project interventions employed two general strategies. The first was to assist the MOH to
provide quality Child Survival services. The second was to develop community and household
level capacity to access and use health care services and to take preventive measures. These
strategies were applied to the following interventions:
Case Management of Diarrhea
1) Household level recognition of danger signs and timely referral
2) Increased home care with usual or more fluids, breast feeding, and solid food
3) Improved diagnosis and treatment at health facilities
At baseline, 40% of children in the three Western districts of Nampula were given ORT, 30%
were breast-fed less and 44% were given less food during a diarrhea episode. These figures,
combined with an extremely low recognition of danger signs, led CARE to focus its community
education project for diarrhea on all aspects of home management.
5 CARE Mozambique Final Evaluation
Case Management of Malaria
1) Household level recognition of danger signs and timely referral
2) Improved diagnosis and treatment at health facilities
3) Prevention of malaria with bed nets
Project messages identified the fundamental danger signs for malaria and the importance of
timely treatment and referral. Training projects improved the skills of health personnel to
diagnose and treat malaria in pregnant women and children 0-5 years. A pilot bed net project was
introduced in each district, promoting the sale and use of bed nets and private sector
commercialization at the local level.
Case Management of Acute Respiratory Infections
1) Household level recognition of danger signs and timely referral
2) Increased access to first line antibiotics via improved stock management
3) Improved diagnosis and treatment at health facilities
Due to the rapid evolution of respiratory illness into serious infection, project activities for ARI
promoted prompt care-seeking and recognition of danger signs. Increased access to first-line
antibiotics was facilitated by collaborating with the Department of Logistics and Stock
Management at the DPS for training in stock management and distribution at the district level and
down. District level personnel received a general course in diagnosis and treatment of ARI, as
well as a refresher course coupled with the training in stock management.
Maternal Health
1) Increased number of births attended by a trained traditional midwife or health worker
2) Training for presumptive treatment of anemia in pregnant women
3) Household and health facility-level recognition of danger signs of pregnancy and childbirth
4) Increased capacity to treat complicated obstetric cases managed in health facilities
5) Promotion of Tetanus Toxoid for women of childbearing age
The MOH capacity to reach pregnant women with services was geographically limited, and thus
continued to rely on traditional birth attendants. The project collaborated to train and equip TBAs
with the basic skills for performing non-complicated deliveries and providing pre-natal and post￾partum care. In addition, the project worked to improve the skill level of health personnel at
health posts and centers to manage basic and complicated obstetric care. Facilities upgrades were
conducted, focusing on basic obstetric equipment and supplies.
Project Strategies and corresponding activities
1) Improve access to health care in target communities through training of health care personnel,
traditional birth attendants (TBAs) community health volunteers (APEs);
• Ninety-one TBAs trained and equipped with birthing kits (plus 33 kits sold to Save the
Children for their TBA activities);
• Sixty-seven MOH staff and 46 APEs trained in sequential management of childhood
illnesses
• Fifty-seven APEs received supervision and equipment. (refer to Annex F & E)
6 CARE Mozambique Final Evaluation
2) Improve the quality of health service delivery through training of health personnel, MCH and
elementary-level nurses (refer to Annex F for a complete list of trainings), and by assisting
the District Health Services to identify supervision mechanisms:
• Training and modeling in supervision. to monitor technical competency of health facility
personnel level as well as TBAs and APEs;
• Structural and material support to rehabilitate the maternity units at center and post levels,
as well as providing basic obstetric equipment; (Annex E)
• Logistical support for vaccination campaigns;
• Logistical support for transport of medicines and Provincial Health Service goods; and
• Financial support for technical scholarships.
3) Improve community health security through community-based participation in the formation
of community health councils:
• Fifty-one community health councils (637 individuals) trained and active in delivering key
health messages using a variety of multi-media materials, mosquito net promotion,
emergency transport systems and support to TBAs and APEs;
• Establishment of a community referral system to identify early danger signs of childhood
illness and pregnancy;
• Promotion and sales for mosquito nets;
• Development of an emergency transport system including SOS alert flags, bicycle
ambulances and plans for medical emergencies; and
• Organization of 2 mobile community theater groups to transmit project messages.
4) Promote improved basic household health practices through a multi-media communication
project, including:
• 5 Radio and audiocassette productions;
• 5 Theater productions;
• 5 Illustrated flip chart productions; and
• 2 Flash card sets.
B.2.b) Progress Report of Interventions
(i)
A comparison of results from KPC surveys in February 1997 (baseline) and August 2000 (final)
reveal positive changes in all intervention areas (refer to KPC 2000 report for specific percentages
and a complete analysis of results). Not only was there an improvement in health behaviors over
time within the project communities, but there was a difference in behaviors between the project
communities and a non-project community (control group) surveyed in August 2000, with the
project communities exhibiting healthier behaviors. The intervention-related results are as
follows:
7 CARE Mozambique Final Evaluation
Case management of diarrhea
According to the KPC 2000 results, a greater percentage of mothers in the project communities
gave more breast milk, more liquid and more solid food when their children had a bout of
diarrhea. There was a modest increase in the use of cereal-based ORS, reflecting strong
promotion from the MOH and project. More mothers could recognize some danger signs of
dehydration from a list that included sunken eyes, dry skin, lack of tears, thirst and vomiting.
Case management of malaria
KPC results showed that more mothers in project communities sought medical help when their
children had signs of malaria. A large percentage of mothers knew that a mosquito net could
prevent malaria. Despite the cost, more families possessed a mosquito net, and even a larger
percentage of families intended to purchase nets when they obtained sufficient funds.
Case management of acute respiratory infections (ARI)
More mothers in project communities could recognize danger signs of ARI and an increased
percentage of mothers sought medical help when their children were afflicted. Antibiotic use for
treating ARI was higher in project communities than in the control community, but was lower
than that indicated in the baseline survey results, which might be a result of the Rational Use of
Antibiotics training.
Maternal health
The greatest improvement in health behavior in the project communities was seen in the area of
maternal health. The evidence showed that more mothers sought prenatal care, possessed a
maternal health card and had received two tetanus shots. The percentage of pregnant women
receiving iron supplements had increased and a larger number of births were attended by a trained
personnel (TBA, nurse-midwife). In addition, many women in project communities reported
having a birth plan and having purchased a new razor blade for severing the umbilical cord. Some
mothers even had managed to put money aside in case of a birth-related emergency.
Other positive changes that cut across all interventions were the effectiveness of the radio
messages and the residents’ awareness of a newly acquired emergency transport system in the
village. KPC results showed villagers in both the project and control communities had heard radio
messages on mosquito nets and the emergency transport system. They had also heard theater
broadcasts on topics such as diarrhea, malaria, ARI and birth planning.
(ii)
The above achievements could be credited to the project’s two pronged strategy to assist MOH to
provide quality child survival services and develop household level capacity to access and use
health care services. The activities in the four intervention areas that led to these achievements are
described in details in Annex B.
8 CARE Mozambique Final Evaluation
(iii)
Despite positive changes evident in the results of the 2000 KPC survey, the project did not always
achieve percentages proposed in the DIP objectives. The following are factors contributing to the
under-achievement of objectives.
Time constraints
At mid-point the project doubled its effort in order to compensate for the poor community-level
performance during the first two years. Nevertheless, effects of the time lost continued to slow
progress in a number of ways:
• The expected improvements in project indicators were overly-ambitious
Most of the targets for indicators were set in an arbitrary manner at the time the DIP was written.
Given the health infrastructure, socio-economic and political conditions in Mozambique, the
targets might not have been appropriate for a project that was not already challenged by limited
time. Due to the lack of attention to the HIS system and later loss of the database, the project was
not able to measure some of the intermediate objectives on utilization and quality of health
services.
• Several community councils had limited experience with their new tasks
Despite attempts, the project lagged behind schedule in forming the new groups of councils
(scheduled for January-February 1999 in the DIP, realized in February 2000). Community council
members were volunteers, and had to fit the work they did for the council into their busy schedule
of household and agricultural work. Time constraints may have prevented them from reaching the
desired number of clients.
• Limited use of the Health Information system (HIS)
The original HIS was revised several times and finally replaced by a more comprehensive system
in 1998. However, the field staff who collected the data neither fully understood the purpose of
the system nor were sufficiently trained in analysis and use the data to respond to problems in the
field - such as the inactivity of some council members.
• The full potential of the IEC strategy was not realized
A dynamic IEC strategy was developed in May 1998. The IEC consultant trained staff and
supplied them with adequate IEC material and instructions for optimizing use. However, there
was insufficient time to realize the full potential of this strategy. Three new field agents were
hired in March 1999 and were trained in IEC in April 1999. New council members were trained
and evaluated one year later. During the evaluation in September 2000, council members were
still using the most convenient method of giving lecture-type health talks with or without the
picture boards/flip charts. The field agents were not adept at assessing the effectiveness of a
particular IEC method in a given situation and modifying their strategy accordingly. Nor were
they skilled at transmitting adult education concepts to the council members.
9 CARE Mozambique Final Evaluation
Harsh economic conditions in the communities
In the campaign to promote mosquito nets, the project successfully overcame the “ no demand, no
supply” slogan of merchants and introduced a new product into the community market (details in
Annex C). However, whether or not a family owned a mosquito net was finally determined by the
family’s ability to pay for one. Even the subsidized the price of 60,000 Meticais (about $4) was
prohibitive for the villagers who had no disposable income. General poverty also limited the
communities’ contributions for purchasing a bicycle-stretcher, as part of the emergency transport
system. Many villagers pointed out to the external evaluator that knowledge and good will were
not in short supply; it was money that was in short supply. Harsh economic conditions also had
an impact on the referral system. While council members practiced their referral system
enthusiastically and encouraged residents to use available health facilities, the villagers whose
ailments did not fall into categories related to “under-five”, “ pre-natal/delivery” and “emergency”
had to pay 500 Meticais (about 3 cents) for health services. Even this sum inhibited some of them
from seeking care.
The health care delivery system is characterized by weak infrastructure, poor management and
political favoritism
Management difficulties in the MOH had a negative impact on the project at the field level.
(This situation is common to all PVOs working in the province) Common problems were
constant turnover in personnel, absences of health personnel from health centers/posts,
inconsistent drug distribution schedules and inefficient communication between the provincial
and district level regarding authorization for training and supervision activities all. Problems
within the MOH Essential Medicines distribution system made it impossible to assure a regular
supply of iron/folate, antibiotics, ORS and other essential supplies. The project worked with
them to stabilize supplies towards the end of the project.
The distribution pattern of health facilities in the three districts varied largely. While the MOH
policy dictates that the distance between heath facilities not exceed 25 kilometers, in some
districts, health posts were situated much closer to each other, while in others they were very
disperse. The evaluator visited two communities where residents had to walk 35 kilometers to the
nearest health post. Those communities had not seen a DDS health mobile unit for an entire year
except during the polio campaign. One district health director traveling with the evaluation team
confirmed that mothers who walked long distances to get their children vaccinated often had to
spend the night near the health facility. No explanation was given as to the uneven geographical
distribution of health facilities, but politics are involved.
MOH policies regarding training of TBAs lacked clarity and purpose
CARE’s CS-XII project was part of a consortium of organizations within Nampula Province that
joined together in 1996 to work with the MOH on developing and enhancing the MOH’s
community-based TBA project. An initial workshop designed to revise the national curriculum
and introduce a Life-Saving Skills Package was held in June 1998 and included representatives
from the national MCH department. This workshop was conducted by an ACNM consultant. The
workshop received mediocre reviews, and since then it has been very difficult for the consortium
to act together and clarify the DPS position on the role of TBAs, supervision responsibilities and
community participation in supporting TBAs. Subsequently, very few TBA training events have
occurred. In addition, it has been difficult to put together TBA kits, which are vital to a TBA’s
10 CARE Mozambique Final Evaluation
performance. This is due to the lack of clear guidance from the DPS regarding the contents and
re-stocking procedures.
Nevertheless, the project has completed at least one training in each district, in cooperation with
the district health services. DPS policy permitted a maximum of 12 participants per training, thus
making unrealistic the project’s plan to train 200 TBAs. The project provided provisional kits and
trained 91 TBAs, but a rumor of the Ministry’s eventual phase-out of TBAs stopped the project
from training more TBAs.
A lack of systematic follow-up and support from the MOH/DPS/DDS following project-sponsored
trainings considerably weakened the capacity building effect.
Despite sustained good communication and interaction with the district-level health services in
the planning and execution of training events, the government did virtually no follow-up
supervision or post-training verification of skills application. CS-XII’s nurse trainers continued to
supervise and support health agents on a monthly basis but were not intended to be a substitute
for the MOH. In September 2000, the project expected a more formalized exchange of
performance indicators and data gathered by the nurse-trainers, but this did not materialize.
Both the original project manager and assistant manager had strong clinical backgrounds and
prioritized the clinical skills training and supervision during the first 20 months of the project.
After the resignation of the assistant manager, this component received insufficient attention since
the CARE nurse-trainers focused almost exclusively on the APEs.
Difficulties at management level impeded work in the community
During the first 2 years of the project, the assistant manager supervised the CARE nurse-trainers
and the field coordinator oriented and supervised the field agents. Unfortunately, neither the
nurse-trainers nor the field agents were given adequate direction and go-ahead to perform their
intended functions until after the project implementation review in April of 1998. Even then, the
distance between the three districts and logistical problems with transport resulted in the
insufficient supervision of field agents, who themselves were responsible for supervising council
members. Supervision of field agents improved considerably under the new field coordinator, but
that individual did not receive adequate supervision from the project manager. Supervision of the
nurse-trainers did not improve significantly under the new project management during the second
phase of the project, particularly since the position of assistant project manager was unfilled for
several months.
The project manager suffered from poor communication financial management personnel based
in Maputo and had little information on which she could base financial decisions in regard to
project activities. Having to deal with administrative problems at the sub-office meant the project
manager had little time to be in the field, which limited her insight into field concerns and
reduced her involvement with the DDS.
11 CARE Mozambique Final Evaluation
(iv)
A major, on-going constraint experienced by the project was the main local partner’s inflexibility
and less than enthusiastic cooperation. The MOH’s prevailing attitude is that PVO interventions
are “supportive” when they bring material benefits (buildings, equipment, and vehicles), and
“donor politics” when they address community mobilization for behavior change. Because of this
attitude, MOH was perceived as merely tolerating CARE’s CS project. The project manager’s
communication with DPS/DDS regarding project plans, directions, requests for cooperation in
training, invitations to gatherings, etc. either, at best, received delayed acknowledgment or were
ignored all together. On several occasions during the evaluation process, the DPS/DDS asked
questions related to project interventions that revealed their lack of awareness of what the project
had achieved in communities. For example, a preventive health agent in one district, who was
presumed to be active in his communities, admitted to the evaluator that he had neither
encountered nor heard of any work completed by a community health council volunteer, and
proceeded to discuss the ‘evils’ of donor politics. Further discussion with the same DDS staff
member and the field facilitator in the area disclosed that the former- although he had been a
classmate and friend of the field facilitator - had refused repeated invitations to meet the health
council members in the area. Furthermore, field travel, accompanied by two district health
directors during this evaluation, served as an excellent opportunity for the evaluator to notice the
change in attitude of these officials after observing and listening to the community residents.
 (v)
Main lessons learned:
• Time is required for even the best-tested IEC strategy and monitoring methods to be
established in a community.
• In an atmosphere of deep-rooted socio-economic and political problems, personnel training,
building renovations and supply of equipment may not be sufficient in bringing about
substantial change in health behavior.
• Repeated internal evaluation of project progress (using similar methodology as the mid-term
and final evaluation) could help detect problems that the monitoring and supervision fail to
capture.
• Cooperation of the provincial and district government could add greatly to the efforts of an
organization in bringing behavior change and sustainability of this change.
• It would be worthwhile to introduce DDS/DPS to the field interventions early in the project,
in ways besides written reports.
12 CARE Mozambique Final Evaluation
• Cooperation with MOH was difficult, and the project could have benefited from lessons
learned by other NGOs and assistance from PVO headquarters and donors.
B.2.c) New tools and approaches/ operational research/ special studies
The CS project conducted the following studies during the life of the project. A detailed
description of some of these studies is found in Annex C.
• Rapid assessment of current practices of pre-natal care : Year two
• Rapid assessment of current home care practices for treating diarrhea, malaria and
pneumonia: Year one
• Participatory Learning Appraisal: Year two
• Rapid assessment of IEC impact in project communities: Year three
• Community-based opinion survey: Year three
• Rapid market research for mosquito net availability: Year three
• Focus groups on obstetric emergencies and transport: Year three
• Physical assessment and inventory of health facilities: Year four
B.3) Results: Cross-cutting Approaches
B.3.a) Community Mobilization
(i)
The CS project approached community mobilization through the formation of community health
councils. The project aimed to introduce key health messages, promote mosquito nets, launch an
emergency transport system, establish a community referral system and extend support to TBAs
and APEs with the help of these health councils. Community mobilization also included the
organization of two mobile theater groups.
When selecting communities, the project consulted DDS and took their recommendations for
those communities considered most in need of intervention. The finalists for the community
council members (12-15 in number) came from a list selected by members in the respective
communities or the village chief. The project staff gave preference to those who showed a
genuine interest in community work and had prior experience related to community mobilization.
The volunteer council members underwent two months of training over time. The first month’s
training covered such topics such as “ How to contact community leaders and start a community
meeting”, “How to explain objectives of activities”, “Responsibilities of the council/community”
and “How to begin an open discussion with the community”. The volunteers received training in
key health messages during the second month (2-3 messages per week). The curriculum also
covered the following: analyzing the health situation in a community, encouraging the community
to use the referral system, making home visits, teaching about danger signs, promotion of birth
plans and emergency transport system. During the third month, the volunteers practiced what
13 CARE Mozambique Final Evaluation
they had learned in their communities while the field agents supervised them, evaluating their
preparedness as council members.
Despite the delayed start, by the end of September 2000 the project had trained 51 community
councils that were actively participating in health promotion activities. The evaluation team was
unable to assess their skills in carrying out their roles, but the interviews did reveal their dominion
of the key messages and their motivation.
ii)
Interviews with a large number of council members, community residents, health post staff, TBAs
demonstrated that the objectives for community mobilization were successfully met. All council
members could articulate the objectives of the council and their duties, and they had retained what
they had learned in training. They reported doing the following activities: planning and organizing
their work as a group, delivering health messages, referring residents to health facilities, keeping
statistics, cleaning the health post, assisting with child weighing and helping ill residents with
house work.
 (iii)
Lessons learned:
• Asking the community leader to propose potential council members may not result in
identification of the strongest candidates. The APEs and TBAs might be better able to identify
the most able and willing individuals.
• DDS may be politically biased in its selection of communities – which could result in the
neediest communities being passed over.
• Building health councils onto already existing community associations (church groups,
farmers’ associations, youth groups etc), could add to the sustainability of the councils.
• In the areas where they overlapped, the three CARE projects (CS-XII, Reproductive Health
and agriculture) could have enhanced sustainability by jointly identifying one entity
(council/association) through which the projects would work.
(iv)
There was a great demand in the communities for the continuation of the project. Those
interviewed, including mothers, fathers, councils (as a group), individual council members, APEs,
nurses and TBAs all regretted the end of the project and asked the evaluator help find a way to
make this project continue. It was clear that communities were just “coming out of the dark”, and
it was too soon to terminate the project. Health post personnel stressed that they appreciated the
training and had learned a lot. They were proud to demonstrate how they used the medical
equipment they had received. Even the two DDS directors who participated in the evaluation
mentioned that the project had just begun mobilizing the population, and that the project would
have achieved a great deal more with more time. The communities without access to a health post
were most affected by the end of the project. According to people in these communities, when
they completed the HIS forms and received supervisory visits, they felt better connected to the
health system, which motivated them. Even the non-literate TBAs found ways to keep track of the
14 CARE Mozambique Final Evaluation
work they did, because they had a sense of pride in communicating how many children had been
delivered with their help.
(v)
Sustainability of project activities:
The project had originally considered the councils as temporary, and they were not incorporated
into the sustainability plan. However, they have become veritable community institutions in
which villagers participate and on which they depend. Motivation is still an issue, however.
“Acquiring new knowledge”, motivated many of the council members interviewed, and
supervisory visits were an integral part of this motivation. After DDS directors observed the
councils working, they proposed that the DDS could continue limited support of the council
work. Field agents believed that those members with common interests (members of another
community-based organization such as church) would continue their work.
Several of the project’s activities will continue for several more years through CARE’s sister
projects: the reproductive health, agriculture and micro-credit projects. The reproductive health
project will continue CS-XII’s maternal health components such as training in danger signs,
referrals and emergency transport systems. The agriculture and micro-credit projects will continue
to promote mosquito nets.
B.3.b) Communication for Behavior Change
(i)
CS-XII introduced an IEC package as part of the project’s strategy for behavior change. The IEC
strategy was based on the idea that certain harmful behaviors contributed to an undesirable state
of health and welfare. The formative research stage identified desirable behaviors and outlined a
process for moving from harmful to healthful behaviors. A complete description of the
components of this strategy and effectiveness of each IEC method, can be found in Annex D.
The IEC strategy was flexible enough to allow experimentation with different kinds of message
delivery. However, the previous training that the field agents had received did not encourage
creative experimentation, resulting in their reluctance to try out new methods. Field staff and
council members would have benefited from training in how to incorporate IEC messages into
day-to-day activities, such as through songs sung during agricultural work, or through lullabies
and proverbs etc. The trained theater groups were very popular and exposed large numbers of
people to health messages, but organizing transport for the groups proved difficult. It might have
been useful to encourage villagers to produce skits or theatre of their own, using field
agents/council members to correct any errors in their messages. The project would have benefited
from operational research on local leisure and entertainment activities.
(ii)
The KPC 2000 results showed that the IEC strategy contributed to positive changes in knowledge
and behavior in all four intervention areas. However, the project’s original behavior change
15 CARE Mozambique Final Evaluation
objectives were not fully attained, given the short period of time during which the IEC strategy
was implemented.
(iii)
Lessons learned:
• A better way to retain health knowledge and sustain health behaviors would have been to
include young audiences in the IEC strategy, for example, through school presentations and
the creation of youth groups and sports clubs.
• Alternative IEC methods could have been explored, such as the adoption of local songs,
proverbs and dances, and the community should been encouraged to replicate the theater
groups’ activities. Alternative IEC materials might have included local artwork, using wood,
bark or traditional dyes, for example, or even sand art.
• Prior investigation of pre-existing theater groups (those trained by DDS or other NGOs) could
have helped lower the costs and promoted better relations with DPS.
(iv)
Sustainability of health behaviors
During interviews in the communities, it was clear that both council members and health
personnel were appreciative of the knowledge they gained from the project. As mentioned, most
community volunteers cited “ acquiring knowledge” as their main motivation for becoming a
council member. It is expected that community council members will retain what they have
learned through the IEC, and that it will be translated into sustained behavior change and they
may continue to influence community norms.
CARE’s reproductive health project has taken over support of the CS project’s theater groups,
and the agriculture and micro-credit projects will continue promotion of mosquito nets. More
importantly, the two DDS directors have realized the project’s efforts to mobilize the community
for behavior change. After their observation of evaluation activities, the DDS directors showed
encouraging signs of making a plan for assisting the councils with their work.
 (v)
Feasibility of the sustainability plans
The CARE sister projects are expected to be effective for sustaining community involvement in
CS activities for several more years. However, a commitment from DDS would be more
successful in sustaining the behavior changes in these communities over the long term. There is
always the issue of the DPS lacking resources to do the supervision and follow-up necessary for
community work, an issue which never seems to be addressed by the donors who fund bi-lateral
projects with the MOH even though such projects very often promote community intervention.
16 CARE Mozambique Final Evaluation
B.3.c) Capacity Building Approach
(i)
Strengthening the PVO
This grant has improved the capacity of the PVO to design and implement effective child survival
projects in the following management areas:
• The PVO is more conscientious about the qualifications necessary for a Project Manager. It
does not underestimate the importance of language ability and the need to hire project
managers who have the complete complement of technical, inter-personal, and management
skills.
• The PVO realizes that the Project Manager should have a complete orientation. They should
arrive at the project site with familiarity with personnel, resource, and financial policies, and
disciplinary measures. It is unrealistic to expect the new manager to learn from the procedure
manual.
• The PVO recognizes that even the most qualified manager will need support from the
headquarters and country office. HQ backstop personnel must understand the language and
the project context.
(ii)
Strengthening the Local Partner Organizations
The project’s only partner was the Ministry of Health. In order to enhance MOH’s institutional
and performance capabilities, the project conducted several needs assessments. Subsequently, the
project undertook training of health personnel and provision of material and equipment. The
questions pertaining to health facilities strengthening (iii), health worker performance (iv) and
training (v) are discussed together, below. Additional information on training is in C.2 “staff
training” section.
The physical assessment and inventory of health facilities was conducted during October and
November of 1999 (described in Annex C section 2.c.5). This assessment led to rehabilitation
and provision of equipment to health units in a total of twenty-two health centers and posts. The
list of equipment provided can been found in Annex E.
Improving the delivery of clinical services related to the interventions depended largely on
supervision by the nurse-trainer and monitoring of individual performance at the health unit.
Unfortunately, it is impossible to summarize the progress made in behavior-based competencies,
due to problems in the project data collection and processing system (HIS). However, a review
of supervision forms by category yielded the following observations for the period of October
1999 to June 2000:
17 CARE Mozambique Final Evaluation
• In general, elementary nurses made improvements in their working environment, especially
regarding workplace hygiene (a critical factor in most health units).
• Nurses continued to meet the minimum requirements for work and workplace organization,
including maintaining regular hours, proper management of drug supplies, practicing triage
for urgent cases, etc.
The project felt it had made a considerable contribution to the effective and efficient use of
resources within clinical service delivery, but these areas deserve continued attention.
As discussed in section 2.c.6 (Annex C), the project also organized focus group discussions and
individual interviews to better understand what health personnel themselves perceived as barriers
to improved quality of health care.
Lessons learned from the capacity building of the local partner (MOH):
• The training of district nurses needs to be on-going rather than one-time.
• Focusing on improving clinical skills is not a comprehensive approach to improving service
delivery.
• Training needs to be accompanied by development and implementation of a performance￾based supervision system that can be maintained by the DDS.
• Lack of medicines at the provincial level is rarely the reason that the peripheral levels do not
receive their medical kits. Poor logistics management, corruption and favoritism appear to be
the most significant factors. In the future, projects should endeavor to clarify from the
beginning what obstacles might impede an effective drug supply system and work diligently
with the decision-makers within the MOH to affect changes.
(iii) – (v)
Health Facilities / Health Worker Performance Strengthening and Training
Health facilities strengthening, attention to health worker quality and training will be sustained
since the project has established a foundation for the transfer of knowledge. CS-XII trained MCH
nurses in TOT, supervision and modeled supervision to the DDS. Another important training was
Rational Drug Use. Those trained are expected to transfer their skills to other personnel within the
health system.
The new USAID mission-funded health program contracted to JSI will soon pilot IMCI training
in Ribaue District. This will serve to reinforce the training in Sequential Management of
Childhood Illness (based on the IMCI algorithm) which CARE gave to all DDS staff.
Presumably, the new training will be accompanied by efforts to improve DDS supervision of
staff. All of this, while not planned by the project, will lead to sustainability of the capacity￾building provided to the DDS.
18 CARE Mozambique Final Evaluation
B.3.d) Sustainability Strategy
Progress in meeting sustainability goals and objectives (as articulated in the revised DIP):
Goal Related to Objectives Progress
1) MOH staff and APEs provide care per
MOH (WHO) guidelines for pneumonia,
malaria and maternal care.
# 3, #4 and #5 1) Trained all health personnel and
APEs in MH and management of
childhood illnessess
2) Referral system implemented
3) Rational Use of Antibiotics training
and improved logistics
4) Quality of care assessments
completed for MH
2) DDS professional staff better able to
train others
All seven objectives 1) DDS staff became trainers in 3
subsequent training events
2) Implemented a system for staff nurse
trainers to supervise TBAs closer to
a post.
3) DDS staff better able to perform
community-based health activities
# 1, #3, #6 and #7 Attempts to coordinate work of CARE
facilitators and their counterpart AMPs
succeeded only in isolated cases
4) Form a better link between MOH
services and the community
All seven objectives Community council members
successfully familiarized about the roles,
functions and tasks of MOH staff with
whom they related
5) Guarantee community use MOH
services
# 1, #2, #3, #4 and #5 1) Community council members
successfully familiarized about
health activities in closest health
centers
2) Increased use of services
6) Guarantee that the community
recognized danger signs which require
referral
# 1, #2, #4 and #5 1) Basic key messages emphasized
2) Supervision system for CCs
developed and implemented
3) CCs trained in recognizing danger
signs and making referrals
4) Referral system implemented
7) Guarantee that the community is able
to perform home-based prevention or
treatment of selected diseases and able to
perform safe deliveries for non￾complicated cases
# 6 and #7 1) All CCs trained in prevention of
dehydration, improved hygiene, use
of mosquito nets
2) All CCs provided with IEC material
to teach prevention and home care.
3) TBAs trained in danger signs of
pregnancy, safe deliveries, and
provided with essential supplies.
4) TBAs and CCs trained in promoting
birth plans
19 CARE Mozambique Final Evaluation
8) Community recognition and support of
role of APEs and TBAs
# 1, #2, #3, #4, #5 and #6 1) Communities sensitized about APEs
and TBAs role, but limited support
for them in some communities due to
the belief that they receive benefits
from MOH and/or from the NGO
2) No sensitization of DDS needed
9) Rational use of available drugs # 3, #4 and #5 MOH staff and APEs trained in Rational
Use of Antibiotics.
10) Development of emergency
transportation system in selected
communities
# 1, #2, #4 and #5 Emergency transport system in place with
plans and flags. Ten CCs purchased
bicycle stretchers.
The project’s sustainability strategy was based on the idea that MOH will maintain and transfer
their new knowledge. Introducing approaches for building financial sustainability was not part of
the project.
As explained earlier, the Reproductive Health project is assuming the theater troupes, promotion
of mosquito nets and bicycle stretchers and promotion of community emergency transportation
plans. The agriculture and micro-credit projects have assumed promotion of mosquito nets since
the high prevalence of malaria also affects agricultural and economic productivity. Phase-over of
these activities began in July.
20 CARE Mozambique Final Evaluation
PROJECT MANAGEMENT
C.1) Planning
While the proposal for this project was well thought out, CARE did not have any health staff in
country and the agriculture project was the main activity in Nampula province after the end of
emergency relief activities. There was no prior relationship with the DPS nor an understanding of
the complex attitude of officials. The project manager who was hired did not have an opportunity
to discuss the proposal with the consultant who prepared it. The HQ backstop person originally
assigned to the project, did not have an understanding of the proposal nor the context of the
project. The project manager had been on site only a month when the KPC needed to be
conducted, which was followed immediately by the arrival of the HQ representative to write the
DIP. Relations had not yet been built with the DPS, DDS, or communities and few project staff
were on board. Therefore, the original DIP as written by the project manager and HQ
representative was not totally adequate for the situation. That compounded the difficulties of
start-up in the incredibly difficult and complex environment.
When the new HQ technical advisor visited the project in early 1998, it was very apparent that a
new implementation plan was needed. While referred to as the “revised DIP” this document, did
not change the overall strategy, interventions, or target population. Rather it adapted the work
plan, sustainability plan, training schedule, and job descriptions to the reality of the project. This
was submitted to USAID with the mid-term evaluation.
This evaluation used the revised (second) DIP and the mid-term evaluation as its point of
reference. The second DIP was prepared with the full participation of project staff. The partner
organization (MOH/DPS/DDS) was informed about the changes to the original project
implementation plan, but it had little involvement in the planning process due to time constraints.
The DIP work plan was practical in the sense that it gave the project direction, and was
occasionally referred to by the Project Manager. However, she found it difficult to adhere to the
work plan closely, due to her lack of understanding of the importance of some efforts and to
circumstances outside the control of the project. An internal amendment to the second DIP work
plan was introduced by the new manager in December 1998 to reflect circumstances at that time.
(This was not approved by HQ.)
When the revision was written, it was completely understood by the then project manager and
assistant project manager. However, it may not have been explicit enough for the new manager
and assistant who came later. There was not enough explanation of the priority of certain
activities over others and not enough detail about the HIS and supervision systems.
C.2) Staff Training
During the CS-XII project, both the project and partner’s (MOH) staff benefited from a total of 23
training sessions. A list of these trainings is found in Annex F. All training curricula included a
pre and post test for evaluating what participants learned from the sessions. However, a tool for
21 CARE Mozambique Final Evaluation
monitoring the overall application of knowledge, skills and competencies of the project and
partner staff was never developed.
Nevertheless, the evaluator noticed strengths among the project staff that could be attributable to
the training they had received. Although she could not make a valid before and after comparison,
the evaluator has had previous experience with a CS-XII project in the same province, working in
a similar atmosphere, with staff of similar educational and ethnic backgrounds. This gives her a
point of reference. In particular, she noticed that CARE’s CS-XII project staff exhibited strong
communication skills. The staff were able to give a comprehensive overview of the project to an
audience that included the DPS members and other PVO staff. They spoke with confidence and
were able to answer difficult questions, articulate work-related problems, suggest feasible
solutions and and analyze hypothetical situations. There was reason to believe that the project
staff, especially nurse trainers and field facilitators, had transferred some of their self-confidence
to those with whom they worked in health posts and village health councils. The district health
director of Mecuburi (the district where the external evaluator saw the least progress in
community work of all three districts) commented that in comparison to health
committees/councils he had seen in Angoche (another district in southern Nampula), council
members in Mecuburi were confident speakers. Discussions with the two supervisory level staff
members further revealed knowledge/skill acquisition in the areas of computers and research
techniques. Staff members also reported increased ability to work in teams and avoid conflicts.
Concerning MOH staff progress, the evaluator (a nurse) tested health workers’ knowledge by
asking technical questions from some of the areas in which health workers had been trained. All
those questioned gave appropriate answers, which may or may not be attributable to staff training
provided by the CS-XII project.
Obviously, this one project could not meet all training needs of project and partner staff. Both
need additional training and practice in order to learn to take initiative in their work and to
analyze data and technical information. Considering the low level and quality of education in
Mozambique, promoting these skills is a remaining challenge for all PVOs.
The project had adequate resources dedicated to staff training. In fact, training was one of the
largest investments of the project.
Overall lessons learned about building the capacity of project staff:
• It is important to conduct needs assessments prior to training sessions and to modify the
training curriculum accordingly.
• Ensure the presence of a DDS and DPS representative at all staff training. For MOH staff
training, obtain prior approval of the curriculum by the DPS.
• It is important to remember that the fact that staff’s previous training may be in non￾participatory pedagogical methods. If so, they will need additional time to adapt to new
techniques.
• Project staff need to be trained at the beginning of the project in the skills that they will use in
the community and transfer to community volunteers.
22 CARE Mozambique Final Evaluation
C.3) Supervision of Project Staff
The assistant manager was responsible for supervising the field coordinator and nurse trainers.
The field coordinator supervised field facilitators/agents. During supervisory visits (every 4-6
weeks), supervisors completed a cumbersome 3-page form that evaluated the workers on their
ability to complete assigned tasks, their professional behavior, and their skill in administration
and logistics. The large geographical area covered by the project and the distance between
communities made it difficult for the two supervisors to keep track of the movements of nine field
staff. The assistant manager suggested that an integrated coordinator/supervisor should have been
posted at each satellite field office (one per district). This person could have supervised the staff
of all three CARE projects sharing the office, and would have had more control over the
movement of the motorcycles, need to replenish fuel and other supplies, and coordination of field
activities between the projects.
Under the second project manager, the supervisors were not given adequate instruction or
modeling in supervision. This resulted in a lack of performance-based supervision of technical
work of field staff. For example, some staff were still teaching councils to make cereal-based
ORS six months after the project had been asked to desist from this in favor of promoting use of
any home available fluid to prevent dehydration from occurring.
There is some evidence that the DDS, particularly in Ribaue District will continue improved
supervision. If project motorcycles are relinquished to the DDS as planned, pending official
approval, this will greatly facilitate continued supervision by the DDS. Lack of transportation is
the biggest impediment to maintaining adequate supervision.
C.4) Human Resources and Staff Management
Since program operations are not continuing, there are no personnel policies and procedures in
place.
April 1998 was a was turning point for the project staff morale and cohesion. During the first 20
months, the three nurse trainers and three field facilitators were attached to health facilities. While
the nurses were able to model clinical procedures and supervision, the facilitators were unable to
function as counterparts to the ambulatory health agents as anticipated. The staff was unclear
about the scope of their work and their morale was low. Morale improved after job descriptions
were re-written and they were able to work independently without waiting for DDS counterparts.
The lack of supervision and support continued to be an issue, however. It was not until March
1999 that the new manager organized a staff retreat with both old and new staff to set new
policies and to communicate the new project direction to staff. The field staff reported
experiencing a sense of cohesion and direction since then. The assistant project manager said that
except for few disagreements (“that is common among those originating from different
provinces” in his own words), the staff worked as a team.
Staff turnover throughout the life of the CS-XII project has been relatively low. The replacement
of the project manager took very little time, but perhaps more time should have been taken to look
for the most suitable person. A new position for a Field Coordinator was created and filled. The
23 CARE Mozambique Final Evaluation
loss of the assistant project manager for health reasons not only set back the clinical work but also
the loss of her overall vision affected the project. Hiring her replacement took over 6 months, due
to the lack of educated Mozambicans, particularly in the province. This seriously affected field
supervision as described earlier.
Regarding staff transition to other paying jobs at the end of project, the management felt that staff
had been given adequate notice; staff members knew they had signed a limited contract. They
were trained, given work experience, helped to prepare their curriculum vitae, and informed of
job opportunities with other PVOs. Additional job search activities are seen as their own
responsibility.
C.5) Financial Management
There was no adjustment in the overall project budget. The changes made between line items
were within USAID regulations.
Financial management, however, posed problems for the project. Staff were neither sufficiently
trained nor supported in estimating costs or programming based on the budget. The Maputo
office had a full finance section that handled the finance activities for all CARE projects in
Mozambique, while the finance officer in the Nampula sub-office was responsible for posting
charges and paying daily bills. Initially, the project manager generally did not have enough
information to manage the budget properly. Budgetary analysis was constantly delayed and it
usually took several months to obtain a statement from the Maputo finance office about
expenditures for activities. Pipeline analyses originating from PVO headquarters in Atlanta often
took six months to reach the Project Manager’s desk. Starting in 1999, this problem was
corrected and the project manager received monthly expense report. Many more improvements
are currently being implemented in CARE Mozambique administration and finance systems.
The other CARE projects have the necessary resources to continue the specific activities
previously described. It was never expected that the MOH would assume financial responsibility
for any of the activities and the strategy was implemented with this in mind. It is a plus that they
are now considering supporting the community councils and that JSI will be following up in one
district with IMCI training and supervision.
No technical assistance to develop financial plans for sustainability was intended in the
cooperative agreement.
C.6) Logistics
The project experienced difficulties in regard to logistics. Firstly, equipment and supplies came
from a great distance, from Maputo and South Africa. This resulted in delays related to air and sea
freight. Secondly, the project had a motor pool (cars and motorcycles) that required constant
repair and the quality of locally available mechanical assistance was inadequate for the task at
hand. The distance between the Nampula office and the three satellite field offices in the districts
and the lack of services there created problems. For example, all fuel to be used in two of the
24 CARE Mozambique Final Evaluation
districts has to be taken there by CARE in bulk. Motorcycles were the only form of transport
available to nurse trainers and facilitators for reaching their assigned communities. For them to do
their field work required the regular delivery of fuel, but the vehicle designated to transport fuel to
the field offices often had break-downs, and other alternatives were time-consuming and costly.
Communication with Maputo regarding procurement was often an issue as the Maputo office
often did not understand the urgency of the request nor the exact needs. This is another system in
CARE Mozambique that is currently being re-vamped to be more responsive to the field offices.
The logistics system is not intended to support CS-XII operations and activities beyond this
cooperative agreement.
C.7) Information Management
The CS-XII project designed a comprehensive system for gathering information from all levels of
health facilities where the project staff was working. There were 10 forms in circulation and field
agents and nurse trainers completed five forms each.
In the communities, each health council member kept a notebook of activities showing the type of
activity, topic and number of people attending for each date. The head council member collected
this information weekly and submitted the forms to the field facilitator each month. The facilitator
summarized this information on a one-page table and submitted the summary to the field
supervisor, along with a monthly report of additional community events and vital statistics. The
field supervisor entered this data onto a computer. On the clinical side, nurse trainers collected
monthly epidemiological and preventive activity reports from health posts. They also completed a
trimester supervisory form and which was submitted to the assistant project manager in Nampula.
This form is described below.
Form A: Guidelines for supervising health personnel
The six-page form was completed by the three nurse trainers in the health facilities. The form
covered inspection of the following areas:
• The environment of the health facility
• The organization of work
• Accuracy in the areas of: pediatric triage, vaccination, case management of diarrhea,
pneumonia, malaria, pre-natal consultation and obstetric exams
• Frequencies (statistical data) of diarrhea, dysentery, malaria, prenatal consultations, births,
transfers
• Status of drug stock
• Accuracy in handling deliveries in maternity
• Accuracy in handling deliveries in community (destined for TBAs)
Health personnel could use the comprehensive form for self-evaluation. In addition to evaluating
health workers’ performance, the form helped collect statistical information in the intervention
areas, which was intended to monitor project progress towards meeting the objectives.. Because
the data was incomplete, this purpose was never achieved.
25 CARE Mozambique Final Evaluation
Although many pieces of paper passed hands, staff members had a poor understanding of the
purpose of this system. In some cases, field agents inflated numbers to prove the effectiveness of
their work in communities. Some council members finished or lost their notebooks and stopped
keeping statistics without informing the facilitators. Statistics kept by the nurse trainers were
more accurate because their forms because their forms were more quantitative. However, there
were gaps in the information they collected.
There was an attempt by the office in Nampula to analyze the statistics in order to identify areas
where additional health education was needed. However, since the system was poorly managed,
entries on the monthly forms did not always represent reality in the communities. Additionally, a
large amount of the data collected had no significance for project monitoring and only added to
volume.
The project staff would have benefited from more training in HIS, especially data analysis at a
lower level (i.e. performance of council members), and training in use of information.
There was no systematic data exchange between the project staff and the government of
Mozambique. Occasionally, project staff submitted some of the TBA and APE’s forms to the
DDS but this was not planned into the project. The MOH generally looked upon data collection
by PVOs with suspicion, and the only numbers relayed to DDS/DPS were those in the trimester
report describing project progress.
The project manager kept project and headquarters staff current of project achievements through
staff meetings and progress reports. The local partner, MOH was informed of all project activities
through trimester reports, and was invited to participate in trainings, etc. However, the level of
communication between the project and MOH was not sufficient to keep the DPS/DDS informed
of project’s achievements. Except for one DDS director, most DDS/DPS personnel were only
vaguely familiar with project’s achievements.
Project monitoring and impact data will serve as an input for advocating with CARE
Mozambique leadership for seeking funding to continue health activities in Nampula.
C.8) Technical and Administrative Support
CS-XII received ample external assistance throughout the life of the project. A record of
assistance received and the names of technical advisors/consultants appears in Annex G. Project
Managers were pleased with the utility, quality and the timeliness of technical assistance they
received. The only exception was the 8-month delay in the arranging for the consultant from the
American College of Nurse Midwives (ACNM), which held up the progress of training of
traditional birth attendants. Since the Ministry of Health refused to use the curriculum suggested
by the consultant and reverted to the use of the original TBA curriculum, this long awaited
external technical assistance proved to be a disappointment.
The project would have benefited from advice on how to deal with the attitude of MOH. Both
project managers who approached the national, provincial, and district level governments through
26 CARE Mozambique Final Evaluation
the proper channels encountered difficulties, and dealings with the MOH were very unpredictable.
A meeting of PVOs (sponsored by USAID or the PVO headquarters) to discuss PVO experiences
with the MOH, lessons learned and recommendations, would have allowed CARE’s CS-XII
project to adopt ideas that had worked for others.
Technical assistance from the PVO headquarters to the CS-XII project has been commendable.
The CS-XII technical advisors made seven visits to Mozambique throughout the life of the project
at opportune times. She assisted the project with an amended DIP and a new set of objectives. She
also introduced the concept of MIS and the referral system as well as providing technical
guidance for all of the interventions. The project received the managerial assistance through the
pipeline budgetary analyses from headquarters. The Nampula project manager reported that she
was most successful with procurement from PVO headquarters; if mistakes were made, there
were immediate efforts to correct them. The Nampula project manager had also received solid
managerial assistance from the CARE Mozambique management staff. Regarding time
involvement, PVO headquarters devoted 20-25% of its time to support the CS-XII project, while
the country office ACD committed 25% of time to this project.
C.9) Management Lessons Learned
Overall management lessons learned are as follows:
• Pay closer attention to the qualifications of the candidates for Project Manager. Ability to
learn the language and previous management experience as well as excellent interpersonal
skills and a broad technical background are essential.
• Arrange adequate orientation for new project managers; they should arrive at the project site
familiar with personnel, resource and financial policies, and disciplinary measures. Do not
expect the new manager to learn these policies from the procedure manual.
• Anticipate that even the most qualified manager will need support from the headquarters and
country office. Ensure that the backstop personnel understand the language and the context
and have relevant experience as well.
• Allow the manager to dedicate his/her time to managing the CS-XII project (attending to
fieldwork and visiting with DDS more often), by ensuring that tasks related to logistics and
procurement are performed in a timely manner by the appropriate personnel.
• Develop a more efficient system for providing timely finance reports to the project managers.
• Communicate with project managers in a clear, concise and more streamlined manner about
staff movement between offices, especially when the same facilities are used by several
projects.
• Encourage project managers to coordinate/share resources among sister projects
(Reproductive Health, agriculture and micro-credit) as much as possible at the province,
district and community levels to overcome problems in logistics, procurement, monitoring
and supervision.
• Encourage managers of all three projects to hold a staff meeting (inviting staff from all three
projects), at least every three months, to discuss the coordination of field work for enhanced
sustainability.
27 CARE Mozambique Final Evaluation
D. OTHER ISSUES IDENTIFIED
Another important issue that arose during the conversation with the project manager was the
difficulty PVOs have in firing a host country staff member. Local disciplinary procedures are
complex and often include legal action, which compels project managers to retain even the most
inefficient worker, out of fear of losing project funds in litigation.
E. CONCLUSIONS AND RECOMMENDATIONS
E.1) Success of the Project in Meeting Objectives
Despite time constraints, the CS project made major strides towards achieving its objectives. KPC
survey (2000) results showed that the project was successful in changing health behavior in all
four intervention areas. Community health council members used various IEC activities to deliver
key health messages, and radio announcements and theater groups emphasized the same
messages. KPC Results showed that more mothers sought medical care when their children had
ARI and malaria. More mothers knew a child with diarrhea needed more breast milk, more
liquids, and more solid food. More mothers had also learned the importance of pre-natal care.
More pregnant women sought pre-natal care, obtained their tetanus shots, planned for their birth
and delivered with the help of a trained attendant. Mothers had were more willing to spend a few
Meticias to buy a new razor blade for severing the umbilical cord and more mothers had decided
put money aside in case of a birth-related emergency. More families knew that a net could prevent
malaria bought mosquito nets, and in spite of the relatively high price, more had made plans to
purchase a net when they had enough funds. The radio projects and education by council
members had mobilized villagers to improve their access to transport for health-related
emergencies. Even people from non-project communities had benefited from the radio projects
sponsored by the project.
In addition to the behavior changes described above, the project has been successful in motivating
health personnel to gain more knowledge and apply that knowledge in their work. Health
personnel have come to appreciate their colleagues who assisted them in improving the quality of
their work. Twenty-two health posts that received equipment and materials have enhanced their
capacity to serve their clientele.
E.2) Achievements and Constraints
However, the project fell short of fully achieving its objectives for a number of reasons. Problems
during the first two years of field work set the project back significantly. The socio-economic and
political environment in the country, poor healthcare infrastructure, the minimal cooperation of
the major partner and several management level problems also had negative effects.
28 CARE Mozambique Final Evaluation
E.3) Lessons Learned
Achievement of objectives:
• Time is required for even the best-tested IEC materials and monitoring methods to be
established in a community.
• In an atmosphere of deep-rooted socio-economic and political problems, personnel
training, building renovations and supply of equipment may not be sufficient in bringing
about substantial change in health behavior.
• Repeated internal evaluation of project progress (using similar methodology as the mid￾term and final evaluation) could help detect problems that the monitoring and supervision
fail to capture.
• Cooperation of the provincial and district government could add greatly to the efforts of
an organization in promoting and sustaining behavior change.
• It would be worthwhile to introduce DDS/DPS to the field interventions early in the
project, in ways besides written reports.
• Cooperation with MOH was difficult, and the project could have benefited from lessons
learned by other NGOs and assistance from PVO headquarters and donors.
Community Mobilization:
• Asking the community leader to propose potential council members may not result in
identification of the strongest candidates. The APEs and TBAs might be better able to
identify the most able and willing individuals.
• DDS may be politically biased in its selection of communities – which could result in the
neediest communities being passed over.
• Building health councils onto already existing community associations (church groups,
farmers’ associations, youth groups etc) could add to the sustainability of the councils.
• In the areas where they overlapped, the three CARE projects (CS-XII, Reproductive
Health and agriculture) could have enhanced sustainability by jointly identifying one
entity (council/association) through which the projects would work.
29 CARE Mozambique Final Evaluation
•
Communication for Behavior Change:
• A better way to retain health knowledge and sustain health behaviors would have been to
include young audiences in the IEC strategy, for example, through school presentations
and the creation of youth groups and sports clubs.
• Alternative IEC methods could have been explored, such as the adoption of local songs,
proverbs and dances, and the community should been encouraged to replicate the theater
groups’ activities. Alternative IEC materials might have included local artwork, using
wood, barks or traditional dyes, for example, or even sand art.
• Prior investigation of pre-existing theater groups (those trained by DDS or other NGOs)
could have helped lower the costs and promoted better relations with DPS.
Capacity Building of Partner:
• The training of district nurses needs to be on-going rather than one-time.
• Focusing on improving clinical skills is not a comprehensive approach to improving
service delivery.
• Lack of medicines at the provincial level is rarely the reason that the peripheral levels do
not receive their medical kits. Poor logistics management, corruption and favoritism
appear to be the most significant factors. In the future, projects should endeavor to clarify
from the beginning what obstacles might impede an effective drug supply system and
work diligently with the decision-makers within the MOH to affect changes.
Management:
• Pay closer attention to the qualifications of the candidates for Project Manager. Ability to
learn the language and previous management experience as well as excellent interpersonal
skills and a broad technical background are essential.
• Arrange adequate orientation for new project managers; they should arrive at the project
site familiar with personnel, resource and financial policies, and disciplinary measures. Do
not expect the new manager to learn these policies from the procedure manual.
• Anticipate that even the most qualified manager will need support from the headquarters
and country office. Ensure that the backstop personnel understand the language and the
context and have relevant experience as well.
30 CARE Mozambique Final Evaluation
• Allow the manager to dedicate his/her time to managing the CS-XII project (attending to
fieldwork and visiting with DDS more often), by ensuring that tasks related to logistics
and procurement are performed in a timely manner by the appropriate personnel.
• Develop a more efficient system for providing timely finance reports to the project
managers.
• Communicate with project managers in a clear, concise and more streamlined manner
about staff movement between offices, especially when the same facilities are used by
several projects.
• Encourage project managers to coordinate/share resources among sister projects
(Reproductive Health, agriculture and micro-credit) as much as possible at the province,
district and community levels to overcome problems in logistics, procurement, monitoring
and supervision.
• Encourage managers of all three projects to hold a staff meeting (inviting staff from all
three projects), at least every three months, to discuss the coordination of field work for
enhanced sustainability.
E.4) Recommendations for USAID/BHR/PVC for Future Projects
The lessons learned mentioned above, especially those in the area of management, serve as
recommendations to PVC. An additional recommendation for USAID/PVC is not to
underestimate the importance of building a solid foundation for cooperation with the host
government before projects are designed. Donors/PVC should take the responsibility of making
their policies clear, and outline the types of projects that are feasible within the limitations of
donor funds. Host governments should have a role in the planning process. Efforts to convince the
host government of the project’s value only at the implementation stage, are often unsuccessful.
E. 5) How CARE will use these lessons-learned: [Judiann McNulty, CARE HQ]
Many of the lessons-learned are specific to the situation in Mozambique: a country office without
adequate support systems and no previous experience in health, the remoteness of the project site
and almost total lack of infrastructure, a very limited pool of educated nationals, and a Ministry of
Health whose attitude was seriously affected by years of war and post-war paternalism. The
management, administration and logistics issues discussed here affected all CARE Mozambique
projects, not just the CSXII project. As a result, CARE Mozambique is currently under-going
major reforms.
The other PVOs in Child Survival in Nampula project experienced similar constraints,
particularly in building relations with the DPS/DDS. The first SAVE the Children project
manager resigned in frustration. World Vision gave in to the DPS demands to pay DDS staff
31 CARE Mozambique Final Evaluation
salaries, provide direct health services and infrastructure, a solution CARE’s policies for
sustainability would never accept. With an abundance of donors, the DPS was not concerned
about building relationships with those who did not meet their demands.
Some of the lessons in dealing with the DPS/DDS and communities have already been applied to
the reproductive health project, which operates in the same districts. That project has become one
of CARE’s most successful reproductive health projects. This emphasizes the importance of
having the right project manager, which is one of the biggest lessons CARE is taking away from
this project.
Two lessons CARE is taking from this project to all others are 1) the importance of developing a
functional monitoring system very early in the project, which includes self-evaluations to assure
the project keeps on track, and 2) the importance of devising a way to fully engage the MOH
partner from the beginning. While these are standard in other CARE CS projects, their lack in
this case has served to reaffirm their importance.
A critical lesson for CARE HQ is assigning the appropriate backstop person. Language skills and
a willingness to accept the reality of the context are essential.
A lesson-learned, not discussed by the external evaluator, has to do with the use of community
councils as a strategy for health education and referrals. While forming and training the councils
did leave a large body of individuals in each community with improved health knowledge, the
process proved to be labor-intensive and not agile. Training one or two volunteers per
community is much more time-efficient and makes it possible to have centralized training since
those from various communities can be brought together. The community councils were not pre￾existing groups and it is unlikely they will continue to meet and function except where the DDS
gives them some recognition and support (Malema district). CARE will not apply this strategy
again but rather will opt for more efficient, sustainable approaches of volunteers and pre-existing
groups.
D. RESULTS HIGHLIGHT ( tear out sheet)
See Annex H
Please see map attached in hard copy
Annex B
Activities for intervention areas
Diarrhea case management:
In order to train communities in home-based treatment and referral, the project
facilitators provided training to the community-based health councils in the project’s 51
target communities on the project’s key messages concerning diarrhea. The council
members have passed on these messages through health education sessions with
community groups and through home visits when they know a child is suffering from
diarrhea. Council members do referrals to the nearest health care unit. IEC materials were
developed by the project and provided to each community council to facilitate their
education work.
Following observations that some communities were introducing secondary messages on
personal and household-level hygiene and performing laborious demonstrations on the
preparation of cereal-based solutions, the project facilitators began re-focusing the
messages with council members and in particular emphasized the importance of giving
“immediate liquids”.
At the clinical level, project supervision visits showed that the diagnosis of complicated
diarrhea is usually accurate. In September 1999, the quality care rating for diagnosing
and treating diarrhea showed that three quarters of the health personnel were performing
exactly and completely according to protocol, where one quarter was in the insufficient
range. The correct use of antibiotics was also shown to be questionable among health
care workers. The project’s nurse trainers continued to work with health post staff to
improve their clinical skills in the treatment of children with diarrhea.
Case management of malaria
The project strategy to train the communities in malaria danger signs included the use of
cue cards (drawn images). In small groups, the community councils use the cards that
show danger danger signs to watch for in malaria cases and the steps which should be
taken if these signs are present. They also use a flip chart storyboard on malaria, which
was developed by the project. Since December 1998 the two theater groups have been
presenting a play on malaria, touring the different project communities. An audio play
was produced and distributed on cassettes in May 1999.
Another important aspect of the fight against malaria was the promotion and sales of
mosquito nets. The project developed a a strategy for introducing mosquito bednets on a
small scale in order to test reception and behavioral compliance. A simple, non-scientific
survey on net use and demand was drawn up and conducted by key members of the
health councils in selected communities. The survey was also a promotional opportunity
as each member of the team had a bednet to display and demonstrate its use. Results of
the survey indicated that only 3 out of 600 people owned a net. Only 32% had actually
seen one or knew what its purpose was.
The project proceeded with plans to develop educational messages on the proper use and
maintenance of the nets, as well as promotional messages via radio and flyers. In
November 1999, a commercial vendor put a stock of 365 untreated mosquito nets at the
disposal of the project to organize sales in the communities. Community councils were
then trained in the use and maintenance of the nets, and together with the facilitators in
each district, set up net sales days. Though early sales were low (the project eventually
had to subsidize the price), over 340 nets had been sold by the end of April 2000. Radio
promotion of nets was so successful that independent vendors began bringing in nets to
sell in the district-level markets. The project maintained close contact with Population
Services International (PSI) throughout this experience, and in May 2000 PSI approved
the idea of making available its supply from neighboring Zambezia Province. The project
decided not to manage this stock, however, but to promote a price list and display sample
nets via the facilitators in all district stores and marketplaces.
Case management of acute lower respiratory infections(ALRI)
Once again, the community councils were in the forefront of carrying the message to
their communities.
CS-12 project had a major role in reinforcing technical skills of the health personnel in
the diagnosis and treatment of ARI. Drug management is a key factor in the effective
treatment of respiratory infections. Many health personnel exhaust their limited supply
of antibiotics on unnecessary treatments, leaving nothing for the child who really needs
cotrimoxazole. In the final year of the project, a series of workshops for APEs
(community health workers) on management, supervision and community participation
emphasized the need for better drug management and administration. A total of 46 health
personnel and 56 APEs from all three districts participated in the week–long training
events. In two districts, members of the District Health Department (DDS) participated as
trainers during the exercises for Rational Drug Use. At the end of the training,
participants showed greater understanding of the relationship between symptoms,
diagnosis and treatment; demonstrated improvement in planning skills and supervision;
and expressed visible enthusiasm for the community referral system and support provided
by community councils. Thermometers, wall clocks for respiratory counts, baby￾weighing scales and reference books were provided to health facilities.
Maternal health
The project strategy to improve maternal health was comprised of four parts. They were
1) Reinforcement of TBAs,
2) Introduction a community council referral system
3) Development of a community based emergency transport system and
4) Rehabilitation and providing of equipment for maternal care units.
1) As an integral part of improving maternal health in the communities, a total of 91
Traditional Birth Attendants (TBAs) were trained by the project over the course of two
years. TBAs living in communities with health councils are members of those councils
and all TBAs trained by the project are equipped with a birthing kit and audio-visual
materials.
Not withstanding the controversy over the contents of the TBA birth kits, with a verbal
nod from the Maternal Health Division at the Provincial Health Department (DPS), the
project went ahead discreetly developing, organizing and distributing kits. Two other
Non-Governmental Organizations (NGOs) also carrying out TBA training obtained kits
at cost from the CARE project. All kits included razor blades and not scissors since
scissors are more easily contaminated due to negligent hygiene practices and are more
difficult for the TBAs to manipulate. The supply problem for razor blades was addressed
through the promotion of Birth Plans and via the community councils. The Birth Plan
calls for the purchase of a new, unused razor blade by the expectant family.
2) A color-coded reference system that was introduced by the project has had
considerable success at the community level. It was designed in consort with the 3-part
reference form for council members that are literate. Council members use them in their
meetings and home visits to identify potential danger signs for childhood diseases and
pregnancy. The colored cards are coded by illness and are acknowledged at the health
and APE posts as legitimate documents. The green card is for pregnancy referrals, which
are always referred through the APE on to a post or health center. APEs have no mandate
to do pre-natal care, despite receiving iron-folate tablets in the Kit C.
3) At the community level, the community councils stressed the importance of pre-natal
consultations and birth plans. In an effort to address the problem of limited transport for
emergency obstetric care, the project introduced a component on planning an emergency
transport system. The activities include, but are not limited to, the subsidized purchase of
a locally manufactured bicycle ambulance and a distress flag to attach to a bicycle or a
local stretcher. A theater play and audio play were produced to promote medical
emergency planning, and in an innovative attempt to collaborate with long-distance taxi
drivers, the state driving manual was reproduced with the project messages included on
each page for free distribution. A radio message campaign targeting the taxi drivers and
police control posts urged them to assist with any medical emergencies they may come
across on the road.
4) The project conducted a physical assessment and inventory of all 23 health facilities in
the three districts, comparing this against the level of training of the health personnel
posted at these units in order to learn what kind of equipment they were trained and
authorized to use. The MOH equipment catalogues for the posts and centers were
consulted as well, and out of this a plan was drawn up and executed to carry out basic
rehabilitation to facilities and to supply basic equipment. A complete list of material and
equipment is in (Annex E).
Annex C
Formative Research
2.c.1 Rapid Assessment of IEC Impact in Project Communities: 15-22 April 1999
The project conducted a rapid assessment of selected communities that have active
councils, in order to get an impact reading on how well the project messages are being
transmitted and applied. The new field staff conducted individual and group interviews at
4 health facilities, and 5 communities. Results were encouraging and also indicated areas
that needed more attention. Some highlights:
• Many mothers know danger signs and action messages
• In one health facility, the agent did not counsel on proper drug use and
management, due to limited time. (The pharmacy clerk did provide some
explanation when providing the medicine, however.)
• Some health facilities are poorly organized in terms of patient comfort, patient
flow, cleanliness, and materials management.
Based on the results, three principal actions were undertaken:
• Focus groups with health personnel to explore their attitudes and practices
regarding client care and general work management
• In-service training sessions that will address problems of attitude, time
management, and basic organizational practices
• Community health councils were encouraged to use a wider variety of sites for
education - including markets, water points, etc.
2.c.2 Community-Based Opinion Survey: August-September 1999
CARE staff conducted interviews with DDS personnel and community councils, and
conducted focus groups with residents of nine communities to determine their interstes
and perceived needs in relation to the current project and a possible follow-on. Malaria
and diarrhea were acknowledged as the major illnesses, affecting both adults and
children. Health care personnel had moderately good knowledge related to treatment of
malaria (due to CSXII training). Community members felt curative services were needed
for nutritional problems and showed no conception of prevention of such. In the
communities where health councils were functioning, the residents were appreciative of
their efforts.
2.c.3 Rapid Market Research for Mosquito Net Availability: Jan.-Mar 1999
One of the project objective states: “80% of families in 3 communities will participate in
a pilot mosquito net project by September 2000.” The basis for this objective was an
intended collaboration in 1998 with PSI for the introduction of treated bednets in one
project district, using a social marketing strategy. This pilot activity was to follow on
PSI’s full-scale implementation of a net marketing project in neighboring Zambezia
Province. PSI experienced major delays in its own Province, and ultimately did not
launch net sales in n Zambezia until April 2000. The CSXII project did a rapid
assessment of the larger stores in Nampula and the port city of Nacala, and learned that
there were virtually no net supplies anywhere. The occasional trader from Tanzania or
Malawi had 2 or 3 at most.
About the same time, a discrete survey was conducted in 20 project communities to
determine whether nets were an item people knew about and/or used. 600 people were
asked 9 questions, including price ranges for net purchase. The results basically
confirmed what is common knowledge: almost no one uses nets (3/600). A third of those
interviewed did not know what a net was nor what it was used for (interviewers had
sample nets on hand).
Based on this and considering that the project’s structure was not suitable for a net
marketing activity, one option was to work within the existing commercial market
system, persuading local merchants to stock and distribute quantities of nets through their
district stores. In order to encourage them, the project would undertake promotional
activities: radio spots announcing net sales in specific stores; banners at store sites, flyers
at the community level, and demonstrations.
However, merchants resisted bringing in a net supply through their own channels and at
their own expense. The majority did not feel that there was a market for mosquito nets;
the procurement and import procedures were too complex for the small quantities in
question (500-1000); and there was not enough of a profit margin to make it worth their
efforts. Eventually, one large merchant in Nampula informed the project that they had
purchased a supply of 365 nets shipped up from Maputo. The merchant put the supply at
CARE’s disposition, to sell at the community level. The community councils assisted the
project field agents to set up Net Sales Days, and the promotional package was
implemented. All proceeds returned to the merchant. Due largely to the promotional
efforts of the project, nets started appearing at district level markets and among the stock
of petty traders. Apart from the health benefits, the project can take some credit for
opening up the local markets to a “new” product.
As a final activity, the project returned to the independent merchants that had participated
in the preliminary market study with a list of the PSI net products and prices. Samples
were shown, and contacts were made in the event that the merchants would like to
purchase a stock. In addition, the list of merchants and prices were distributed to CARE’s
other projects that have field activities, notably farmers’ associations and credit/savings
groups, in the hope that they will find it beneficial to develop a net sales activity.
2.c.4 Focus Groups on Obstetric Emergencies and Transport: May 1999
Obstetric emergencies often result in death or permanent physical injury to a pregnant
woman, due to delayed intervention for lack of transport to the nearest capable facility.
CSXII project had a mandate to investigate and promote emergency obstetric transport
plans in its target communities as an accompaniment to its Maternal Health activities for
safe childbirth The first phase of this activity consisted of a number of tasks:
• gather information concerning local beliefs on what is considered an emergency, and
how people currently go about addressing emergencies. References for this
information will be local leaders, traditional midwives, and community health
workers;
• focus groups conducted with bush taxi drivers at park stations in the district and
Nampula city to elicit their opinions and ideas about instituting an emergency
notification system;
• identification and listing of all community means of transport in each target
community;
• identification and listing of distances from each community to the next nearest
community, to the nearest major road, and to the nearest health facility.
Results from this research revealed that:
• Different community members had different priorities for what constitutes an
“emergency”. For example, an elder always considered attendance at a funeral to
require emergency transport.
• Bush taxi drivers had all assisted at least once with a medical situation, and the
majority of those were obstetric emergencies. Their main obstacles to not providing
more support included complications about who would disembark to allow for the
patient, and difficulties with the police at the control stations along the road.
• All communities had bicycles owned by individuals. This would eliminate the need to
provide a bicycle in addition to a wheel cart.
• Distances to main arteries are great, and many communities use bush tracks to cut
corners. This would have impact on the kind of transport that might be proposed.
Following the research, the project began the development of action plans in the target
communities. A package of options were proposed:
• development of an emergency identification system to alert taxi drivers of a nearby
emergency;
• development of a network of bicycles or stretchers between communities;
• radio and print promotional campaign to taxi drivers and police officials.
The resulting system produced:
• a purchase scheme for a bicycle ambulance at a subsidy of 50%;
• red and yellow SOS flags distributed to every community council’
• a live theatre and audio play on obstetric emergencies;
• an adaptation of the national drivers’ manual with project messages on each page that
was distributed free of charge to all bush taxi drivers and police officials.
2.c.5 Physical Assessment and Inventory of Health Facilities: Oct.-Nov. 1999
CARE’s Child Survival XII project conducted an inventory of the physical conditions at
the project’s target health facilities in order to assess the degree to which inadequate
equipment and physical structures are impeding the timely and effective execution of
safe childbirth services delivery. The project used the WHO definitions that identify the
parameters of basic and emergency obstetric care.
The project confirmed that the Provincial Health Department (PHD) protocols conform to
the WHO standards. A list was drawn up that identified the specific health facilities that
are authorized to perform BEOC/CEOC. Next, the equipment and material catalogs for
health centers and posts were consulted to determine what corresponding equipment and
material was required in order to carry out the BEOC/CEOC functions. A review of the
organizational charts indicated which category of health personnel has the qualifications
to perform the respective services. Finally, the individual facility registers were studied
to count the number and cause of obstetric referrals, as well as the categories listed for
complicated deliveries. Neither category gave sufficient data, listing many cases as
undiagnosed. A quick comparison with category listings of personnel and their
qualifications suggested that this might be due to under qualified staff serving in place of
their superiors, who are often called away for a variety of reasons.
The inventory revealed that the majority of the health facilities are operating without the
basic equipment necessary to adequately conduct BEOC, much less CEOC. The findings
included:
• Lack of medicine cabinets and shelves to properly store medicines; most end up on
the floor;
• Lack of filing cabinets to store paperwork and health forms;
• Clients sit on the floor or on the ground outside the facility for lack of benches or
chairs;
• One brand new facility built by an NGO was not equipped with a desk or chair for the
nurses to work from;
• Some facilities do not have beds or bed linen for inpatient care;
• General lack of instrument boxes for needles, syringes, forceps, gloves, etc.;
• Some autoclaves that serve for sterilizing instruments are defective, impeding
effective sterilization of equipment;
• Some of the maternity wards do not have obstetrical exam tables or beds for birthing,
no IV poles, and no basic surgical equipment for the most basic obstetric
interventions;
• Poorly maintained patient registers, without notations for referral causes, complicated
births, or obstetric deaths;
• Poorly organized routines for maintaining stock records and conservation of
medicines, as in many cases the senior nurse keeps the keys and when absent (which
may be frequent) no one has access to medicine and material stocks;
• No efficient system for ordering and managing consumables (lanterns and petrol,
aprons and sheeting, syringes and needles, etc.) exists at the central provincial level.
These results of this inventory justified the study’s hypothesis that it was not just lack of
material and equipment that is impeding effective service, but also the inadequate
management of existing resources. Thus, simply supplying equipment and material that
will not be properly used or maintained would not resolve the problems. The District
Health Services management teams will need to work to ensure proper use and
distribution of resources, maintenance of equipment and proper recording of data.
The study made the following recommendations that were discussed with Provincial and
district level representatives:
• Provide the maternity wards with the minimum basic equipment and material
necessary for conducting effective BOEC and COEC;
• Proceed with plans to conduct in-service training sessions in management and
workspace organization at the facility level in collaboration with the DPS/DHS;
• Involve the facilities’ senior MOH staff in studying how to solve the problem of
inaccessibility to essential medicines and material when no senior staff is available;
• With the assistance of the nurse-trainers, re-organize the work environment in the
maternity wards, including storage of medicines, patient flow, paperwork
management, etc.;
• Coordinate with other donor organizations operating in the districts to share the
equipment and material costs for the facilities.
Based on this, the project proceeded with plans to rehabilitate and equip the maternity
units in health centers and posts. Contractors were identified and materials supplied to 22
different health facilities.
2.c.6 Client and Clinic Management focus group discussions and individual
interviews: March –May 2000
This exercise began with the hypothesis that technical training and material/equipment
support were not enough to assure delivery of quality health services. The critical missing
factor appeared to be poor morale and behavior-based performance management. In an
effort to identify outstanding obstacles (apart from technical competence and material
support) to improved service delivery, it was important to listen to the health personnel
themselves, and to understand the difficulties and barriers they face. Both health projects
collaborated on the effort. Focus Group Discussions (FGD) and individual interviews (II)
were conducted at the health centers in March 2000. Prior to initiating this activity,
project staff selected to conduct the discussions participated in a 2-day training to review
and learn the specific techniques involved. Three FGDs were held with 5 participants in
each District. The experience level and professional capacity of the participants varied,
allowing for a more interesting range of perspective and opinion on the topics discussed.
21 individual interviews were conducted across a range of job levels.
Results identified an number of trends. Notions of time management and work
organization were practically non-existent, as personnel responded to the task at hand.
Mistrust and resentment were prevalent between clinic staff and their clients. There
appeared to be a significant information gap between what the health personnel expected
from clients and vice versa. In addition, a significant number of personnel appeared to be
performing duties for which they have no training. The results were finalized and shared
with the DPS and DDS, following which a variety of interventions were initiated:
Annex D
Results – Cross Cutting approaches
(ii) Communication for Behavior Change
ii.a. The project IEC strategy
The project IEC strategy was developed based on the premise that certain harmful
behaviors contributed to an undesirable state of health and welfare. During the formative
research activities, desirable behaviors were identified, and the process for moving from
harmful to helpful behavior was outlined:
• Justify the desired behavior with authoritative supporting information and the
promised benefits;
• model the new behaviors; initially through surrogate characters as is possible in
drama and visual aids and eventually in the form of local model mothers, for
example;
• carefully identify the steps in the behavior change and ensure they are feasible to
accomplish;
• repeat the desired behaviors (multiple exposures to media modeling behaviors);
• correct and encourage new, desired behaviors;
• repeat new, desired behaviors.
ii.b Summary of project research activities
CS XII conducted a considerable amount of formative research, destined to inform both
the IEC strategies and provide direction for training needs. Following is a list of the
various research activities, and their intended use.
Type of research Intended use of results
Situation analysis – trained TBAs Baseline for training evaluation
Skills assessment of health workers:
Diagnosis/treatment of ARI
Baseline for evaluation of project objective
#4
Focus Group Interviews: pregnancy and
childbirth - mothers, fathers, grand mothers
Design of IEC materials: identifying
constraining/enabling behaviors
(ref. Behavior Change Schematic)
Focus Group Interviews: childhood
illness/ARI – mothers, fathers,
grandmothers
Design of IEC materials: identifying
constraining/enabling behaviors
(ref. Behavior Change Schematic)
“Access Mapping”; review of all health
facilities
Reference decisions on first-line antibiotic
supplies; rehabilitation of facilities
ii.c. Project Target Groups
The target populations for the project were segmented into three groups:
1. women of reproductive age and pregnant and lactating women;
2. children ages 0-5;
3. persons having household and community-level influence and decision￾making power over the above two groups.
ii.d. Choice of Channels and Message Approach
Taking a cue from the success of the “Enter-Educate”*
 approach, the media that was used
for the messages included: live theater performances; audio cassette presentations for
listening clubs; print storyboards in flip chart form, and motivational materials kits for the
field agents.
The project employed the “Near Peer” model, whereby the principle characters in the
media materials are drawn from a profile of the typical target group member, but
enhanced with project desired behaviors. Target population members aspire to be like the
“Near Peer”, and can see the real possibility of attaining this status.
ii.e Project IEC Activities
ii.f.1 Short term promotional campaigns
 Model Mother campaigns were designed to reinforce the modeling behavior promoted
through the theater, audio programs and visual aids. The community councils identified
the mothers, organized the ceremonies, and presented certificates. The mothers’ names
were announced on the radio. A Well-Baby photo contest was planned but due to time
and material constraints, this was not executed. The Community Self-Image Campaigns
were not developed for the same reasons.
Mosquito Net sales campaigns were held in every community. These amounted to a day
planned by the local health council to demonstrate how to use and to sell nets. The
project provided banners and flyers, and buckets. Radio messages ran for two months
advertising the sales days. Beyond the short-term publicity run, the nets continued to be
delivered and sold.
A campaign to promote and sell bicycle ambulances was also launched. This involved a
radio promotion over three months, demonstrations of the bicycle in each community, a
drivers’ manual for taxi drivers and police that included project messages and encouraged
these groups to assist during road emergencies.
ii.e.2 Mobile theatre groups
 
*
 Johns Hopkins University/Population Communication Services , Baltimore, Maryland.
The project organized two theatre groups to conduct mobile performances throughout
the three districts. Each group had 5-7 performers who were trained in both educational
theatre techniques and a repertoire of 6 scenarios. A “soap opera” format was adopted,
and the same characters appeared in different situations in each scenario. This medium
proved to be extremely popular, although expensive. Five training events were conducted
over two years, and performance costs included transport, meal and lodging costs and
minimal costumes. Each training cost around $700, and one tour for four days cost
approximately $180. Between the two health projects, 96 presentations were made, to a
total of 18,972 spectators.
ii.e.3 Audio Listening Clubs
The theatre performances were recorded professionally in the studios of Radio
Mozambique, and duplicated on to audio cassettes. Cassette players were provided, one
to each community. The council members organized groups of listeners, relying on
existing social, religious, co-op, etc. networks. After a listening session, the council
members hosted a discussion. The radio cassette activity was the most performed activity,
although the notion of an established “club” never really caught on.
ii.e.4 Message Cue Cards and Discussion Starters and Card Games for ORS
For the cue cards, project key messages were printed individually on 8x11 paper with
key data and supporting information to justify the message. These cards were intended to
reinforce the educational activities and knowledge base of the project facilitators when
answering questions from council members.
The discussion starters were 8x5 cards with open-ended questions; corresponding
message and tips for answers on the other side. The ORS card games were a set of 8x5
laminated cards detailing the ORS mixing procedure in pictures only. For use with small
groups While all products were very useful tools, it was not monitored adequately to
determine how they were used.
ii.e.5 Storyboards or Flip Charts
 The storyboards were flip charts with pictures and text of the same story displayed in the
theatre and audio cassettes. This was the third media channel by which the messages
were reinforced. They were used constantly. One drawback was that they were written in
Portuguese. However, it has been observed that while everyone speaks Macua language,
very few people can read it, and those that are literate tend to be literate in Portuguese.
People enjoyed the visual aspect of the drawings. Due to the high number of storyboard
productions to be made, the quality of composition was modest at best – black and white
line drawings on card stock. The contracted artist was not very accomplished.
ii.e.6 Radio broadcasts
Radio broadcasts were not written into the initial strategy. The 1997 DHS statistics
reported only 15% of the population had access to radio, and considering the broadcast
costs at the time, it was not judged very cost-effective. However, in the last six months
of the project the entire series of audio cassettes were broadcast in an effort to be able to
evaluate increased listening capacity. The final KPC survey revealed that 62% of all
those interviewed had heard the theatre and promotional messages.
All in all, the IEC materials provided a variety of approaches for council members to
educate and mobilize their target audience and hold their interest. Making the most of the
products depended heavily on how the council members were instructed in their use, and
the opportunities they were able to create for message dissemination.
Annex E
List of Material and Equipment Provided to Health Facilities
Material/Equipment Level Year
WHO Algorithm maps and ARI
wall clocks
 Health Posts 1997-98
WHO diarrhea algorithm maps Health posts/APEs 1997-98
Radio Cassette players Community councils 1998-2000
Set of 5 audio cassette plays Community councils 1998-2000
Set of 5 flip chart plays Community councils 1998-2000
Set of 2 flash card games Community councils 1998-2000
Rational Drug Use Guide Posts/APEs 1999
Medical Registers/rulers APEs 1999
Waiting room benches Health Centers/Posts April-June 2000
Neo-natal tables Health Centers/Posts April-June 2000
OB/GYN step ladder Health Centers April-June 2000
Medical storage cabinets Health Centers May 2000
Buckets, waste cans Health Posts May 2000
Anemia Technical Manual Posts June 2000
TBA kits Community TBA (91) 1999-2000
“Where there is No Doctor”
books
Community APE (60) July 2000
Thermometers Community APE (120) July 2000
Baby-Weighing (SALTAR)
scales
Community APE (60) July 2000
OB/GYN exam tables Health Centers (3) July 2000
Manual Vacuum Aspirators Health Centers/Posts (5) July 2000
Adult scales Health Centers/Posts(5) July 2000
Baby scales (tray style) Health Centers/Posts (5) July 2000
Manual steam sterilizers Health Centers/Posts (8) July 2000
Bed pans Health Centers/Posts (9) July 2000
Instrument Boxes (various sizes) Health Centers/Posts (44) July 2000
Kidney pans and lotion bowls Health Centers/Posts July 2000
Ophthamaloscope Malema Health Center July 2000
Annex F
MATERNAL AND CHILD HEALTH PROJECT – Training Events 1997-2000
Date Participants
#/level Institution
Title and Content Days Trainers Locale
1. Aug 97 3 trainers
28 health staff
CARE
NGO &
MOH/DPS
Training of Trainers 5 Asst PM; 2 DPS Nampula
2. Nov 97 –
Apr 98
67 EMCH, EE,
AMP, AMC
MOH-DDS Sequential Management of
childhood illnesses
12 Asst PM; Field
Coor., 3 CARE
trainers; 3 MOH
trainers
3 districts, 2 tours
per district
3. March 98 3 nurse trainers
3 facilitators
CARE Participatory Learning
Assessment
5 PM; Field Coord.;
local consultant
Nampula
4. April 98 3 nurse trainers
3 facilitators
CARE Community Organisation 5 PM, Field Coord.,
Asst PM with local
consultant
Nampula
5. April 98 3 nurses
3 AMP
21 EE
DDS – 3
Districts
Sensitization about
Community Organizing
1 Field Coord.,
Asst.PM, 3
facilitators
3 Districts
6. May 98 3 facilitators CARE TOT for content of CSXII
messages
5 Asst PM; Field
Coord., DPS & DDS
Nampula
7. May 98 –
ongoing
Community
Leaders in 51
communities
Community
Councils
Training Package for
Council Health Education
Activities
2 months;
1 session
per week
Field Coord.; 3
Facilitators, 3 Nurse
trainers
51 community
councils
8. June 98 12 PMs, nurses DDS, 6 PVOs TOT in obstetric first aid to
teach TBAs
3 Marylou Carr,
ACNM
Nampula
EMCH = nurse; EE = elementary nurse; AMP, AMC = health technicians; PM = project managers
Date Participants
#/level Institution
Title and Content Days Trainers Locale
9. July 98 46 APEs, first aid
workers, servants
Communities in 3
districts
Sequential management of
childhood illnesses
5 Mr. Sumalgy/DPS; Asst.
PM, 3 CARE nurse
trainers
3 districts (5
tours)
10. July 98,
March 99,
June 99,
Dec.99
March 2000
17 community
theatre actors
Namina,
Metacusse
councils, Casa
Velha Cultural
Center
Basic techniques in using
theatre for health
communications; rehearsals
for 5 plays
10 days
each
Performance artists from
Casa Velha and Casa
Cultural
Nampula
11. Sept. 98 19 District & field
staff
CARE, 3 NGOs,
DDS, DPS
Supervision techniques –
planning
5 Asst. PM; DDS Ribaue Malema
12. Oct. 98 10 CARE, DDS Supervision techniques –
application
4 Asst. PM; DDS Ribaue Ribaue
13. Nov. 98 16 EMCHs,
midwives and
nurses
CARE,
MOH/Maputo,
DPS/DDS
TOT for provincial and district
level nurses to train TBAs 15
National Director
MOH/MCH;
Pathfinder, SALAMA
Malema
14. April .99
 May 99, July 99,
August 99
March 2000, May
2000, June 2000
August 2000
91 Traditional Birth
Attendants
CARE,
communities with
councils
4 week training in Basic
techniques for TBAs; using
MOH national curriculum and
supplemented by project
guidelines on community
involvement, emergency
transport, and developing a
birth plan.
28 days 3 Project nurse trainers; 4
district MCH nurses
(training included Lalaua
district)
4 District
health centers
15. March –July
2000
53 community
council leaders
CARE,
community
councils
Council to council peer
education training
3 days
each
CARE, council members 3 districts
16. March 99 15 project staff CARE Project retreat for performance
standards, CARE policies, and
work ethics
3 days Project Manager Nampula
Date Participants
#/level Institution
Title and Content Days Trainers Locale
17. April 99,
May 99
34 EE, EB, AMG
CARE,
DDS field
personnel
Refresher training for field
personnel in diagnosis and
treatment of specific ARI
illnesses
3 days each DDS, Project nurse
trainers, consultant
Each of 4 Districts
(including Lalaua)
18. April 99 3 project
facilitators
1 project nurse
trainer
CARE Orientation and IEC training for
staff new hires (field agents)
4 days Project Manager Nampula
19. June 99,
July 99, August
99
48 EE, EB, AMG,
pharmacy clerks,
some APEs
CARE, DDS
field personnel
Orientation to DPS new system
of drug distribution and inventory
management
3 days each DPS Drug
Distribution Unit
Each of 4 Districts
20. Dec. 99 3 project nurse
trainers
6 project
facilitators
2 project
management
CARE
TOT in community development
and IEC training
17 PM, Assistant PM,
Field Coord.,
consultant
Nampula
21. Feb/March
2000 56 APEs and first
aid workers
CARE, DDS Refresher training for APEs in
organizational management,
community participation
3 days each Assist. PM, Nurse
trainers,
facilitators
3 districts
22. March 2000 9 project staff CARE – CSXII
project and RH
project
Training in Focus Group
Discussion techniques
2 days Project Managers,
Assist. PMs
Nampula
23. March –
July 2000
53 council leaders Care,
community
councils
Peer Training – Council to
Council
2 days each Field Coord.,
Facilitators
X communities in
3 districts
Annex G
List of external technical assistance
Date Source Activity
Jan￾Feb.97
Unuversity Alabama Intern Design & implementation of
baseline survey
March
May 98
BASICS Concepts & techniques of
IEC
May 98 Consultant ( Marydean Purves) Designing IEC strategy &
developing supporting
material
May 98 CARE Atlanta sponsored workshop in
Nicarauga
Child survival workshop on
supervision with other child
survival managers
(Assistant project manager
and a DDS partner
participated)
May 98 HAI (Mozambican NGO from Manica) Sharing experience in
community organization
June 98 American College of Nurse-midwives
(consultant) Mary Lou Carr
TBA curriculum & training
July
98
Local consultant Training 13 rural
Mozambicans in using basic
theater techniques
Sep 98 Mary Ruth Horner (External evaluator) Mid-term evaluation
Sep 00 Sunny Wijesinghe (External evaluator) Final evaluation
ANNEX H
Results Highlights: “Theatre as a Behavior Change Tool”
Taking a cue from the success of the “Enter-Educate”*
 approach, one of the media
channels chosen by the project included establishing mobile theatre groups to present live
theater performances. Theatre in Mozambique is universally popular and used for
education and entertainment alike; by government and public sector as well as the private
commercial sector. The project deliberately decided not to work with a professional
group, choosing instead to host contests in the target districts to select willing volunteers.
Two groups were thus organized, and participated in a highly successful first workshop in
July 1998. A renowned local performance artist transformed the rural inhabitants into a
tightly knit team that continued throughout the two years of the project to develop both
professionally and personally. In large part, the overwhelming success of the theatre
interventions was due to the cohesiveness and dedication of the two groups. Each group
had five to seven performers who were trained in both educational theatre techniques and
a repertoire of six scenarios. A “soap opera” format was adopted, and the same characters
appeared in different situations in each scenario.
The “Near Peer”*
 model was employed whereby the principle characters in the play were
drawn from a profile of the typical target group member, but enhanced with project
desired behaviors. Project qualitative research was consulted for this activity, as well as
field visits to talk with some community people to discuss traits and personalities of
people they admire. Target population members aspire to be like the “Near Peer”, and can
see the real possibility of changing their behavior.
Five training events were conducted over two years, and performance costs included
transport, meal and lodging costs and minimal costumes. Each training cost around $700,
and one tour for four days cost approximately $180. Between the two health projects, 96
presentations were made, to a total of 18,972 spectators.
The Reproductive Health project conducted an exit interview activity at the end of one
performance. Ten simple questions were asked regarding the key messages and the
spectator’s impression of the show. Accuracy on the message retention was 90%, and
people were extremely enthusiastic about the spectacle.
It is the opinion of the project management that one of the benefits for the project is that
the group members themselves have become very adept at communication, and involved
in the welfare of their own community and of those that they visit.
While generally IEC promotional activities such as this are doomed as unsustainable,
there is optimism for these groups. Already they have the sponsorship of the
Reproductive Health project. The nutrition component of CARE’s agriculture program
 
*
 Johns Hopkins University/Population Communication Services , Baltimore, Maryland.
*
 Academy for Educational Development, Washington DC.
will be linking with the groups to insert another show into their repertoire. The Provincial
Health Department and the District Health Services have asked for their contacts and the
terms of their work, so they too may continue to support them.
Annex I
Evaluation Team Members
Azevedo, Amadu Field facilitaor in Malema (CARE staff)
Bila, Ilda Nurse Trainer in Mecuburi (CARE staff)
Braganca, Mucalava District Health Director of Malema (DDS)
Cusse, Augusto saide Field facilitaor in Ribaue (CARE staff)
Chaimite, Maria de Fatima Nurse Trainer in Malema ( CARE staff)
Francesa, Pedro Driver (CARE staff)
Gil, Gorge Patricio Field facilitaor in Ribaue (CARE staff)
Jose, Fernando Assistant Project Manager (CARE staff)
Leopoldo, Alberto Field facilitaor in Malema (CARE staff)
Marentes, Jose Driver ( also interviewer/interpreter) CARE staff
McNulty, Judiann Deputy Director for Child Health (CARE Atlanta )
Mecedes, Marcilino Field facilitaor in Mecuburi (CARE staff)
Purves, Marydean Project Manager (CARE staff)
Sumalgy, Alberto District Health Director of Mecuburi (DDS)
Wijesinghe, Sunny External Evaluator (Independent)
Annex J
Assessment methodology and schedule
The evaluation team consisted of the CS-12 project manager, CARE atlanta representative,
external evaluator, CS-12 project staff and DDS directors of Malema and Ribaue. DDS
director from Ribaue was unable to attend. The names of the evaluation members are given in
Annex I.
The selection of communities included, both, who showed rapid progress, and those who
lagged behind in health promotion activites. The distance also played a major role in chosing
the communities to visit during a three-day field visit.
Interviews conducted during the three day-visit were with the following personnel:
• Seven personnel of health facilities ( either nurses or APEs)
• Six TBAs
• Six community councils (as collective groups)
• Fifteen members of community councils from different communities
• Ten mothers & farthers from different communities
• Two DDS directors
• Nine CS-12 staff members
• One APM
• One theater group
Each DDS director visited a district other than his own, and took an activie part in
interviewing councils, council members, or mothers and fathers. Field agents (facilitators )
accompanied the team to the communities where they worked, but assisted in interviews in
communities other than their own to avoid bias in answers. The external evaluator conducted
the majority of the interviews including those with the two DDS directors and nine CS-12
staff.
The questionnaires utilized in the interviews are given in Annex K.
The schedule of the interviews and visits follow.
Schedule of final Evaluation
Date Team
Members/participants
Event
9/7-9/9 Sunny Wijesinghe Review reports
9/11
7.30 –10
a.m
2.00 p.m.
7.00 p.m
Sunny Wijesinghe
Judiann McNulty
CARE staff ( project director,
Assitant project manager, field
coordinator, and field staff) 4 DPS
members, 2 World Vision staff &
Sunny Wijesinghe
Judiann McNulty
Sunny Wijesinghe
Judiann McNulty
Travel to Nampula
Presentation of program interventions by
CARE staff
Dinner-discussion with Judith, project
manager of CARE family planning project
9/12
8-12 a.m
2.00-5.00
p.m.
Judiann McNulty
Marydean Purves
Sunny Wijesinghe
Sunny Wijesinghe
Planning for field trip/logistics
Preparing interview questions
Intervew with Eunice Queiroz about field
work/supervision
9/13
a.m.
p.m.
Judiann McNulty
Marydean Purves
Sunny Wijesinghe
DDS director Ribaue
Malema CS-12 staff
Sunny Wijesinghe
Murumbo: intervew CCs, APE, mothers &
fathers
Metacusse: view theater presentation &
interview thearter group
Interview with Malema CS-12 staff
Interview with DDS director of Malema
9/14
a.m.
p.m.
Judiann McNulty
Marydean Purves
Sunny Wijesinghe
DDS director Ribaue
Malema CS-12 staff
Sunny Wijesinghe
Chilulu: Interview health facility personnel
Intacasse: interview TBA, APE, mothers &
fathers
Interview with MCH nurse at Malema
health center
9/15
a.m.
Judiann McNulty
Marydean Purves
Sunny Wijesinghe
DDS director of Ribaue
DDS director of Malema
Ribaue CS-12 staff
Sunny Wijesinghe
Mavil II: interview with CCs, mothers &
fathers
Matharia: interview with APE, nurse￾midwife
Chica: Interview CCs, health facility, TBA
personnel, mothers & fathers
Nauconhua: visit health post & interview
Interview Ribaue CS-12 staff
Date Team Members Event
9/16 Judiann McNulty
Marydean Purves
Sunny Wijesinghe
Discuss KPC results and other issues re:
evaluation guideline
9/17 Sunny Wijesinghe Interview with project manager
Analysis of data
9/18
a.m.
p.m.
Sunny Wijesinghe
Fernando Jose
Sunny Wijesinghe
Interview with DDS director in Mecuburi
Metapua: Interview with TBA, APE, CCs,
mothers & fathers
Marririmu: Interview CC, mothers &fathers
Melule: interview with TBA, visit health
facility
Interview CS-12 Mecuburi staff
9/19 Sunny Wijesinghe
Fernando Jose
Ribaue: interview with MCH nurse, AMP,
visit hospital & maternity
Travel to Nampula
9/20 Sunny Wijesinghe Analysis of data gathered, prepare final
questions to staff members in Nampula and
DPS
9/21
a.m
p.m.
Sunny Wijesinghe
Sunny Wijesinghe
Marydean Purves
Sunny Wijesinghe
Marydean Purves
Nampula: Interview with Joana pedro,
nursr-trainer of Ribaue
Interview with Beatrice Biaz, DPS
coordinator community education
Interview with DPS director, Dr. Augstino
Sunny Wijesinghe departs for Maputo
9/28
9 a.m.
9/22-9/30
Sunny Wijesinghe Interview with Marily, Assistant Country
Director in Maputo
Data analysis & writing the report
Annex K
English translation of questionaires used in evaluation
Questionnaire for community residents (mothers & fathers)
1) Do you know a member of the community health council in this village?
2) What do community members do in this community?
3) Did a member already visit your family? If yes, how many times have they come to
your place?
4) Do you feel comfortable with a council member?
5) Did you learn any thing new from a council member?
6) Was this information useful? Yes/ No ?
7) Do you do some thing different now after she/he talked to you?
8) If yes, please give an example.
9) Do you know that these community council members are not paid for their work?
10) Why do you think they want to work for the community without any compensation?
11) Would you like to see them continue their work after the CARE project leaves?
12) Do you think the community could do some thing to keep these members motivated?
13) Explain what they could do?
English translation of questionaires used in evaluation
Questionnaire for the community council ( as a group)
1) What are the objectives of this project?
2) What are the resposibilities of this council?
3) Do you participate in other community activities in addition to the duties of the
council?
4) Please explain how the emergency transport system function in this community
5) Did you already use the referral system in the community? If yes, did the referred
bring back the paper portion that shows counter reference?
6) Will you continue your work in the community when CARE project leaves?
7) What can the community do to motivate you to continue work?
8) What can district health department do to motivate you to continue to work?
Questionnaire for traditional birth attendants
1) Did you receive training or retraining during the last two years?
2) What did you learn from this training?
3) How did this training help you in your work?
4) Who supervises your work? How often?
5) Did you receive a KIT?
6) How does the KIT help you in your work?
7) What material in the KIT is the most useful ?
8) What do you do when items of the KIT are finished or lost?
9) Do families buy a new razor-blade for births?
10) Do you have a link to the nearest health post or APE?
11) Would you like to have more training? If yes, in what area?
12) What are the danger signs during delivery that prompt you to send a woman to the
hospital?
Questionnaire for individual health council members
1) What are the objectives of this project?
2) What are your resposibilities as a council member?
3) What is the area in which you work? ( for example how many families do you work
with?)
4) How were you selected to be a council member?
5) Do you think that your training prepared you adequately to do this job?
6) How do you plan your work?
7) Please explain how you deliver messages in the community
8) Do you use the audi-visual methods?
9) If yes, were these methods useful? Explain how they are useful or why they are not
useful.
10) What other material would have being useful in teaching in the village?
11) How do you keep track of your work in the community?
12) Has somebody come to supervise your work?, if yes how often?
13) Please explain how he/she supervises you?
14) How does this supervision affect your work? Do you feel motivated by someone
observing your work, or do you work better when you are left alone?
15) How do you use the referral system?
16) How many times did you use this system so far?
17) What are the benefits of being a community council member?
18) What are the disadvantages of being a community council member?
19) For those council members who are also TBAs or traditional healers, how does being a
council member affect your work?
20) Please explain how the emergency transport system works in this community?
21) Are some of the members also part of the family planning program (i.e ACSR Agents
Comunitaria de Saude Reproductiva)? If so, don‘t others too have interest in
participating in the program?
22) Do you think the behavior of the community changed because of the Child Survival
project?
23) What motivated you to become a council member?
24) When the project has ended, would you still be working in the community?
25) What could the project have done to enhance your motivation?
26) What could the District Health Department have do to motivate you ?
27) What could your community do to motivate you?
Questionnaire for the personnel at the health facilities
1) What is your defined area (catchment area) of work?
2) How many people consult you daily (approximate number)?
3) How do you maintain your statistics?
4) What kind of a register do you use?
5) What Kit (A, B, or C) do you use?
6) What sort of training /retrainig did you receive?
7) Which training was very useful to you?
8) Training in what other area will be useful to you?
9) What recommendations would you give regarding training health personnel?
Questionnaire for Directors of District Health Departments
1) Do you know the objectives of the CS-12 program?
2) How did this program improve the Mother & Child Health Service?
3) How did this program help prevent Malaria?
4) How did this program help control and treat diarrhea?
5) How did this program help treat ARI?
6) How do you think the supervision (done at health posts) by CARE staff was helpful?
7) Do you have evidence that many more people use health services now, after CARE
has worked in communities?
8) How does the existence of community councils help the health care system?
9) Have you heard about emergency transport system in villages? If yes, how does this
help the work of DDS, and community?
10) Do you think the councils will continue their work once the project is gone?
11) What can the DDS do to motivate these councils to continue to work?
12) How do you characterize the relationship between the DDS and CARE along the life
of the project?
13) What is the future of the APE and TBA programs?