Food and Nutrition Technical Assistance (FANTA) Project
Academy for Educational Development 1825 Connecticut Ave., NW Washington, DC 20009-5721
 Tel: 202-884-8000 Fax: 202-884-8432 E-mail: fanta@aed.org Website: www.fantaproject.org
An Operations Evaluation of
World Vision’s Maternal and
Child Health and Nutrition
Program in Central Plateau,
Haiti
Cornelia Loechl, Purnima Menon,
Marie T. Ruel, and Gretel Pelto
February 2004
This publication was made possible
through the support provided to the Food
and Nutrition Technical Assistance
(FANTA) Project by the Office of Health
and Nutrition of the Bureau for Global
Health and the Office of Food for Peace
of the Bureau for Democracy,
Governance and Humanitarian Assistance
at the U.S. Agency for International
Development, under terms of
Cooperative Agreement No. HRN-A-00-
98-00046-00 awarded to the Academy for
Educational Development (AED). The
opinions expressed herein are those of the
authors and do not necessarily reflect the
views of the U.S. Agency for
International Development. Financial
support for this research is also provided
by the Government of Germany, World
Vision-Haiti, and the World Food
Programme.
Published February 2004
Recommended citation:
Loechl, Cornelia, Purnima Menon, Marie
T. Ruel, and Gretel Pelto. An Operations
Evaluation of World Vision’s Maternal
and Child Health and Nutrition Program
in Central Plateau, Haiti. Washington,
D.C.: Food and Nutrition Technical
Assistance Project, Academy for
Educational Development, 2004.
Copies of the publication can be
obtained from:
FANTA Project
Academy for Educational Development
1825 Connecticut Avenue, NW
Washington, D.C. 20009-5721
Tel: 202-884-8000
Fax: 202-884-8432
Email: fanta@aed.org
Website: www.fantaproject.org
ACKNOWLEDGEMENTS
The authors wish to thank all of those who made it possible to conduct this operations research
and produce this report.
The field costs for the research were supported by World Vision Haiti and we are thankful for
their commitment to this evaluation. In particular, we thank Frank Williams, the Director of
Operations, Bekele Hankebo, the Director of the Development Activity Program (DAP), and Dr.
Lesly Michaud, the National Health Coordinator for World Vision, for the support provided
throughout the data collection process. In the Central Plateau, the logistics and transportation for
the survey team were facilitated by Jean Robert Francoeur, the Regional Coordinator for World
Vision in Hinche; Dorlean Delince, who helped with logistical arrangements; and Wagnes Desir,
who provided administrative support. We also thank Makayi Chinyam, the Regional
Commodity Officer, for his support, particularly his help in providing transport facilities and
computer access for the survey team. Numerous other staff members at the World Vision office
in Hinche contributed in different ways to the success of this research, and we are extremely
grateful for their assistance.
The fieldwork for the operations research was made possible through the commitment of our
survey team, and we are grateful for their hard work under the particularly harsh conditions of
the Central Plateau of Haiti in the rainy season. In particular, we thank Dr. Arsène Ferrus for his
excellent preparation, coordination and supervision of the fieldwork, and Elisabeth Elysée,
Josianne Loredan, Remy Lafalaise, Mathieu Honoré and Dominique Pierre Lenz for the data
collection. We also thank Slande Celestre, an intern in the health section of World Vision at the
time of the survey, for her help with the translation of the survey instruments from English into
Creole and for her assistance during the training of the survey team. Finally, the safe transport of
our survey team was facilitated by the drivers, and we are grateful for their efforts to smooth out
our rides.
At the International Food Policy Research Institute (IFPRI), we are grateful to Wahid Quabili for
his assistance with data processing and analysis. Finally, we are extremely thankful to Mary
Arimond, both for her thoughtful comments and insight and for her excellent editing.
Our acknowledgments would be incomplete without sincere thanks to the program beneficiaries
and the World Vision health and commodity staff in the Central Plateau, who welcomed the
survey team and generously gave some of their valuable time.
ACRONYMS
ADP Area Development Program
BCC Behavior Change Communication
BF Breastfeeding
CF Complementary Feeding
CTS Commodity Tracking System
CU Cornell University
EBF Exclusive Breastfeeding
FANTA Food and Nutrition Technical Assistance
FDP Food Distribution Point
FFH Freedom from Hunger
GMP Growth Monitoring and Promotion
HH Household
IHE Institut Haitien de l’Enfance
IFPRI International Food Policy Research Institute
LAM Lactational Amenorrhea Method
MC Mothers’ Club
MCH Maternal and Child Health
ORS Oral Rehydration Salt
PVO Private Voluntary Organization
RP Rally Post
SFB Soy-Fortified Bulgur
USAID United States Agency for International Development
WSB Wheat-Soy Blend
WV World Vision
TABLE OF CONTENTS
EXECUTIVE SUMMARY ............................................................................................................. i
1. Introduction.................................................................................................................................1
1.1. Background......................................................................................................................... 1
1.2. Organization of the report................................................................................................... 1
2. The Operations Research ............................................................................................................3
2.1. Objectives ........................................................................................................................... 3
2.2. The system studied in the operations research.................................................................... 3
2.3. The key stakeholders included in the research ................................................................... 5
2.4. Methodology....................................................................................................................... 6
2.4.1. Structured observations................................................................................................ 7
2.4.1.1. Observations at Rally Posts 7
2.4.1.2. Observations at Mothers’ Clubs 8
2.4.1.3. Observations at Food Distribution Points 8
2.4.2. Structured interviews with beneficiaries...................................................................... 8
2.4.2.1. Exit interviews at Rally Posts 9
2.4.2.2. Exit interviews at Mothers’ Clubs 9
2.4.2.3. Exit interviews at Food Distribution Points 9
2.4.3. Semi-structured interviews ........................................................................................ 10
2.4.3.1. National and regional health coordinators and regional commodity officer 10
2.4.3.2. Health agents at Rally Posts, Mothers’ Clubs, and Food Distribution Points 10
2.4.3.3. Beneficiary women 10
2.4.4. Focus group discussions ............................................................................................ 11
2.4.5. Data analysis .............................................................................................................. 12
2.5. Fieldwork logistics, geographic area and sample ............................................................. 12
2.5.1. Research staff and training......................................................................................... 12
2.5.2. Geographic area and sample ...................................................................................... 13
3. Program Operations at the Rally Posts .....................................................................................15
3.1. Description of Rally Post operations according to the implementation plan.................... 15
3.2. Methods used to assess the Rally Post operations ............................................................ 16
3.3. Operations and quality of services at the Rally Posts....................................................... 16
3.3.1. Attendance and participant/staff ratio........................................................................ 17
3.3.1.1. Number of pregnant/lactating women and children under 5 years 17
3.3.1.2. Number of health staff at Rally Posts 18
3.3.2. General organization and sequencing of activities at the Rally Posts ....................... 19
3.3.3. Education sessions ..................................................................................................... 21
3.3.3.1. Implementation of education sessions and attendance 21
3.3.3.2. Quality of education at the Rally Posts 24
3.3.4. Registration, vitamin A, deworming, and ORS ......................................................... 25
3.3.4.2. Quality of vitamin A supplementation and deworming services 28
3.3.5. Growth monitoring and promotion ............................................................................ 29
3.3.5.1. Implementation of growth monitoring and promotion activities 29
3.3.5.2. Quality of the growth monitoring and promotion activities 32
3.3.6. Immunization ............................................................................................................. 36
3.3.7. Time commitment required for Rally Post attendance .............................................. 38
3.4. Staff perceptions about Rally Post operations .................................................................. 39
3.4.1. General perception of Rally Posts.............................................................................. 39
3.4.2. Perceived problems with Rally Post operations......................................................... 39
3.4.2.1. General problems 40
3.4.2.2. Supply situation 40
3.4.2.3. Weighing scales 43
3.4.2.4. Participants arriving at RPs without health cards and ration cards 44
3.4.3. Suggestions on how to improve Rally Post services ................................................. 44
3.5. Beneficiary perceptions about Rally Posts ....................................................................... 45
3.5.1. Perceived importance of different Rally Post services .............................................. 45
3.5.2. Ease of use ................................................................................................................. 46
3.5.3. Suggestions on how to improve the Rally Post services ........................................... 47
3.6. Summary of findings regarding operations at the Rally Posts.......................................... 47
3.6.1. Operations and quality of services at the Rally Posts................................................ 47
3.6.2. Staff perceptions and suggestions for improvement of the Rally Posts...................... 49
3.6.3. Beneficiaries’ perceptions and suggestions for improvement of the Rally Posts...... 50
3.6.4. Conclusions on operations at Rally Posts .................................................................. 50
4. Program Operations at the Mothers' Clubs ...............................................................................51
4.1. Description of the Mothers’ Clubs operations according to implementation plan ........... 51
4.1.2. Communication materials used.................................................................................. 52
4.1.3. Final note on organization of the Mothers’ Clubs ..................................................... 53
4.2. Methods used to assess the Mothers’ Club operations ..................................................... 53
4.3. Operations and quality of services at the Mothers’ Clubs ................................................ 53
4.3.1. Organization of the Mothers Clubs and attendance................................................... 54
4.3.1.1. Topics discussed in Mothers’ Clubs 54
4.3.1.2. Size of the Mothers’ Clubs and beneficiary characteristics 54
4.3.1.3. Time commitments required for Mothers’ Club attendance and facilitation 55
4.3.1.4. Adherence to Mothers’ Club categories 56
4.3.1.5. Adherence to planned schedule of learning sessions 57
4.3.2. Quality of education at the Mothers’ Clubs............................................................... 60
4.3.3. Recall and use of the information discussed at the Mothers’ Clubs.......................... 62
4.3.3.1. Sessions #1-4—Breastfeeding practices 62
4.3.3.2. Session #5—Lactational Amenorrhea Method (LAM) 62
4.3.3.3. Session #8—Cooking and tasting of enriched meals 64
4.3.3.4. Sessions #9, 10 and 12—Complementary feeding practices 64
4.3.4. Overall comprehension of new concepts discussed in Mothers’ Clubs..................... 67
4.4. Staff perceptions about the Mothers’ Clubs...................................................................... 69
4.4.1. General perception of the Mothers’ Clubs................................................................. 69
4.4.2. Problems encountered in facilitating the Mothers’ Clubs.......................................... 70
4.4.3. Suggestions on how to improve the Mothers’ Clubs................................................. 71
4.5. Beneficiary perceptions about Mothers’ Clubs................................................................. 71
4.5.1. Perceived importance of Mothers Clubs.................................................................... 71
4.5.2. Ease of use ................................................................................................................. 72
4.6. Summary of findings regarding operations at the Mothers’ Clubs................................... 72
4.6.1. Organization and operations of the Mothers’ Clubs.................................................. 72
4.6.2. Quality of education at the Mothers’ Clubs and recall and use of information......... 73
4.6.3. Staff perceptions about the Mothers’ Clubs............................................................... 73
4.6.4. Beneficiary perceptions about the Mothers’ Clubs.................................................... 74
4.6.5. Conclusions on operations at the Mothers’ Clubs ..................................................... 74
5. Program Operattions at Food Distribution Points.....................................................................75
5.1. Description of operations at Food Distribution Points according to implementation plan75
5.2. Methods used to assess operations at the Food Distribution Points ................................. 76
5.3. Operations and quality of services at the Food Distribution Points.................................. 76
5.3.1. Attendance and beneficiary/staff ratio....................................................................... 76
5.3.1.1. Beneficiary attendance and characteristics 76
5.3.1.2. Health and commodity staff 79
5.3.2. General implementation characteristics..................................................................... 79
5.3.2.1. Venue and communities covered 80
5.3.2.2. Amount of food received 80
5.3.3. Quality of implementation ......................................................................................... 82
5.3.3.1. Timeliness of operations 82
5.3.3.2. Quality of the food commodities 83
5.4. Use of food rations............................................................................................................ 84
5.4.1. Length of time food lasts in the household................................................................ 84
5.4.2. Recipes prepared with the food commodities............................................................ 85
5.4.3. Sharing and selling of food commodities .................................................................. 88
5.5. Staff perceptions about the Food Distribution Points....................................................... 90
5.5.1. Perceived problems with the food distribution .......................................................... 90
5.5.2. Suggestions on how to improve the food distribution process .................................. 92
5.6. Beneficiary perceptions about Food Distribution Points.................................................. 92
5.6.1. Perceived problems with the Food Distribution Points ............................................. 92
5.6.2. Perceptions about the importance of the food distribution and the food rations ....... 93
5.6.3. Suggestions on how to improve the food distribution ............................................... 93
5.7. Summary of findings regarding operations at the Food Distribution Points .................... 94
5.7.1. Operations and quality of services at the Food Distribution Points........................... 94
5.7.2. The use of food rations .............................................................................................. 95
5.7.3. Staff perceptions about the Food Distribution Points................................................ 95
5.7.4. Beneficiaries’ perceptions regarding the Food Distribution Points........................... 96
5.7.5. Conclusions on operations at the Food Distribution Points....................................... 96
6. Staff Perceptions of Program Organization and Management .................................................97
6.1. Management structure and key stakeholders in the program............................................ 97
6.2. Methods used to understand the program organization and management........................ 97
6.3. Perception of responsibilities of World Vision staff......................................................... 98
6.4. Motivation and perceived value of the program............................................................. 101
6.5. Constraints to job satisfaction......................................................................................... 104
6.5.1. Suggestions from staff for ways to increase support ............................................... 109
6.6. Supervision ..................................................................................................................... 110
6.6.1. Suggestions from staff on ways to improve supervision ......................................... 113
6.7. Coordination between the health and food components of the program ........................ 114
6.8. Summary of issues related to program organization and management .......................... 119
7.1. At the Rally Posts ........................................................................................................... 121
7.2. At the Mothers’ Clubs..................................................................................................... 124
7.3. At the Food Distribution Points ...................................................................................... 124
7.4. Final comments............................................................................................................... 125
REFERENCES ............................................................................................................................132
Annex 1. List of Clusters ....................................................................................................... 133
Annex 2. Protocol for the Operations Research..................................................................... 134
Annex 3. Details of the Different Focus Groups ................................................................... 136
Annex 4a. Schedule of Learning Sessions at Mothers’ Clubs for Pregnant and Lactating
Women (Preventive and Recuperative Program Groups)............................................ 137
Annex 5. Quality of Education Provided at Mothers’ Clubs................................................. 140
Annex 6. Organizational Structure of the Health Component of the Program...................... 141
Annex 7. Organizational Structure of the Food Component of the Program ........................ 142
LIST OF TABLES
Table 2.1. Summary of methods used in operations research and sample sizes...........................13
Table 3.1. Number of pregnant/lactating women and children under 5 years per RP, by program
group ..................................................................................................................................17
Table 3.2. Number of health staff present at RPs .........................................................................19
Table 3.3. Organization of activities at RPs .................................................................................19
Table 3.4. Sequence of program services for tracked respondents at RPs ...................................20
Table 3.5. Number of education sessions at RPs and number of participants per session ...........22
Table 3.6. Attendance of education sessions at RPs and reason for non-attendance....................22
Table 3.7. Observations of education sessions held at the RPs ....................................................23
Table 3.8. Quality of education provided at RPs..........................................................................24
Table 3.9. Spontaneous recall and application of advice received at RP education sessions (home
interview sample of mothers exposed to specific education topics)..................................25
Table 3.10. Use of registration, vitamin A, and deworming tablet dosage and ORS packets at
RPs and reason for non-usage............................................................................................26
Table 3.11. Status of beneficiary health and ration cards.............................................................28
Table 3.12. Timing of vitamin A and deworming tablet dosage (tracked respondents)...............29
Table 3.13. Use of weighing and other steps in growth monitoring and promotion activities and
reasons for non-usage ........................................................................................................30
Table 3.14. Observations of weighing sessions at RPs.................................................................31
Table 3.15. Quality of weighing sessions at RPs (observations and re-weighing by our staff) ...33
Table 3.16. Quality of post-weighing counseling in RPs (among tracked respondents who
received counseling (n=14)) ..............................................................................................35
Table 3.17. Type of individual advice given at RPs, by nutritional status ...................................36
Table 3.18. Use of immunization at RPs and reason for non-usage.............................................37
Table 3.19. Time commitment required for RP attendance..........................................................38
Table 3.20. Availability of medical supplies in RPs.....................................................................40
Table 3.21. Availability of vaccines in RPs..................................................................................42
Table 3.22. Availability of health cards in RPs ............................................................................43
Table 3.23. Perceived importance of services at Rally Posts .......................................................45
Table 3.24. Suggestions for RP services improvement made by participants..............................47
Table 4.1. Session topics discussed in the different categories of MCs observed........................54
Table 4.2. Number of participants of Mothers’ Clubs..................................................................55
Table 4.3. Duration of Mothers’ Clubs.........................................................................................56
Table 4.4. Adherence to MC categories........................................................................................57
Table 4.5. Adherence to Mothers’ Club schedules: age (in months) of children who attended the
different MCs (from exit interviews; n=41) ......................................................................58
Table 4.6. Quality of education provided at Mothers’ Clubs .......................................................60
Table 4.7. Spontaneous recall and application of advice received at Mothers’ Club education ..63
Table 4.8. Application of advice received at Mothers’ Club education and reasons for not using66
Table 4.9. Overall comprehension of new concepts discussed in Mothers’ Club ........................67
Table 4.10. Examples of a very good and a very poor overall comprehension............................68
Table 5.1. Composition of direct and indirect food rations, per beneficiary category .................75
Table 5.2. Number of beneficiaries at the FDPs...........................................................................77
Table 5.3. Characteristics of beneficiaries at the FDPs ................................................................78
Table 5.4. Time commitment for beneficiaries who attend FDPs ................................................79
Table 5.5. Number of MCH and commodity staff present at FDPs .............................................79
Table 5.6. Amount of food received .............................................................................................80
Table 5.7. Differences between amount of food received and allocated......................................81
Table 5.8. Timeliness of operations at FDPs ................................................................................82
Table 5.9. Duration of use of food in beneficiary households, by number of rations received and
by number of indirect beneficiaries ...................................................................................84
Table 5.10. Recipes prepared with the commodities (from exit interviews, n = 45)....................86
Table 5.11. Use of additional ingredients when preparing the food commodities for children
(from exit interviews, n = 45) ............................................................................................87
Table 5.12. Sharing and selling of food commodities (from home interviews, n = 29)...............88
Table 6.1. Themes with respect to current responsibilities...........................................................98
Table 6.2. Themes with respect to job motivation......................................................................101
Table 6.3. Themes with respect of external constraints to satisfaction ......................................105
Table 6.4. Themes with respect to supervision...........................................................................110
Table 6.5. Themes related to coordination and interactions between health and food staff.......115
Table 7.1. Summary of discussions held in Haiti with World Vision staff on the operations
research findings and follow-up actions ..........................................................................126
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
i
EXECUTIVE SUMMARY
Background and objectives
This report describes the methods and results of operations research undertaken to assess the
effectiveness of World Vision’s maternal and child health1
 (MCH) program in the Central
Plateau region of Haiti. The research had three main objectives: (1) to assess the effectiveness
of implementation and operations of the program relative to plans; (2) to assess the quality of
delivery of the various services; and (3) to explore the perceptions of different stakeholders (i.e.,
beneficiaries and field implementers) regarding the operations and quality of services provided
by the program.
This research is part of a larger evaluation being conducted by the International Food Policy
Research Institute (IFPRI) and Cornell University (CU) in collaboration with World Vision
(WV)-Haiti. The larger evaluation compares two models for delivering an integrated food and
nutrition program, both with a take-home food ration component. The two models, which are
being implemented by WV-Haiti, are: (1) the traditional recuperative approach, whereby
children under 5 years of age are targeted to receive food supplements, nutrition counseling, and
follow-up when they are identified as being underweight for their age; and (2) the preventive
approach, which targets food supplements and other preventive interventions to all children
below 2 years of age, irrespective of their nutritional status.2
It is important to note that the operations research described here was carried out in mid 2003,
i.e., more than six months prior to the finalization of this report. Thus, the results depict the
program as it was in 2003, a few months after all program components were fully implemented
in the intervention area. As highlighted in the final section of this summary, several of the
operational constraints identified in our research have been addressed since our data were
collected, as part of the program’s routine monitoring and strengthening activities.
Methodology
The operations research used a variety of qualitative and quantitative methods, including
structured observations at different program delivery points, structured and semi-structured
interviews with stakeholders, and a series of focus group discussions with WV program staff.
The following three main program service delivery points were included in the evaluation:
(1) the Rally Posts (RPs), where health education, growth monitoring and promotion (GMP), and
other preventive health-care services are provided, and where program beneficiaries are
identified; (2) the Mothers’ Clubs (MCs), where smaller groups of participants gather to discuss
health and nutrition topics in the context of the program’s behavior change and communication
 
1
 The World Vision program does include a strong nutrition component and should therefore be referred to as “maternal and child
health and nutrition program (MCHN)”; however, we use the term MCH in this report in order to be consistent with the
terminology used by World Vision to refer to its program.
2
 Note that the main objective of this round of operations research was not to assess differences between the two program groups,
but rather to identify potential operational constraints in the program as a whole. The next round of operations research to be
carried out in 2004 will address the issue of potential differences in operations between the two program groups.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
ii
(BCC) strategy; and (3) the Food Distribution Points (FDPs), where beneficiaries receive their
monthly food rations.
The stakeholders included in the research were beneficiaries of the program and the WV
administrative and field staff at different levels of the program. This includes the health agents,
colvols (community program volunteers) and food monitors, who are the direct program
implementers, their supervisors (the maternal and child health [MCH] supervisors and the
commodity supervisors), the regional and national health coordinators, and the regional
commodity officer.
A summary of the methods used and sample sizes at different delivery points and with different
stakeholders is provided in Table S1.
The research was undertaken in the area of the larger evaluation project, which covers 20 zones
or clusters of communities (localités) in three communes: Hinche, Thomonde, and Lascahobas.
Each cluster is the responsibility of one health agent, who is usually assisted by 1-3 colvols. The
research included one program delivery point (i.e., RP, MC, and FDP) for each health agent, for
a total of 19 RPs, 20 MCs and 10 FDPs.
The field team for the operations research consisted of two supervisors (Cornelia Loechl and
Arsène Ferrus), two observers (responsible for observations at the RPs and FDPs), two
fieldworkers (responsible for conducting the focus group discussions and semi-structured
interviews with the health agents and beneficiaries), and one World Vision nurse (responsible for
observations at the MCs). The training of the team, including the field-testing of questionnaires
and interview guides, was done in June 2003 and data collection took place between July and
September 2003.
In the following sections, the results of the operations research are presented separately for the
three service delivery points—the RPs, MCs, and FDPs. This is followed by a summary of the
results concerning staff perceptions of program organization and management. Note that the
information derived from the different research methods used is combined and that the findings
are presented by topic, rather than by research approach. This triangulation of research methods
provides a rich and complete picture of each service delivery point.
The Rally Posts
Operations and organization of the Rally Posts
The RPs offer preventive health-care services to pregnant and lactating women, mothers with
children less than 5 years of age, and women 15 to 49 years old. RP meetings are held monthly
in each community and are managed primarily by the health agent responsible for that
community, who is usually assisted by other health agents and colvols. As per the program
implementation plan, the sequence of activities for each participant is as follows: (1) education
session; (2) registration of participants, which includes updating the participant’s health card, and
the distribution of vitamin A, deworming tablets, and oral rehydration salts (ORS); (3) growth
monitoring and promotion (GMP) activities; and (4) immunization, if required. Our main
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
iii
findings regarding the efficiency of implementation of the different activities at the RPs and the
quality of the services offered are presented below. This is followed by a summary of the
perceptions of beneficiaries and program implementers regarding the RPs and their suggestions
for potential improvements.
Table S1. Summary of methodology used in the operations research
Research method/
stakeholder Purpose Location and sample size
Structured
observations at main
service delivery points
Assess operations, flow of
activities, time involved in
attending the delivery point
Unit of observation/analysis is the service delivery point
RPs: n = 19
MCs: n = 20
FDPs: n = 10
In addition, at the RPs, information was collected on 38
tracked beneficiaries (2 per RP), using both observations
and interview techniques (see below).
Structured interviews
with beneficiaries
Assess access, services used,
perceived importance of
different services, and use of
food
Unit of analysis is the participant/beneficiary
RPs (n = 19):
Exit interviews with tracked respondents (n = 38)
Exit interviews with random sample of respondents (n = 59)
MCs (n = 20):
Exit interviews with random sample of respondents (n = 41)
FDPs (n = 10):
Exit interviews with random sample of respondents (n = 45)
Semi-structured
interviews with health
agents
Assess perceptions and opinions
regarding operations, quality of
services, constraints to program
implementation, and suggestions
for improvement
Unit of analysis is the health agent
RPs (n = 19)
MCs (n = 20)
FDPs (n = 20)
Semi-structured
interviews with health
coordinators and
commodity officer
Assess perceptions regarding
current responsibilities,
constraints to program
implementation, supervision,
coordination, and suggestions
for improvement
Unit of analysis is the health coordinator/commodity officer
National health coordinator
Regional health coordinator (Hinche)
Regional commodity officer (Hinche)
Semi-structured
interviews with
beneficiaries in their
home
Assess perceptions regarding the
program services, their relative
importance, the constraints faced
in using the services, and
suggestions for improving the
program
Unit of analysis is the beneficiary
In beneficiaries’ homes (n = 30)
Focus group
discussions with five
types of stakeholders
(total of 7 focus
groups)
Assess perceptions about current
responsibilities, roles, workload,
time constraints, supervision and
incentive structure, coordination,
and training
Health agents (2 groups)
Colvols (2 groups)
Food monitors (1 group)
MCH supervisors (1 group)
Commodity supervisors (1 group) a Throughout this report, women attending the RPs are referred to as “participants,” whereas mothers and children attending the MCs
and the FDPs are referred to as “beneficiaries.” This differentiation is made because the RPs are open to all women of reproductive age
and children under 5 years in the communities served, whereas the other two program delivery points, the MCs and the FDPs, are
specifically targeted to program beneficiaries.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
iv
Implementation and quality of services at the Rally Posts
Attendance and participant/staff ratio. On average, 74 participants (children and pregnant or
lactating women) attended the RPs observed, with considerable variability between the sites—
from a low of 9 participants to a high of 169. These numbers do not include the caregivers
accompanying the children and other family members who may also be present. Not
surprisingly, the overall impression is that RPs are generally crowded. The participant/staff ratio
also varies widely between RPs, ranging from 5 to 151 participants per health agent, with an
average of 55. When including the colvols in addition to the health agents in the ratio, the
numbers are more reasonable (on average, 21 participants per health staff person, but with a
range of 2 to 59).
Sequencing of activities and bottlenecks. Overall, it appears that the sequence of activities in the
implementation plan (described above) is rarely followed. It is not clear, however, whether or
not departure from the proposed sequence of activities affects the effectiveness of
implementation. Our data show no evidence of an association between the likelihood of
experiencing bottlenecks (i.e., inefficient flow of participants through the different program
activities) and the specific program sequence used in the different RPs. In fact, bottlenecks were
not avoided even in the RPs that used a sequence of activities similar to the one proposed in the
program implementation plan. Factors responsible for bottlenecks were invariably the large
numbers of participants, and the high participant/staff ratios. The two RPs where no bottlenecks
were observed had less than 40 participants and a low participant/staff ratio (< 14).
Education. Our observations indicate that the actual organization of the education sessions in the
RPs departs considerably from the program implementation plan, which suggests that several
education sessions should be held throughout the day, to accommodate all participants. The
implementation plan also provides a 12-month schedule for education topics. In contrast to this
plan, at most of the RPs, only one education session was held, and this session was often held
early in the morning as the first activity at the RP. This resulted in significant numbers of
participants missing the education session because they arrived late. Additional sessions, to
accommodate late arrivers, were observed in only 4 of the 19 RPs. Second, none of the
education sessions observed corresponded to the recommended topic for the month. While some
flexibility in the choice of topics may be desirable to ensure that topics of particular relevance
are addressed in a timely fashion (for example, the use of ORS during periods of high diarrhea
incidence), the risk is that some other important topics may never be addressed.
Insufficient use of communication materials was also observed in most education sessions, in
spite of the fact that such material had been provided to the health staff. This is likely to limit
the effectiveness of the education, as is the large number of participants (on average, 30, and up
to 76 in RPs where only one session was held), the distracting environment, and the short time
allocated to this activity (between 1 and 23 minutes).
The potential effectiveness of the education conducted at the RPs was assessed by asking
respondents from the home interviews what they could recall from the last education session they
had attended at an RP. Most mothers had attended a session on immunization; their recall of the
role of immunization for protecting children against illnesses and the importance of receiving all
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
v
doses to get full protection were quite good; some mothers could also recall the role of certain
vaccines. Most mothers also reported that they regularly took their child to the RP for
immunization. Thus, it appears that the recall of details and the adoption of recommended
practices among mothers who had recently attended an immunization session at the RP were
quite good, in spite of the generally unfavorable conditions in which the education sessions were
conducted at the RPs.
Overall, our findings suggest that the generally distracting environment due to the many
activities taking place at the same time and in the same area, the crowding, and the high
participant/staff ratio make effective education difficult in the context of the RPs. However, the
RPs can still be used as effective venues to reinforce some of the topics discussed in other more
appropriate BCC venues (like the MCs; see below) and also to present simple messages on topics
of general interest such as hygiene, the use of ORS, or the importance of immunization.
Registration, vitamin A supplementation, and distribution of deworming drugs and ORS. The
registration of the participants at the RPs was by far the most significant bottleneck observed
(primary bottleneck in 16 of the 19 RPs). This appears to be caused by the large numbers of
participants and insufficient staff. While it is a slow process, the registration of participants
generally seems to proceed as planned—that is, health cards are usually updated with the
information on immunization, vitamin A, and the deworming dosage received on the day of
observation. In contrast, bringing the food ration card to the RP—as required by the program to
register the date of attendance in order to be eligible for the monthly food ration—has clearly not
yet turned into a routine.
The quality of vitamin A supplementation and distribution of deworming drugs were assessed
indirectly by verifying whether children who received it on the day of observation were actually
due, and calculating the percentage of children due who did not receive it. Vitamin A
supplementation is to be administered every six months to children less than 5 years of age, and
deworming tablets are to be provided to all children 2 years or older, also every six months. In
general, it appears that when given, the schedules of administration of vitamin A and deworming
were followed. However, the lack of availability of vitamin A, and particularly of deworming
tablets and ORS, was a generalized problem in the RPs observed. For instance, 26 percent of the
tracked participants attended a RP where vitamin A supplementation was not provided on the
day of their attendance. The situation was even worse for deworming tablets; half of the children
who were eligible did not receive the tablets because the tablets were not distributed at the RP
that day. The ORS packets, which should be handed out to all participants, were received by
only two of our respondents from the exit interviews and none from the tracked interviews.
Observations at the RP level confirm that the supply of vitamin A, deworming tablets, and ORS
packets was inadequate; vitamin A was unavailable in 5 of the 19 RPs observed, whereas
deworming tablets and ORS packets were unavailable in 8 RPs and 12 RPs, respectively.
Growth monitoring and promotion (GMP). In general, the GMP activities at the RPs appear to
function relatively smoothly. All the caregivers, either tracked or interviewed, reported that their
child was weighed, and in most cases, the weight was plotted and the child’s nutritional status
assessed. Only about half of the caregivers, however, were informed about their child’s weight
and even fewer received some post-weighing advice (n = 14/38). The likelihood of receiving
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
vi
counseling appeared slightly higher among caregivers who had a malnourished child, but even
then, more than half of the caregivers with a child classified as M2 (malnourished according to
the Gomez classification)3
 received no individual advice following the weighing and plotting
procedures.
The quality of the weighing and plotting of the weight to assess children’s nutritional status was
tested through the re-weighing of children by our trained and standardized fieldworkers using
electronic scales. The findings show that both the weighing and plotting of the weight were less
than optimal and resulted in 28 percent of the children being misclassified into different Gomez
malnutrition categories. For the recuperative group, where children’s eligibility to receive food
rations is based on whether or not they are malnourished (M2 or M3), this is likely to result in
the misclassification of children as beneficiaries or nonbeneficiaries of the program. In our
sample of 38 tracked respondents, misclassification errors resulted in a similar number of eligible
(M2) children who were missed (i.e., misclassified as Normal or M1) and therefore excluded
from the program (n = 2), and of non-eligible children (Normal or M1) who were mistakenly
classified as M2 and thus included in the program (n = 2). Although in this example the errors
cancel out with regards to the amount of program resources such as food and other services
required, misclassification problems reduce the cost-effectiveness of programs. This is because
it results in the leakage of program benefits to children who are not malnourished, and therefore
are less likely to benefit from the program, and in a lower coverage of those children who are
malnourished and hence more likely to benefit from it.
Our assessment also confirms a widely documented problem with GMP activities carried out in
settings such as the RPs, which is that the opportunity to use the child’s weight and growth
information to interact with the caregiver and to tailor individual counseling messages to the
specific needs of the child is lost. In the RPs observed, even transmitting the information to the
mother about the child’s weight and nutritional status was difficult to achieve, let alone
providing relevant and tailored individual counseling. Given the high participant/staff ratio, the
distracting atmosphere, and the numerous activities going on at the same time in the RPs, it is
probably unrealistic to expect that GMP could offer a venue for effective education and
individual counseling of caregivers.
Immunization. A large proportion of children and caregivers included in our tracking or exit
interviews at the RPs were fully immunized. For those who were not, however, approximately
40 percent were not immunized on the day of the RP because the service was not offered. This,
in turn, was usually due to a lack of supply of vaccines. Clearly, the lack of supplies of vaccine
and, as indicated earlier, of vitamin A, deworming tablets, and ORS packets, were common
problems in the RPs observed, and this is likely to result in inadequate coverage of these
important preventive services in the area.
 
3
 The Gomez classification is widely used in MCH programs to classify children as normal or malnourished. In this
classification, normal (N) corresponds to a child having a weight for his/her age that is ≥ 90 percent of the median of the weight￾for-age CDC/NCHS/WHO standards (WHO 1979); mild malnutrition (Grade M1) corresponds to a weight-for-age between 75
percent and < 90 percent of the median; moderate malnutrition (Grade M2) to 60 percent - < 75 percent of the median; and severe
malnutrition (Grade M3) to ≤ 60 percent of the median (Cogill 2003).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
vii
Time involvement. On average, participants spent 20 minutes to get to the RP and close to 2
hours at the RP. The total time, including traveling to and from the RPs and the time spent there
could be as high as 7 hours for some mothers, and as low as 35 minutes for others. Observations
with our tracked respondents revealed that on average 75 percent of the time that caregivers
spent at the RP was used in waiting in line for the services (on average, 94 minutes).
Staff perceptions and suggestions for improvement of the Rally Posts
General perceptions. Overall the data suggest that the health staff view the RPs as an extremely
important component of their services to the communities and particularly of their role in
fostering children’s growth and health. MCH supervisors consider the RPs as the key activity of
the MCH program because it is the venue where program beneficiaries are identified. In
addition, the health staff expressed that they consider the RPs important because the services
they offer also reach families who do not have access to the food aid component of the program.
Most of the health agents interviewed felt that the RPs were working well in general and they
reported high participation rates. They also felt that participants enjoyed going to the RPs.
Perceived problems. Some of the factors that the health staff felt hindered the efficient
implementation of services at the RPs included problems with the lack of supplies such as
medical supplies as well as scales and megaphones, insulated containers and ice for the vaccines,
and furniture (chairs, benches, and tables). Other problems mentioned were the high
participant/staff ratios, the late arrival of participants, and the fact that this prevented them from
attending the education sessions, and the problem of participants forgetting their health or ration
cards. Furthermore, the health agents felt that it was imperative that the program management
provides more training for colvols so that they could better assist them in their duties.
Another important problem that many health agents mentioned was the lack of transport to pick
up the vaccines at the dispensary or the WV office, which causes delays in the start of the RPs.
They also mentioned the large distances between the different communities where they worked
as a key constraint.
Suggestions on how to improve the RP services. The suggestions made by the health agents to
improve the services offered at the RPs were very much in line with their comments regarding
the problems they faced. In order of importance, they recommended that the program
management takes the following actions: (1) provide medical supplies and vaccines regularly
and in sufficient amounts, as well as all other materials and furniture needed; (2) assist with
transport facilities; (3) hire more colvols and provide them with better training; also facilitate
interaction among health staff to increase collaboration; (4) provide ice to maintain the cold￾chain for vaccines; and (5) offer training to improve the logistics at the RPs and to improve their
own performance.
Beneficiaries’ perceptions and suggestions for improvement of the Rally Posts
Data from interviews with program beneficiaries suggested that overall, they were quite satisfied
with the RP services, and considered most of them to be important for their child’s health and
their own. Immunization was perceived to be the most important service by the majority of the
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
viii
respondents. Several of them, however, felt that all services were very important and they had
difficulty ranking them in order of importance. Few respondents had suggestions on how to
improve the RP services, but those who did mainly focused on ways to improve the internal
organization of the RPs. Examples of their suggestions include the following: improve the
venues for the RPs, ensure that the dates of the RPs do not change and that the dates are regularly
announced, and encourage the health staff to arrive on time. Some also suggested that additional
services be offered at the RPs, such as general medical consultations, free distribution of drugs
for sick individuals, and regular availability of chlorine for water treatment.
Conclusions on operations at Rally Posts
Overall, our assessment suggests that the RPs are generally operating in accordance with World
Vision’s implementation plan. The problems encountered with operations relate primarily to
organizational and logistical issues such as crowding, the high participant/staff ratio, long
waiting times, bottlenecks at registration, and lack of supplies and transport for staff. In terms of
quality of services, the areas that could benefit from some improvement are the general
education sessions and the communication between health staff and caregivers in the context of
the growth monitoring and promotion activities.
The Mothers’ Clubs
Operations and organization of the Mothers’ Clubs
World Vision, in collaboration with our research team, designed a new behavior change
communication (BCC) strategy in 2002-03, focusing mainly on improving infant and young
child-feeding practices. Visual and communications materials were developed for a series of 13
learning sessions on infant and young child feeding. The health staff were trained in the use of
the new materials and their technical content, as well as in adult education techniques. The BCC
strategy was fully implemented in early 2003, only a few months before the data were gathered
for this operations research. In the context of this new BCC strategy, the MCs are the key venue
for most of the educational and BCC interventions provided by the program.
Separate MCs are organized for pregnant women, lactating women (0-5 months postpartum), and
depending on the program model, for mothers of children 6-23 months old (preventive model) or
for mothers of malnourished children 6-59 months old (recuperative model). In the preventive
model, the MCs organized for children are expected to be as homogenous as possible in regard to
the age of the children of participating mothers. For example, it is recommended that separate
groups be held for mothers of 6-11 month-old children and for mothers of older children (12-23
months). The rationale is to ensure that the learning sessions and the specific messages they
contain are delivered to mothers at the time when they are most likely to need the information
and to translate the newly acquired knowledge into behavior change.
Findings of our evaluation of operational aspects and of the quality of services offered at the
MCs are presented below, followed by a brief summary of the perceptions of staff and
beneficiaries about the MCs and their recommendations on how to improve them.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
ix
Implementation of services at the Mothers’ Clubs
Organization of the MCs and attendance. The MCs are meant to gather small groups of 15-20
participants. The number of registered participants attending the 20 MCs observed was within
this range, but a number of other people—children, fathers, or other relatives—were also present
at most sessions, raising the average number of participants to 18 and the upper end of the range
to 41 participants. Although it is difficult to restrict participation only to registered beneficiaries,
some advice may need to be provided to health agents/colvols on how to ensure that the size of
the MCs remains manageable.
The time involved in holding or attending a MC is also a real concern. Although the learning
sessions lasted, on average, one hour, the time involvement for participants and health staff was
much longer, mostly because of delays in starting the session, while waiting for all participants
to join. The average time health agents spent on MCs was 2.7 hours, with a range of 1.7 to 5.3
hours.
Adherence to the MC categories and planned schedule of learning sessions. In general, the MCs
for pregnant and lactating women respectively included the right categories of women, except for
a few mothers in the lactating groups who had children older than the maximum of 6 months
recommended by the program. The schedule of learning sessions for the MCs for pregnant and
lactating women was also generally as recommended—they included several sessions on
breastfeeding and a session on the introduction of complementary foods and one on “children
learning to eat.” Many of the sessions targeted to mothers of children < 24 months in the
preventive group, however, did not follow the recommended schedule of learning sessions for
children of different ages, and quite a few mothers received the information at the wrong time.
For instance, mothers of children 12 and 18 months, respectively, participated in a learning
session about feeding of children 6-12 months of age. The same was true for a mother of a 4-
month-old child who was instructed on how to feed children 12 months and older. Our results
suggest that there are still some logistical difficulties in forming MCs that are homogeneous,
especially with respect to the small age ranges recommended for the MCs in the preventive
program group. It is important to note, however, that this new system had been implemented
only a few months prior to our research, and therefore may still have been in a transition phase.
Quality of education at the Mothers’ Clubs and recall and use of information
Quality of education at the MCs. The quality of education at the MCs was observed to be very
good, especially considering that training in the use of the new materials and adult education
techniques had been completed only a few months before our observations. The technical
content of the education was correct and complete in a large majority of the MCs observed and
the management and organization of the sessions were generally good. The health staff’s
facilitation and teaching skills were also generally good; they used open-ended questions to
probe and encourage the participants, and approximately one-half of them managed to establish a
real dialogue and interaction with the group. These results are very encouraging and show a
major improvement compared to the situation observed before the training. Previously, the
education provided at the MCs was generally based on a lecture-type approach to teaching and a
classroom-like arrangement for the group, both of which are considered inappropriate for
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
x
effective BCC. The improvement in facilitation and teaching skills and in attitudes observed
during the operations research show that the training of the health staff conducted in 2003 was
highly effective. Clearly a behavior change process at the level of the health staff itself has
commenced.
Recall and use of information discussed at the MCs. Overall, the mothers’ recall of the sessions
they had attended 1- 2 weeks prior to the interviews was very good. Most could recall the main
topic of the session and at least two or three more specific concepts related to the main topic.
The rate of reported adoption of the recommended practices was also quite high, especially with
regards to exclusive breastfeeding and the use of enriched gruels for the complementary feeding
of infants and young children. The main reasons reported by mothers for not adopting the
recommendation to feed their child enriched foods were financial. For three of the mothers who
were taught to express breast milk, their reason for not adopting the recommended practice was
their fear of contamination of the milk during storage.
We also used a subjective rating approach to assess the overall comprehension of the concepts
discussed at the MCs by beneficiary women. The results indicated that one-third of the
respondents had very good comprehension—i.e., they recalled information correctly and were
consistent in what they recalled and then reported translating into practice. Very few women had
very poor comprehension (n = 3/30), and the remaining women were in the middle two
categories (upper and lower middle). Thus overall, it appears that the learning sessions were
quite effective in transferring knowledge to women and also in initiating a process of behavior
change, at least in some of the participants.
It is important to highlight that the BCC strategy had only recently been implemented and that
the findings may be a reflection of a great momentum among the health staff about what they
had recently learned and were eager to put into practice. The same is true for the participating
women who may have been particularly enthusiastic to try some of the new recipes or other
practices they had just learned at the MCs. It will be important to reassess in the next round of
operations research to be carried out in 2004 whether this momentum could be sustained and
whether the quality of the intervention and its impact on behavior change are maintained over
time.
Staff perceptions and suggestions for improvement of the Mothers’ Clubs
General perceptions. All health agents interviewed felt that the MCs were working well in
general. The colvols expressed the same view and indicated that participating mothers liked the
clubs. Health agents, colvols, and the MCH supervisors all referred to the MCs as an important
forum for mothers to exchange ideas, receive important information, and acquire skills that will
help them take better care of their children. There was also a general sense that MCs had
improved since the previous year, both in relation to organizational aspects as well as in the
content of the lessons, the availability of communication materials, and the supervision.
Perceived problems. The most common problems reported were the late arrival of participants
and absenteeism. Strategies to address these problems were also discussed and included
threatening the participants with considering them absent if they arrived late at the MC and
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xi
refusing to sign their ration card if they were either late or absent. Other problems indicated by
the health agents were the poor venues of the MCs and the lack of appropriate seating
arrangement for the participants, and the problem with mothers who forget their ration cards.
Suggestions on how to improve the MCs. Again, most of the suggestions made by the health
agents were in direct relation to the problems identified. Their recommendations focused on the
following aspects: (1) staff training: ensure retraining, monitoring, and supervision to maintain
quality; (2) resources: provide more suitable and better-equipped venues and more colvols to
assist them in their work; (3) time and workload: provide transport; (4) calendars: improve the
scheduling of the MCs.
Beneficiary perceptions about the Mothers’ Clubs
The MCs were well appreciated by the respondents, who indicated that the clubs were important
for their children’s health. Some of them considered that the MCs were the most important
service offered by WV, although many had trouble ranking the different services according to
their importance because they felt that all were important. The majority of respondents felt that
attendance at the MCs improved their knowledge about infant and child feeding and care.
Nearly two-thirds of the home interview respondents (n = 18) felt that the MCs were the easiest
service to use among the different WV activities that they attended. The main reasons were that
the clubs were relatively close to their homes and the time involvement was reasonable.
Conclusions on operations at the Mothers’ Clubs
Overall, our results suggest that the MCs are appreciated by both the staff and the beneficiaries
and are perceived to be the least time-consuming requirement of the program for beneficiaries.
MCs are also recognized as being a key element of the package of services offered by the
program and as being the primary venue for the education and BCC strategy. Our results also
showed marked improvements in the quality of the education provided at the MCs, both with
regards to the technical content of the sessions and the health staff’s facilitation and teaching
skills. Ensuring the intended composition of the clubs (especially in the preventive program
group) in order to maximize their effectiveness and impact on behavior change is still a
challenge and will probably require continued supervision and re-training of the staff. The same
is true for sustaining the quality of education over time and maintaining the motivation of both
staff and beneficiaries.
The Food Distribution Points
Operations and organization of the Food Distribution Points
The distribution of food aid commodities to beneficiaries of the WV program occurs on a
monthly basis at special Food Distribution Points (FDPs). Beneficiaries from several
communities are scheduled to receive their food rations at a central distribution point on the
same day.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xii
Beneficiary households receive both direct and indirect rations. The direct ration is intended for
the beneficiary child and/or mother, whereas the indirect ration is to complement the family diet
and is to be shared among all family members. The direct ration for a beneficiary child consists
of 8 kg of wheat-soy blend (WSB) and 2 kg of oil; the indirect ration includes 10 kg of soy￾fortified bulgur (SFB) and 2.5 kg of lentils. For the pregnant or lactating mother, the direct
ration is 5 kg of SFB, 2 kg of lentils, and 1.5 kg of vegetable oil; the indirect ration is 5 kg of
SFB, 2 kg of lentils, and 1.5 kg of vegetable oil. Families with more than one direct beneficiary
can receive a maximum of 1 indirect ration.
There are three steps in the process of food distribution at the FDPs: (1) verification of
eligibility based on information in the beneficiary ration card; (2) collection of food by the
beneficiary; and (3) verification by the food monitor that the ration received corresponds to the
ration allocated, and signature of the ration card.
The key findings of our evaluation of the operations and quality of services at the FDPs are
presented below, along with a summary of the perceptions of beneficiaries and staff.
Operations and quality of services at the Food Distribution Points
Attendance and beneficiary/staff: Attendance at the FDPs is very high (average of 240
beneficiaries, with a maximum of 375), and so is the beneficiary/food monitor ratio, with an
average of 77 (ranging up to 187). Not surprisingly, the time involvement for participants in this
activity is extremely high, averaging 6 hours including travel time (average 1 hour each way) and
the time spent at the FDP (average 4 hours). This varied from 2.5 hours in the best case to a very
high 9 hours in the worst case. The main reasons for the large amount of time involved in
receiving the food are delays in both the food and the staff getting to the FDPs. For example,
although beneficiaries are typically asked to come to the FDPs at 8:00 a.m., 4 of the 10 FDPs
observed did not start their activities until between 11 and 12 a.m. Food was late to arrive in 3
out of 10 FDPs and food monitors were late in half of the FDPs observed.
Amount of food received: Our measurement of the amount of food received by different types of
beneficiaries showed that, on average, they received close to one-half a kilo less of the SFB than
allocated and a quarter of a kilo more oil. On average, the amounts of WSB and lentils received
differed from the amounts allocated by only approximately 100 g. Assuming that a margin of
error of 200 g is acceptable, only 16 percent of the beneficiaries received the correct amount of
SFB, less than half received the correct amount of WSB, approximately half received the right
amount of oil, and 68 percent the right amount of lentils. The magnitude of the gap between the
amounts allocated and the amounts received ranged from –2 kg to +4 kg for SFB and from –2 kg
to +2 kg for WSB, the two most problematic commodities in that regard. These levels of errors
are very high, especially for WSB—a difference of ±2 kilos represents 25 percent of the 8 kg
allocated to a beneficiary child (note that WSB is provided only as a direct ration to beneficiary
children; it is not included in the direct ration for pregnant and lactating women nor in the
indirect ratio for other family members). Of course, from the perspective of the program and the
total amount of food distributed, these errors tend to cancel out. However, from the point of
view of the beneficiaries, these problems lead to feelings of injustice and dissatisfaction with the
program.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xiii
The food commodities were usually measured using containers tailored to the amounts allocated
for each of the commodities. Clearly, this approach is not as exact as using a scale, and our
results confirm that it seems to result in significant levels of errors.
Quality of the food commodities: Problems with the quality of the food were observed only in
one FDP, where three of the five bags of SFB inspected were visibly humid from the outside and
the product quality had been altered. Neither visible infestation of food commodities with
insects or worms, nor any visible color changes, were observed or reported in the interviews.
The use of food rations
Our interviews with beneficiaries showed that, on average, the commodities received lasted two￾to-three weeks, but none lasted for the entire month. Sharing of the commodities among all
family members was almost universal for SFB and for lentils, but not for WSB. Approximately
40 percent of the beneficiaries reported feeding the different WSB-based recipes mostly to the
beneficiary child. For the other products, and especially for lentils, only between 5 and 10
percent of the mothers reported feeding the donated foods mostly to the beneficiary child. A
large majority (70 percent) of beneficiaries reported sharing the food commodities either with
other family members (not living in the same household), or with neighbors or other people.
These results confirm the well-recognized sharing of donated food commodities among family
members and in many cases also with other families and individuals. Although there is some
indication that the WSB—which is specifically targeted to the young child and has higher levels
of multi-micronutrient fortification than the other commodities—is used preferentially for the
young child, more than half of the households (60 percent) reported sharing WSB with other
family members.
The practice of sharing donated commodities reflects the reality of rural Haiti, where most
families are extremely poor and rely heavily on social networks, and where sharing of received
goods is an obligation in order to benefit in turn from goods that others may receive. A few
families also admitted to selling some of the donated commodities at times, in order to purchase
other basic needs. It will be difficult for the program to change these practices unless other
supporting program activities can be put in place to significantly increase the resource base
available to these poor households.
Staff perceptions and suggestions for improvement of the Food Distribution Points
Perceived problems: The program staff identified several concerns regarding the functioning of
the FDPs, and most of them were related to logistical and organizational aspects. Delays in the
start of the food distribution due to the late arrival of the food were perceived by the health
agents as a major operational problem. The food monitors and their supervisors also expressed
their concerns regarding the start time of the food distribution, and mentioned that adherence to
the schedule of food distribution was also an issue. Differences in the list of beneficiaries
included in the monthly distribution lists and the master list, which includes all beneficiaries with
information about entry/exit dates in the program, were also noted by both the health and the
commodity staff. The problem related to the beneficiaries not having their ration cards with
them was also raised by all groups. Problems with the lists and/or with the ration card are a real
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xiv
concern because they result in some beneficiaries returning home without their food ration. This
causes a lot of frustration among beneficiaries and dissatisfaction with the program, and also
results in problems for the staff.
Other concerns raised mostly by the health agents included crowding at the FDPs, a lack of
respect from the food monitors towards the beneficiaries and the health staff, and the non￾hygienic handling of foods in some cases.
Suggestions for improvements: The health agents recommended that the program management
addresses the overall logistical and punctuality issues by providing support to ensure the timely
arrival of the food at the FDPs and enable adherence to the scheduled distribution dates. They
also made two specific suggestions on how to reduce crowding, i.e., establishing smaller
distribution points and scheduling beneficiaries from different zones to arrive at different times.
They also recommended that additional efforts be made to ensure that all eligible beneficiaries
receive their food. They suggested that this problem could be addressed by encouraging the
health staff and the commodity staff to work jointly on the preparation of the lists. The health
agents also recommended that hygiene in food handling be improved and that local containers be
standardized to ensure that beneficiaries receive the amount of each commodity they are entitled
to.
Beneficiaries’ perceptions regarding the Food Distribution Points
As expected, the food distribution component of the program was well appreciated by the
respondents. Many indicated that they particularly appreciated the WSB because it contains
vitamins, which keep children healthy. Other respondents felt that the food distribution helped
them to cope with food insecurity, and prevent hunger.
Perceived problems: The problems expressed by the beneficiaries were also not entirely
unexpected, considering our findings reported above. Most beneficiaries considered that the
time involved in attending a FDP was a major obstacle to the use of the service, and it was due to
a combination of the long distances traveled to and from the FDP and delays in the start of the
food distribution. A significant number of respondents expressed concerns about the food, and
especially the WSB and the SFB, which they believed caused diarrhea. In some cases, this was
reported in adults as well as in children, but only when they first started to consume the product
(50 percent). For others, diarrhea was believed to result when the foods were consumed
frequently. It is important to recognize that in environments such as these rural Haitian
communities, where the incidence of diarrhea is generally so high (Menon and Ruel 2003), it is
difficult to tell whether the donated foods are really responsible for the reported diarrhea.
Suggestions for improvement: Only a few beneficiaries proposed suggestions on how to improve
the food distribution. Their recommendations were very similar to those made by the health staff
and focused mainly on ways to improve the logistics and punctuality issues. They also
recommended that the program staff visit the local markets after a food distribution day to assess
the extent of food selling. Finally, they indicated their wish that ration sizes be increased and
that everybody receive food donations so that there is no need for sharing with non-recipients.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xv
Conclusions on operations at the Food Distribution Points
In summary, our data confirm the existence of logistical constraints related to the transport and
distribution of food commodities in the study area. Transport problems, which affect the timely
arrival of both food and staff, are difficult to overcome in areas such as the Central Plateau,
which are characterized by extremely poor road conditions, especially during the rainy season.
Other problems, such as excessive crowding and long waiting times, which are also affected by
logistical constraints inherent to the environment, continue to be a challenge but need to be
addressed. Solving the measurement problems to ensure that beneficiaries receive the amount of
food they are entitled to is also important for fairness as well as to ensure efficient use of
resources. The sharing of food commodities is probably unavoidable, but the program should
continue to emphasize the importance of prioritizing the targeted beneficiaries within the family,
and the differential use of selected food commodities such as WSB for young children who have
high nutrient requirements. The other commodities, which are provided specifically to
complement the diet of other family members, can continue to be used for this purpose.
Staff perceptions of program organization and management
Data from focus group discussions with five categories of program implementers were used to
assess their perceptions regarding various aspects of the program’s organization and
management. The focus groups were organized as follows: two groups each of colvols and
health agents, and one group each of MCH supervisors, food monitors, and commodity
supervisors. The data from the focus groups were supplemented by semi-structured interviews
with the national and regional health coordinators and the regional commodity officer.
A consistent finding from these focus groups is that, at all levels, staff feel that they are engaged
in important activities that are leading to improvement in the lives of the people the program
serves. They believe that their jobs entail serious responsibilities, and they express confidence in
their preparation and ability to meet these responsibilities. The conviction that they are
contributing to the development of the communities in which they work, and facilitating
behavior change in individuals and families, are important sources of motivation. Many also
find the work itself enjoyable.
The acquisition of professional status and the value of training were also highlighted by
community-level staff as a source of job satisfaction and a factor in the development of their
capacities to carry out their jobs. Some of the respondents contrasted the current training
orientation with their previous experiences, emphasizing not only the new technical knowledge
they have acquired, but also that they have developed improved modes of interacting with their
clients (e.g., using adult learning principles in the design of their education activities).
At every level of the program delivery system, staff called attention to expansion in their
workloads. Various perspectives were expressed concerning the significance of this fact. While
some felt it to be a source of discouragement, this was not universally the case. One
consequence is that expanded work time puts greater pressure on the availability and
management of time for other personal obligations (e.g., in agriculture and other domestic
activities). For community volunteers (colvols), a significant source of tension vis-à-vis health
agents is that they see themselves as working as hard as the agents without appropriate financial
compensation.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xvi
The operations research identified five issues that negatively affect the satisfaction people
experience in their jobs. For health agents and community volunteers (colvols), these are:
(1) lack of respect from supervisors and higher management levels; (2) low salaries and
problems with payment arrangements; (3) lack of transport and supplies to facilitate their work;
(4) responsibility for too many activities; and (5) weak planning and coordination between the
different levels in the program. MCH supervisors expressed concern only about lack of transport
and supplies to facilitate their work and responsibility for too many activities. Commodity
supervisors noted only lack of transport and supplies to facilitate their work as a problem that
affects their job satisfaction.
At each level, staff were asked to reflect on their relationships with their direct supervisors. At
every level, from colvols to supervisors, positive comments on supervision reflected two general
features: (1) the feeling that supervision is stimulating and motivating; and (2) the view that they
received adequate technical guidance for problem-solving, planning, and other technical issues.
The value of technical guidance and appreciation for it was articulated in all the focus groups,
with the exception of the MCH supervisors, where this perception was not brought up in the
discussions. On the other hand, concerns were also expressed about how supportive supervisors
were. At the opposite extreme from “stimulating and motivating” was the view that supervision
was sometimes demoralizing (expressed in one of the colvol focus group discussions).
“Moderate support” and “nonsupport,” intermediate positions on this dimension, were also
expressed.
With respect to coordination between the health and commodity sectors, positive features of the
relationship noted by respondents emphasized the value of exchange of ideas, and the fact that
there is joint preparation for the food distribution, and joint transportation of staff. The themes
representing aspects of the coordination that were seen as negatively affecting the program were:
(1) insufficient communication and dialogue between the two teams, (2) insufficient
collaboration in preparing for the food distribution and during the distribution process, and (3)
lack of respect from the food team for the health staff and also for the beneficiaries.
In all of the focus groups, respondents were asked to provide their ideas about how to increase
support and address problems. A variety of suggestions were offered, ranging from providing
field staff with WV badges and transportation facilities to more training on job responsibilities
and technical health and nutrition issues.
In all of the focus groups, a number of suggestions were also made about how to improve
supervision. Suggestions covered a range of matters under the general topics of strengthening
coordination and collaboration between different levels in the program and shifting workloads to
provide increased opportunities for supervision.
Finally, in all seven focus groups, a number of suggestions were made about how to alleviate the
constraints for good coordination between the health and the commodity team. These ranged
from activities to increase dialogue and collaboration between the two teams to suggestions
about how to improve mutual understanding and respect.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xvii
Feedback to program management and plan of action developed by World Vision
to improve selected aspects of program operations and service delivery
The results of this first round of operations research were presented by the IFPRI-CU team in
Haiti in February 2004. The general presentation of the results to PVOs and USAID staff was
followed by a one-and-a-half-day-long meeting of the IFPRI-CU team and the WV-Haiti
management staff. The main objectives of the meeting were:
ƒ To review and discuss the findings of the operations research;
ƒ To discuss the constraints to implementation that were identified in our assessment and to
identify potential solutions to address these constraints and to strengthen program operations
and quality of service delivery; and
ƒ To prioritize the constraints to be addressed and the potential solutions to be implemented,
and to develop an “action plan” for implementing the selected actions to strengthen the
program.
A matrix was used to guide the discussions and to facilitate prioritization of the constraints to be
addressed by the program in the short term. The matrix consisted of five columns; the first two
columns listed the operational constraints identified at the different service delivery points and
for different activities; the next column was used to list the potential solutions identified by the
management staff to address the different constraints; the fourth column summarized discussions
regarding the feasibility of adopting the proposed solution(s) in the short term; and the fifth
column provided an assessment of the potential impact of addressing the different constraints
identified on program operations as well as on the overall impact of the program on its targeted
beneficiaries.
The process utilized by the IFPRI-CU-WV team to review and prioritize potential solutions to
the constraints identified was based on consideration of the following three criteria: (1) the
possibility of identifying a solution (corrective measure) that was within the scope of current
program activities; (2) the feasibility of implementing these corrective measures, given the
program’s current financial and human resources; and (3) the estimated impact of implementing
these corrective measures to strengthen a particular aspect of program operations on the overall
program’s effectiveness and impact. These latter criteria are important because it may be that for
some aspects of the program, very simple and low-cost solutions exist, which if implemented
could have a major impact on operations and on the effectiveness of the program. By contrast,
efforts to improve other aspects may not warrant the level of investment necessary to implement
corrective actions because their overall potential to improve the programs’ effectiveness is small.
These aspects were carefully gauged in the discussions held in Haiti.
A summary of agreements on the corrective actions to be implemented by the program at the
different delivery points is provided below and summarized in Table S2 (which is a reduced
version of the original matrix developed at the meeting and which presents only four columns,
focusing on the constraints identified at different service delivery points and for different
activities, the corrective actions to be implemented to address these different constraints and their
potential impact on effectiveness and impact of the program).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xviii
At the Rally Posts
The logistical constraints related to crowding, high participant/staff ratio and time involvement
for both staff and participants at the RPs were discussed in length, but few solutions were
identified to solve these problems in the short term. Given that the program is already perceived
as having a large number of health agents and colvols, it was not considered feasible to increase
the number of staff at this time. It was also considered inappropriate to recommend a major
effort to split existing RPs into smaller ones, because the staff is already stretched to their
maximum, with health agents being responsible for up to nine RPs per month. It was agreed,
however, that the management would continue to examine the situation on a case-by-case basis
and that RPs that are clearly unmanageably large would be reexamined and split if deemed
possible.
A suggestion was made by some of the health agents interviewed during the operations research
to improve the training of colvols in order to increase the support they provide to the program.
The management staff indicated that this measure had already been implemented and that colvols
were now being included in all field staff training. The colvols had also recently been
“promoted” and their salary was raised from 30 percent of the health agents’ salary to 50 percent.
This change was accompanied by a change in title, whereby the colvols are now referred to as
“agents promoteurs de santé,” or health promoter agents.
It was suggested that improving the venue of the RPs with some basic furniture, such as chairs,
benches, and tables, could facilitate the logistics and organization of the RPs. The management
staff agreed to follow up on this aspect and felt that it was feasible to mobilize community
resources to improve the venue of the RPs.
Finally, the group revisited the program’s proposed sequence of activities at the RPs that,
according to our operations research, was not particularly effective in preventing bottlenecks at
registration and long waiting times for participants. The group agreed on a revised sequence of
activities to be tested and implemented if effective (see Table S2, second row, third column, for
the new sequence). The main changes in the proposed sequence of activities included: (1)
giving a number to each participant as they arrive at the RP, (2) weighing the children before
they are registered, (3) evaluating the nutritional status and individual counseling of the mother
along with registration rather than with weighing, and (4) providing the deworming, vitamin A,
and ORS tablets as the last activity for participants (along with immunization), rather than
providing these at the time of registration.
It is hoped that these relatively simple modifications in the RP operations will help improve the
flow of participants through the different activities, and reduce the time burden for both health
staff and participants, making the RP generally more efficient and pleasant. Avoiding
bottlenecks may also help improve the quality of services if it allows the health staff to dedicate
slightly more time to each participant and if the overall environment is less crowded.
Education. The group agreed that it was worth pursuing efforts to increase the coverage of the
education sessions by offering more than one education session at the RPs. It was felt that this
could easily be achieved by ensuring that supervisors reinforce the importance of providing
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xix
additional education sessions to accommodate late arriving participants. There was a general
consensus, however, that the RPs were not the most suitable venue for effective education, given
the generally crowded, busy, and noisy environment and the number of activities taking place
simultaneously. It was therefore agreed that the education sessions at the RPs should be used to
deliver simple messages on topics of general interest, such as hygiene, immunization, or family
planning. The RPs were to be considered as a complementary opportunity to cover additional
topics and to reinforce some of the messages provided at the MCs (the main venue for the
education and BCC strategy). The team also agreed that supervisors needed to be more active in
promoting the use of the 12-month calendar of education topics to ensure that participants were
exposed to the complete set of topics planned by the program, which ultimately should help
increase their general knowledge of health, nutrition, and hygiene.
Supplies. The lack of supply of vitamins, ORS, deworming tablets, and vaccines at the RPs
appeared to be mostly due to a problem with the main providers (Ministry of Health, UNICEF,
etc.). It was therefore considered important that the program management continues to pursue
every effort to ensure constant supply of these important preventive health services, but no action
was recommended for field staff because it was considered to be outside of their control. High
coverage of these preventive services is extremely important in the context of MCH programs, as
they are key complementary strategies to improve child growth, health, and survival and should
therefore continue to be prioritized by the program management.
Growth monitoring and promotion. Growth monitoring and promotion in the context of the WV
MCH program has two main purposes: (1) to identify malnourished children in the recuperative
program communities; and (2) to communicate with mothers (from both preventive and
recuperative groups) about their child’s nutritional status and growth and provide individual
counseling. For the first purpose, it is important to minimize measurement error in order to
reduce misclassification of children during the screening process. For the second purpose, i.e., to
communicate with the mother, it is important that the health staff spend some time with the
mother to inform her of the results of the weighing and nutritional status assessment, and to
provide her with specific advice regarding her child’s progress. Problems with both of these
aspects were identified through our operations research and the solutions proposed at the meeting
to address them are summarized below.
With respect to the measurement problems, again the group agreed that given the environment at
the RPs and the equipment used to measure children, a certain level of error is to be expected. It
was agreed, however, that re-training and increased supervision of the health staff will be used to
reduce measurement errors and misclassification problems. One simple recommendation, which
according to the results of our operations research, could significantly reduce measurement error,
is to make sure that the child’s weight is recorded when the needle of the scale is stable (as
opposed to when it is moving significantly). Plans to re-train and improve supervision of the
plotting of children’s weight on the growth card were also agreed upon.
The discussions on process of GMP and the communication with mothers led to an agreement
that all mothers should be informed about the child’s weight, nutritional status, and progress
since last weighing and that a brief of individual advice should be provided. The group was very
clear about the limitations of conducting GMP in the context of the RPs, and especially with
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xx
regards to the time that the health staff can allocate to individually counseling each mother about
her child’s growth. Given the crowded and noisy environment and the severe time constraints
faced by the health staff, it was agreed that the brief advice would consist of praising and
encouraging the mother if her child is growing well. If her child was not growing well, the
advice would consist primarily of a recommendation to follow up with other program activities
and to adopt the recommended practices discussed at the MCs (e.g., exclusive breastfeeding, use
of enriched porridges, etc.).
At the Mothers’ Clubs
Implementation of the new BCC strategy in 2003 required significant reorganization of the MCs,
especially in the preventive program group. This is because the strategy emphasizes the
importance of the timely delivery of education messages to mothers, based on the specific age
and developmental stage of their child. This approach is based on the premise that a much
greater impact on behavior change is expected if the intervention reaches the beneficiaries at
their best learning moment. Our findings showed that there were still some problems with the
reorganization of the clubs and that some mothers were in the wrong groups relative to the age of
their child, and thus likely to receive education messages that were not particularly relevant.
Management staff agreed that improving this aspect of the MCs was feasible since so much had
been achieved already, but that it would require strengthening supervision in the field. The
management staff emphasized the need to strengthen the training of the supervisors themselves
in both the rationale for the recommended structure of the clubs and how to operationalize it. It
was also agreed that one modification to the current system would be made in the preventive
group, which now uses two age groupings for the MCs for beneficiary children: mothers of
children 6-8 months and mothers of children 9-23 months. The change recommended was to
replace these two groups with the two groups that were originally planned: 6-11 months and 12-
23 months. These two groups are better balanced and include children with more similar needs
relative to child-feeding practices. It was also agreed that the MCs with children 6-11 months
would emphasize the differences in recommended practices for children 6-8 and 9-11 months of
age regarding the number of meals, and the consistency and quantity of food.
Remarkable improvements in the quality of education delivered at the MCs were achieved since
health staff were trained in the implementation of the new BCC strategy in 2003. The
importance of effective supervision and feedback were emphasized during the meetings as being
key to maintaining the quality of the intervention, and even more important, to maintaining the
enthusiasm and motivation of the staff, and the interest of the beneficiaries.
At the Food Distribution Points
The logistical problems related to the distribution of the food, and especially the transport of
food and staff to the point of distribution, and the time burden of this activity for staff and
beneficiaries, were discussed at length in the meeting. The commodity team is fully aware of the
problems highlighted by our operations research and many of the constraints identified have
already been addressed since our data were collected. The commodity staff have been actively
working on improving the logistics of the food distribution, including taking measures to
improve local storage facilities. Transport and other logistical problems, however, can never be
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
xxi
fully overcome in areas such as the Central Plateau, where road conditions are so bad and the
availability of fuel continues to be a major constraint. These problems are systemic to the
context in which this program operates. It is worth pursuing efforts to try to improve the
situation, but flexibility and patience will always be required by staff at all levels and by the
beneficiaries themselves.
Errors in the amount of food received by the beneficiaries are also difficult to avoid, but the
management staff recommended that the existing supervision system be strengthened. This
system consists of having the supervisors systematically verify the ration received by a
subsample of beneficiaries as they leave the RP. This system is clearly worth reinforcing to
minimize errors in food allocation, and consequently improve beneficiary satisfaction.
The communication difficulties that had been reported between the commodity staff and the
health staff and beneficiaries respectively were also well understood by the management staff,
who indicated that these cases had been addressed individually by the supervisors or regional
coordinators on a case-by-case basis. Continued supervision will be required to maintain good
communication among all staff and with the beneficiaries.
Finally, the problems of hygiene in the distribution of food commodities, which had been raised
in our operations research, had also been addressed in part by the program. Plastic sheets are
now being used and the suggestion to provide gloves to food handlers was perceived as feasible.
An additional suggestion was to provide a small compensation to the food handlers to motivate
them to do their work more carefully and more hygienically. The possibility of acquiring larger
containers with a tap to distribute the oil more hygienically was also discussed. It is important to
recognize that these proposed approaches to increasing hygiene in the food distribution process
will require identifying the financial means to acquire the materials proposed.
Final comments
The action plan described above to address some of the operational constraints identified through
our operations research relies almost entirely on a strong and effective supervision system. At
the time of the meeting, the management staff indicated that they had already been taking steps
to strengthen the supervision of the program at all levels, and the health team presented their
newly developed supervision plan for the region.
Our next round of operations research will be conducted in 2004, starting in June. This will
allow some time for the WV team to experiment with the implementation of the corrective
measures agreed upon. Our operations research will focus on assessing the effectiveness of
implementation of these corrective measures and on studying the supervision structure. An
additional objective of the next round of operations research will be to assess differences in
implementation between the two program intervention groups; understanding of differences will
facilitate interpretation of the final impact evaluation findings.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
Table S2. Summary of discussions held in Haiti with World Vision staff on the operations research findings and follow up
corrective actions to be implemented
A - RALLY POSTS (RPs)
Aspects/activities Constraint(s) identified Corrective action to be implemented
Potential impact of improving this
aspect on effectiveness and impact of
program
Too crowded, too many beneficiaries; ratio
of participants/staff is too high
- Continue to work on a case-by-case basis to split the RPs that are
really too big and can be split
- Improve venue of the RPs by providing basic furniture (chairs,
benches, tables); this will be done through community
mobilization
- Will reduce staff and participants’ time
burden and frustrations
- May increase participation at RPs, and
increase program impact
- May increase quality of services at RPs
Organization
Registration is a major bottleneck; sequence
of activities varies a lot between RPs and
does not usually follow the implementation
plan
The following sequence was recommended:
1) give a number to each participant;
2) provide general education session
3) weigh child, plot weight on growth chart in health card;
4) do general registration (including register weight in registry);
5) evaluate nutritional status
6) inform and advise mother (see below in GMP)
7) give required immunization, deworming, vitamin A, and ORS
Supervisors will have to assist staff in implementing and testing
this sequence
See above
Specific activities (organization and quality)
Still mostly one session, first thing in the
morning; late arrivers miss the session
Give at least two education sessions and, if needed, up to three;
supervisors need to motivate staff to adopt this practice
Will improve coverage of participating
mothers, and possibly increase knowledge and
reinforce messages discussed at the MCs.
1. Education
- Quality of education is non-optimal
(messages tend to be vague, little use of
visual materials, limited interaction
between staff and participants)
- Monthly education topics recommended
by the program are rarely used
- Focus on simple topics and messages of general interest (e.g.,
ORS, immunization, family planning, hygiene)
- Ensure that the 12-month calendar of topics is distributed to all
supervisors and discussed in monthly meetings with health staff
- Supervisors will have to monitor accuracy and clarity of messages
and adherence to monthly schedule of topics
Following calendar of monthly topics will
allow mothers to receive education on the full
set of topics deemed important and of general
interest to mothers. This can increase their
general knowledge of health, nutrition, and
hygiene.
2. Vitamin A/
deworming/ORS
Many participants do not receive the
services and this seems to be due largely to
a lack of supply of the products
This problem is due to a lack of availability from the main
providers (Ministry of Health, UNICEF, etc.), and is therefore
outside of the program’s control, but efforts should be pursued to
ensure a constant supply
These preventive health services are very
important for child’s health, growth,
development, and survival. Therefore it is
important to achieve high coverage.
xxii
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
Aspects/activities Constraint(s) identified Corrective action to be implemented
Potential impact of improving this
aspect on effectiveness and impact of
program
Children are weighed, their weight is
recorded and their nutritional status
assessed, but there is little use of the
information to communicate with the
mother and to give her tailored individual
advice regarding her child’s growth and
progress
Retrain personnel, supervise and motivate staff to ensure that each
mother receives the following information:
1) Child’s weight
2) Child’s nutritional status
3) Child’s progress (gained or lost weight)
4) Brief message of encouragement (if child is growing well); and
recommendations for follow-up with other program activities
(e.g., MCs) if child is not growing well. If child is M3, make
sure s/he is scheduled for a home visit.
The information provided to mothers about
their child’s nutritional status and growth can
help raise awareness among mothers and
motivate them to engage more actively in the
BCC activities and to adopt the recommended
practices.
3. Growth monitoring
and promotion
Quality of weighing and plotting varies;
errors result in misclassification of children,
and thus in errors in screening beneficiary
children (only for recuperative group, which
uses children’s nutritional status as a
screening criteria)
- Retrain health staff to improve quality of weighing (making sure
that needle of the hanging scale is still before recording the
weight will help) and of plotting the child’s weight on the growth
chart.
- Supervisors need to ensure that information is provided to
mothers and that quality of weighing and plotting is improved.
Improving the weighing and plotting of the
weights in the growth chart will reduce
misclassification errors, thereby improving the
cost-effectiveness of the program.
4. Immunization Lack of supplies See above (vitamin A, ORS, and deworming) See above (vitamin A, ORS, deworming)
B - MOTHERS’ CLUBS (MCs)
Aspects/activities Constraint(s) identified Corrective action to be implemented
Potential impact of improving this aspect on
effectiveness and impact of program
Organization Problems with reorganization of
MCs for different subgroups of
mothers/children (especially in
preventive group)
- Ensure that supervisors understand the rationale for the
reorganization of the clubs, and continue to reinforce
importance of implementing this reorganization as
planned.
- Modify current system for preventive group, which is to
form two groups with 6-23 month old: 6-8 and 9-23.
Replace with the following groups: 6-11 and 12-23.
The importance of the timely delivery of the
education for the effectiveness of the BCC cannot be
overemphasized. A much greater impact on
behavior change is expected if the intervention
reaches the beneficiaries at their best learning
moment. Achievement of behavior change may, in
turn, reduce childhood malnutrition.
Quality of the education and BCC
intervention
Very good quality overall; marked
improvements since staff was
trained in new BCC; performance
varies, however, and improvements
can still be achieved
- Strengthen supervision and ongoing training of staff;
continue to use supervision checklists to provide
feedback to staff and to monitor progress.
- Strengthen communication of concepts and technical
content of sessions, as well as utilization of the
communication material and the adult education
techniques.
The greater the quality of the BCC in MCs, the
greater the potential impact of the program on
improved child feeding and care and consequently
on children’s nutritional status, health, and
development.
xxiii
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
C - FOOD DISTRIBUTION POINTS (FDPs)
Aspects/activities Constraint(s) identified Corrective action to be implemented
Potential impact of improving this aspect
on effectiveness and impact of program
Very large number of beneficiaries per
FDP and high beneficiary/staff ratios
Important delays in arrival of food and
staff at FDP
Organization
High time burden for beneficiaries and
staff
- Some organizational and logistical constraints have already
been addressed by the program, including improving local
storage facilities.
- Difficult to solve completely, because most logistical
problems are inherent to the environmental conditions
prevalent in the area.
Improving the logistics of food distribution
could reduce the staff and beneficiary time
burden and increase satisfaction.
Amount of food received by
beneficiaries is not always the amount
allocated by the program
Strengthen the existing system to ensure that supervisors
systematically check that the amount of food received by
beneficiaries corresponds to the amount allocated (on a
subsample of beneficiaries as they exit the FDP).
Could increase beneficiary satisfaction and
reduce food losses.
Quality
Communication difficulties reported
between commodity and health staff
and commodity staff and beneficiaries
Continue to address these problems on an ongoing basis, as
they occur; this as been done successfully in the past.
Could improve staff satisfaction with their work
and beneficiary satisfaction with the program.
Certain problems of hygiene in the
manipulation of the food during the
distribution process have been
identified.
- Suggestions have been made to provide gloves to food
handlers and a small compensation to motivate them to adopt
more hygienic practices.
- Use of plastic sheets and cleaning of measurement tools have
already been adopted.
- The possibility of acquiring large containers with a tap for
the distribution of oil will be assessed.
- Funds will be necessary to acquire the materials proposed
above.
Would reduce the risk of contamination of the
food.
xxiv
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
1
1. INTRODUCTION
1.1. Background
This report describes the operations research that was undertaken for the purpose of assessing the
effectiveness and quality of implementation and delivery of World Vision’s maternal and child
health and nutrition4 (MCH) program in the Central Plateau region of Haiti. The main purpose
was to identify features that may require modifications to strengthen program activities. This
research is part of a larger evaluation being conducted by IFPRI and CU in collaboration with
WV-Haiti to compare two models for delivering an integrated food and nutrition program, both
with a take-home food ration component. The two models, which are being implemented by
WV-Haiti, are: (1) the traditional recuperative approach, whereby children under 5 years of age
are targeted to receive food supplements, nutrition counseling, and follow-up when they are
identified as being underweight for their age; and (2) the preventive approach, which targets food
supplements and other preventive interventions to all children below 2 years of age, irrespective
of their nutritional status.
In addition to conducting the impact and operations research of the two program models, the
IFPRI-CU team also provided WV-Haiti with technical assistance for the development and
refining of the program models. The focus of the technical assistance was the strengthening of
the education and communication component of the MCH program. Thus, following an
intensive formative research study in 2002 (see Menon et al. 2002a and Menon et al. 2002b), the
IFPRI-CU team worked with WV staff to develop a Behavior Change Communication (BCC)
strategy, and to define the implementation of this BCC strategy within the separate contexts of
the two program models (see Loechl et al. 2003a and Loechl et al. 2003b).
The full field-based implementation of the BCC strategy in conjunction with the other program
components (i.e., the food distribution and preventive health-care services) started only in May
2003. Data collection for the operations research described in this report was initiated in July
2003, three months after the program became fully operational, and concluded in September
2003. Thus, the results presented here depict the program as it was in mid 2003, i.e., more than
six months prior to the finalization of this report. As highlighted in the final section of the report
(Section 7), several of the operational constraints identified in our research have already been at
least partially addressed since our data were collected, as part of the program’s routine
monitoring and strengthening activities.
1.2. Organization of the report
The report is structured as follows. Section 2 presents an overview of the objectives, design, and
methodology of the operations research. The following three sections describe the results of the
operations research carried out at the three main program delivery points, i.e., the Rally Posts
(Section 3), the Mothers’ Clubs (Section 4), and the Food Distribution Points (Section 5). This is
 
4 The World Vision program does include a strong nutrition component and should therefore be referred to as “maternal and child
health and nutrition program” (MCHN); however, we use the term MCH in this report in order to be consistent with the
terminology used by World Vision to refer to its program.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
2
followed by Section 6, which describes staff perceptions of program organization and
management. Issues such as perceptions of the staff relative to their roles, motivation and
incentives, and staff supervision and coordination are examined. The report concludes with a
description of the process and outcomes of our discussions with WV-Haiti to design a plan of
action to improve program operations and service delivery and quality based on the results of the
operations research (Section 7).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
3
2. THE OPERATIONS RESEARCH
2.1. Objectives
The main purpose of the operations research reported here was to gather information on the
effectiveness of delivery of the different components of the program and to determine whether
the program was operating as planned. This first round of operations research was designed
primarily as a “troubleshooting” exercise; i.e., its main purpose was to identify bottlenecks or
constraints and possible solutions to ensure smooth implementation of the program and its
various components.
The specific objectives of this first round of operations research were:
ƒ To assess the implementation of the program, using operations research methodologies to
identify constraints and potential solutions to improving program operations.
ƒ To assess the quality of delivery of the intervention with respect to the planned delivery
system (for example, quality of the different services provided by the local staff, quality of
the food distributed, quality of the education provided).
ƒ To explore the perceptions of different stakeholders about the program with a special
emphasis on their perceptions regarding its effectiveness, the quality of services provided,
and their roles and responsibilities within the program structure.
2.2. The system studied in the operations research
The WV MCH program (both preventive and recuperative) offers services at five major points of
contact between program staff and participants. These are: (1) the Rally Posts (RPs), where
health education, growth monitoring and promotion, and preventive health care are provided and
beneficiaries are identified; (2) the Mothers’ Clubs (MCs), where smaller groups of participants
gather to discuss health and nutrition topics in the context of the program’s behavior change and
communication (BCC) strategy; (3) the Pre- and Postnatal Consultations, where pregnant and
lactating women receive preventive health care and education; (4) the Food Distribution Points
(FDPs), where beneficiaries receive their monthly food rations; and (5) the Home Visits, where
beneficiary households with a newborn infant, a severely malnourished child, or a child with
growth faltering are visited by the WV health staff.
The main beneficiaries of the preventive MCH program are all children between 6 and 23
months of age who reside in the program areas, whereas the beneficiaries of the recuperative
MCH program are malnourished children5
 between 6 and 59 months of age who reside in the
program areas. In both programs, pregnant and lactating mothers (until their infant reaches 6
months of age) are targeted as well.
 
5 Malnourished children are defined as M2 and M3 according to the Gomez classification. In this classification, normal (N)
corresponds to ≥ 90 percent of the median of the weight-for-age CDC/NCHS/WHO standards; mild malnutrition (Grade M1), to
75 percent - < 90 percent; moderate malnutrition (Grade M2), to 60 percent - < 75 percent; and severe malnutrition (Grade M3),
to ≤ 60 percent (Cogill 2003).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
4
Severely malnourished children between 24 and 59 months of age in the preventive program are
also eligible to participate in the preventive program. These children (classified as M3 according
to the Gomez classification) are identified through the regular growth monitoring and promotion
activities done at the RPs. The services provided for the severely malnourished children in this
age group include: (1) distribution of food rations for 9 months, (2) two meetings for the mothers
where issues related to malnutrition and recuperation are discussed, and (3) home visits by health
agents during the first weeks after identification.
The RP is the entry point in both programs, and is used to refer beneficiaries to the appropriate
program services. New beneficiaries are identified at the RPs every month; eligible children are
admitted into the program on a monthly basis, whereas pregnant and lactating women can enter
the program only every four months. The upper age limit for admitting children into the
preventive program is 18 months, to ensure that all children in the program receive food aid and
other services for at least six months (up to 23 months of age). For the recuperative model, there
is no defined upper age limit for admittance, although there is an upper age limit for eligibility
for program services. For instance, a child can be admitted at 58 months, but s/he would have to
exit the program one month later when she/he reaches the age of 59 months.
For mothers of children 6-23 months old in the preventive program and mothers of malnourished
children in the recuperative model, monthly attendance at the RPs and at MCs is mandatory to be
eligible to receive the monthly food rations offered by the program. Pregnant and lactating
women are also required to participate in MCs and pre- and postnatal consultations to be eligible
for the monthly food rations (see Figure 2.1).
For the first round of operations research, it was decided that the study would begin with an
examination of the services delivered at the RPs, MCs, and FDPs and would conclude with
interviews with beneficiaries in their homes. The latter were to be conducted with women two
weeks after they had attended an MC meeting.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
5
Figure 2.1. Beneficiary requirements for participation in the World Vision MCH program
RALLY POST
Identification of beneficiaries
Preventive program: Pregnant and lactating women up to 6 months, mothers of children 6-23
months of age
Recuperative program: Pregnant and lactating women up to 6 months, mothers of malnourished
children 6-59 months of age
Mandatory monthly attendance by identified beneficiaries/caregivers at:
Rally Post and Mothers’ Clubs
(for mothers of beneficiary child)
Mothers’ Clubs and
Prenatal/postnatal consultations
(for pregnant/lactating women)
Eligible to receive food rations at
Food distribution points
2.3. The key stakeholders included in the research
For the purpose of this research, the main stakeholders included administration and field staff at
different levels of the program as well as program beneficiaries. The roles of these different
stakeholders in the program structure are outlined below.
ƒ World Vision health agents and colvols. The health agents and colvols are the direct
implementers of the MCH program in the field and thus, are the frontline staff in contact with
the program participants. They are in charge of the interventions being delivered at the RPs
and MCs and of assisting the food monitors at the FDPs. They attest to the attendance by
beneficiaries at the different MCH activities (which determines their eligibility to receive
food rations).
ƒ World Vision food monitors. Food monitors are responsible for the distribution of the food
rations at the FDPs and for ensuring that only eligible beneficiaries receive the food. They
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
6
are also responsible for verifying the beneficiary lists provided by the MCH supervisors.
This verification is done mainly by consulting the beneficiary card, which contains
information about the beneficiaries and indicates whether or not they have complied with
their attendance requirements at the MCH activities.
ƒ World Vision health (MCH) and commodity supervisors at local level. MCH supervisors are
nurses responsible for the supervision of the health agents and colvols, and commodities field
supervisors are responsible for the supervision of the food monitors. The MCH supervisors
generate monthly lists of beneficiaries eligible to receive the food rations, based on lists of
attendance prepared by the health agents for each service delivery point. The Commodity
Section uses these lists to program the food amounts needed per distribution point.
ƒ World Vision management staff at the national and regional levels in MCH and Commodity.
We identified several key informants at the regional level: (1) the Assistant of the Regional
Health Coordinator who supervises the MCH supervisors, (2) the Assistant of the Regional
Commodity Officer who supervises the commodity field supervisors, (3) the Regional Health
Coordinator, and (4) the Regional Commodity Officer at the Hinche regional office, both of
whom are responsible for the WV activities in the Central Plateau. At the national level, the
key informant was the National Health Coordinator at Port-au-Prince headquarters, who
oversees the implementation of the various health activities in all the program areas of WV in
Haiti.
ƒ Beneficiaries. Program beneficiaries eligible for the food rations are all pregnant and
lactating women, and malnourished children 6-59 months old (in the recuperative program
areas) and children 6-23 months old (in the preventive program areas). The
pregnant/lactating women and the mothers of the targeted children are the key recipients of
all BCC activities. As described in the previous section, beneficiaries receive food rations,
nutrition counseling, and other preventive interventions. They must meet the conditions set
by the program in order to benefit from the food rations (i.e., regular attendance at RPs, MCs,
Pre- and Postnatal Consultations).
2.4. Methodology
The operations research used a variety of qualitative and quantitative research methods. These
included: (1) structured observations at the different program delivery points; (2) structured
interviews with beneficiaries; (3) semi-structured interviews with various types of stakeholders,
including national and regional health coordinators and regional commodity officers, health
agents, and beneficiary women; and (4) focus group discussions with WV program staff. A
description of each of these approaches is provided below.
Structured observations, as well as the semi-structured, individual interviews and focus group
discussions with program staff were used to identify bottlenecks and constraints in the
implementation of different program activities (first objective of the research).
In order to assess the quality of intervention delivery (second objective of the research),
structured observations and semi-structured interviews with beneficiaries were used.
Finally, qualitative methods (focus group discussions and semi-structured interviews with
program staff and beneficiaries), as well as structured interviews with beneficiaries, were used to
explore attitudes, opinions, and the level of satisfaction of the different stakeholders (third
objective of the research).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
7
2.4.1. Structured observations
2.4.1.1. Observations at Rally Posts
A tracking form was designed to follow randomly chosen caregivers who were accompanying a
food aid beneficiary child through the different services provided at the RP. The main purpose
was to observe the different activities that took place in the RPs, the quality of services, and the
duration and sequence of the activities from the caregiver’s perspective. Therefore, we
concentrated on the following aspects:
ƒ Time allocated to the different activities. The fieldworkers noted each activity of the
caregiver and the start time of this activity. Activities recorded included both program
activities (registration, education, child weighing, deworming, immunization, etc.) as well as
nonprogram activities (feeding the child, cleaning the child, talking to a friend, etc.).
ƒ Unused services and reason for non-usage. Using a checklist of all program services that are
supposed to be offered at the RPs, the fieldworkers checked off each of the services not used
by the caregiver on the day of observation and asked the reason for non-usage. When
necessary, the fieldworkers verified information in the health agents’ registers to verify the
accuracy of the caregivers’ response.
Growth monitoring and promotion (GMP). The observation of this activity included the
following aspects: who was involved in this activity, the assessment of the quality of method of
measurement, the reweighing of the child on an electronic scale to determine the extent of
measurement error and misclassification (into malnourished/versus well-nourished child), the
assessment of whether the plotting was done accurately, and the assessment of the type and
quality of interaction of the health agent or colvol with the caregiver.
Education session. The purpose of observing the education sessions was: (1) to document some
technical aspects, such as who conducted the session and whether any communication materials
were used for the session; and (2) to assess the quality of the education/communication session
by observing the interaction between the educator and the mothers (e.g., whether the educator
was engaging and dynamic, whether s/he asked questions and was successful in soliciting
responses, etc.), and observing the general ambience in which the session was conducted (noise
level, distractions, etc.).
Status of cards. The fieldworkers checked whether the participating mothers had their health
card and their ration card for the child. They then verified the content of the cards, including the
following aspects: (1) whether the weight of the child was entered, whether it was properly
plotted, and whether information on vitamin A, vaccination, and deworming was entered on the
health card; and (2) whether information was properly entered regarding the type of beneficiary
the child was, and whether the attendance at the RP was noted on the card. If information on
vitamin A and deworming was not entered on the card, the fieldworkers used the health agent’s
register to verify the last date that the child received a vitamin A capsule and/or a deworming
tablet.
An additional observation sheet was used to capture the general ambience at the RP. For
example, observations were made regarding the venue where the RP was held, where the
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
8
different activities took place, the types of activities that appeared to create bottlenecks to
participant movement through the RP, and how many health agents and colvols were present.
Finally, information was also obtained on attendance, either through observation or through
consulting the health agents’ daily reports of the RPs. The following data were recorded: the
total number of adult participants per education session, the number of pregnant and lactating
women, the total number of children under 5 years and under 24 months who attended the RP,
and the number of malnourished children identified.
2.4.1.2. Observations at Mothers’ Clubs
The objective of the observations at the MCs was to assess the quality of education provided at
this delivery point. It was particularly important to evaluate the education at this point because
the MCs are intended to be the primary venue for the BCC activities of the MCH program. The
observation instrument was based on an observation checklist developed by Freedom from
Hunger (FFH) for use with their Credit for Education programs in Haiti. The checklist focused
on the following aspects: technical content, session management and organization, facilitation
and teaching skills (small-group management, use of open-ended questions and visual materials),
attitudes displayed, and atmosphere at the learning session.
A separate form was used to obtain general data on the venue for the MC, the size of the group,
the number of mothers arriving late and the number of mothers who were absent. We were also
interested in knowing who else attended the Club meeting (e.g., children, grandmothers, fathers,
etc.) and how many of them were present. The fieldworkers also observed whether the meeting
started at the scheduled time and how much time the health staff dedicated to this activity
(measuring from the time the first participant arrived until the last one left).
2.4.1.3. Observations at Food Distribution Points
An observation form was designed (1) to capture the general organization and ambience at the
FDPs, and (2) to assess the quality of the food commodities distributed. Observation of the
general organization and ambience included the following: the location of the food distribution;
the number of food monitors, health agents, colvols, MCH supervisors and commodity field
supervisors present at the distribution point; the number of beneficiaries planned for and the
number present; whether all beneficiaries from different communities (localités) were invited to
come to receive their food ration at the same time; whether the food and staff arrived on time at
the FDPs; and the start time of the distribution.
Observation of the food quality focused on the following: visible infection of the food
commodities with insects or worms, visible moisture in the different food commodities and/or
any visible color change. This observation was done before the food was distributed to the
beneficiaries. In addition, five bags of wheat-soy blend (WSB), soy-fortified bulgur (SFB) and
lentils were randomly checked at each distribution point to see whether the bags appeared humid
from the outside and to check their expiration date. The expiration date of five containers of oil
at each distribution point was also verified.
2.4.2. Structured interviews with beneficiaries
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
9
Structured questionnaires were used to conduct exit interviews with beneficiaries at various
delivery points. The questions concerned access, services utilized, and perceptions of these
services. Beneficiaries were also asked to provide suggestions on how the program could be
improved.
2.4.2.1. Exit interviews at Rally Posts
At the RPs, two different types of exit interviews were conducted:
1) Exit interviews with the tracked respondents. These interviews targeted those beneficiaries
who had been involved in the tracking observations described above. In the exit interview,
the respondents were asked questions about access to the RP (type, time, and cost of
transport), perceived importance of the different services at the RP, and suggestions for
program improvement.
2) Exit interviews with randomly chosen caregivers who were bringing a food aid beneficiary
child to the RP. This interview included a larger number of topics than the exit interview
with the tracked respondents. In addition to the questions related to access and perceived
importance of the different services at the RP, the caregivers were also asked their
relationship to the child. If the respondent was not the mother of the child, further questions
were asked to learn whether the respondent usually brought the child to the RP, where the
child’s mother was and what she was doing when the respondent brought the child to the RP.
Information on the child’s date of birth and nutritional status (according to the health card)
was also collected. Finally, the respondent was asked to list the services that the child had
received that day and the reasons for non-usage of services.
2.4.2.2. Exit interviews at Mothers’ Clubs
The exit interviews at the MCs were conducted with randomly chosen program beneficiaries, as
they were about to leave the Club. Again, participants were asked questions related to access to
the Club, and their perceptions regarding the importance of various topics discussed at the MCs.
They were also asked how many times they had attended this MC. Finally, the birth date of the
child was obtained from the ration card and from the respondent.
2.4.2.3. Exit interviews at Food Distribution Points
Exit interviews were conducted at FDPs with randomly chosen caregivers who had a child in the
program. These interviews included the following questions:
ƒ Access to the FDP. The respondents were asked about transport to the FDP (type, time and
cost).
ƒ Ration card. The information collected concerned the type of program beneficiary that the
child was, how long the beneficiary had been receiving food rations (according to the
respondent’s recall), the number of direct and indirect beneficiaries in the household, and the
type of indirect ration received on the day of the interview.
ƒ Use of food. The types of questions asked regarding the use of food included: how long the
different food commodities usually lasted, what types of recipes families usually prepared
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
10
using this food, and which family members usually consumed the preparations made with
donated foods. The respondents were asked if they added different ingredients (for example:
eggs, goat milk, breast milk, cows milk, beans, groundnuts, dried fish, meat, liver, etc.) to the
recipes prepared for the beneficiary child/children using the food aid commodities, and if so,
how frequently they added the different ingredients.
ƒ Amount of food received. In order to verify whether the respondents were receiving the
correct amount of donated food rations, the fieldworkers reweighed the foods that the
respondents had received that day, using an electronic scale.
ƒ Perceptions of the food quality. The respondents were asked whether they ever faced
problems with the quality of the food and if so, what kinds of problems.
ƒ Time spent at the FDP. The respondents were asked how much time they had spent at the
FDP on the day of observation and how much time they usually spent there on previous
visits.
Finally, information was also collected on the relationship between the beneficiary child and
respondent, and the birth date of the child (comparing the information from the respondent and
the ration card).
2.4.3. Semi-structured interviews
Semi-structured interviews were conducted with several types of stakeholders, namely, the
national and regional health coordinators and regional commodity officer, the health agents at
RPs, MCs, and FDPs, and beneficiary women.
2.4.3.1. National and regional health coordinators and regional commodity officer
The objective of these individual interviews was to gather information about their perceptions
related to their current responsibilities, the constraints on their performance, the supervision
structure in the program, the coordination between Commodity and MCH staff, and to gather
their suggestions for program improvement.
2.4.3.2. Health agents at Rally Posts, Mothers’ Clubs, and Food Distribution Points
These individual interviews with the health agents were held at the end of the sessions at each of
the delivery points. The objective was to obtain information about their perceptions and
opinions regarding the three service delivery points, the types of problems they faced, and, for
the RPs, their opinion about the supply situation (vaccines, vitamin A capsules, ORS packets,
deworming tablets, health cards, ration beneficiary cards). They were also asked whether they
had any suggestions on how to improve the delivery of the services at the different delivery
points.
2.4.3.3. Beneficiary women
Semi-structured interviews with beneficiary women in their homes were conducted to capture
their perceptions about program services, their relative importance, and the types of constraints
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
11
they faced (if any) in using the services. The following topics were explored regarding the
different services provided by the program.
ƒ Rally Posts. The respondents were asked about the education session at the last RP they
attended, what topics they remembered, whether the ideas and information were new,
interesting, and useful to them. They were also asked whether they were able to use the
advice received at the RPs and if not, why not. Mothers were also asked to comment on the
child weighing at the RP: whether the health agent/colvol told them their child’s weight,
informed them of whether the child was growing adequately or not, whether they gave them
advice about their child, and if so, whether it was useful, and finally, whether they liked to
find out about their child’s weight when they came to the RP.
ƒ Mothers’ Clubs. The interviews with participants in the MCs were designed to find out about
their knowledge of the topics that were taught. Mothers were asked to describe topics
covered in the previous session of the MCs meeting (spontaneously and prompted) and
which ideas or information were new, interesting, and useful to them. They were also asked
about the feasibility of using the advice received at the MCs and possible constraints to using
it.
ƒ Use of food received from the program. The questions asked included the following: what
kind of food they had received, what types of recipes they had prepared with these food
commodities, who consumed these foods, whether they liked it, and whether they faced any
problems with the commodities. They were also asked about the food distribution process:
whether they had encountered any problems and whether they had any suggestions on how
this process could be improved.
ƒ Sharing and selling of food received from the program. Our main interest here was to find
out whether beneficiary families shared, sold or exchanged the donated food they had
received. If the respondent indicated that they felt that they had to share the food, they were
asked with whom they shared it, which types of foods they shared, how much of the food
they shared, and what their reasons were for sharing. If the respondent admitted selling or
exchanging the food commodities, they were asked which foods and how much of them they
had sold or exchanged, and what they did with the money or the services received in
exchange.
ƒ Suggestions for program improvement. Similar to other stakeholders, beneficiary mothers
were asked to provide their suggestions regarding how the different components of the
program could be improved, either from the point of view of the effectiveness of service
delivery or the quality of services.
2.4.4. Focus group discussions
Focus group discussions were organized separately for five different types of stakeholders:
health agents, colvols, MCH supervisors, food monitors, and commodity supervisors.
The focus group discussion was the only method used to obtain information from colvols, MCH
supervisors, food monitors, and commodity supervisors. For all groups, the objective was to
gather information on perceptions about their current responsibilities, workload and time
constraints, the supervision structure in the program, and the support they receive from the
program. They were also asked for their views about the coordination between Commodity and
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
12
MCH staff and the constraints on their performance, as well as their suggestions for program
improvement.
In the focus group discussions conducted with the food monitors and the commodity supervisors,
an additional discussion point was their experience with food distribution, whether they
experienced any problems, and if so, what types of problems.
The focus groups with the health agents, colvols, and the MCH supervisors were also used to
obtain information on their perception of the training (and re-training) that they received in 2003.
In addition, information was gathered in the focus groups with the colvols and the MCH
supervisors about their views of the three service delivery points, the types of problems they
face, and, for the RPs, their opinion about the supply situation (vaccines, vitamin A capsules,
ORS packets, deworming tablets, health cards, ration beneficiary cards).
2.4.5. Data analysis
The emphasis of the data analysis was to assess the overall implementation of the MCH program,
since the primary objective of this phase of operations research was to identify bottlenecks to
implementation of the program models. We did not aim to evaluate the differences in evaluation
of the two program models in this round of operations research.
Data entry of the quantitative information was conducted by the Institut Haitien de l’Enfance
(IHE). A standard statistical software package (SPSS) was used for the analysis of these data.
The focus group discussions, the semi-structured interviews with the health agents, and the semi￾structured interviews with beneficiaries were conducted in Creole. Extensive field notes were
taken during the interview. The focus group discussions and the semi-structured interviews with
beneficiaries were also audio taped. Every evening, the interview notes were transcribed in
French and complemented with the information from the audiotapes, preserving emic terms or
phrases in Creole where necessary. The field notes were organized under the themes of the
semi-structured interview guides to facilitate later analysis. The transcripts were then analyzed
using a qualitative data analysis software package called Atlas-ti.
2.5. Fieldwork logistics, geographic area and sample
2.5.1. Research staff and training
The field team for the data collection included the following staff.
ƒ Two supervisors. Cornelia Loechl (postdoctoral nutritionist) and Arsène Ferrus (IFPRI
consultant). A. Ferrus was also responsible for the semi-structured interviews with the health
agents at the RPs and FDPs as well as the collection of general data and observation at these
two service delivery points.
ƒ Two observers responsible for the quantitative data collection at the RPs and FDPs: Remy
Lafalaise and Mathieu Honoré.
ƒ Two fieldworkers experienced in qualitative research methods and responsible for conducting
the focus group discussions and semi-structured interviews with beneficiaries: Josianne
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
13
Loredan and Dominique Pierre Lenz. They were also responsible for the semi-structured
interviews with the health agents at the MCs.
ƒ A World Vision nurse, Elisabeth Elysée, to observe the education sessions at the MCs. The
person responsible for this task had to be extremely familiar with adult learning principles for
effective communication and with the adapted communication materials on infant and young
child feeding used in the study. Since it was not possible to identify a person external to the
program who met these conditions, we decided to assign this responsibility to a member of
the WV staff, in spite of the fact that she was part of the implementation team. She was an
excellent person for this task, having participated in all training of trainers’ sessions on the
new communication strategy and the adapted communication materials. Also, as part of the
trainers’ team, she had been directly involved in the training of health agents and colvols in
the use of the infant and young child learning sessions and thus, had a clear idea of what to
evaluate when observing an education session.
The training of the team, including the field-testing of questionnaires and interview guides, was
done in June 2003 and data collection took place between July and September 2003.
Ethical approval for the study activities was obtained from the CU Commission on Human
Subjects. Informed consent was obtained from all study participants before any data collection
was conducted.
2.5.2. Geographic area and sample
The operations research took place in the intervention area of the IFPRI-CU-World Vision
evaluation project, which covers 20 zones or clusters of communities (localités) in three
communes: Hinche, Thomonde, and Lascahobas. These 20 clusters of communities, with
program services for each cluster covered by one health agent, were selected from WV’s
program areas at the beginning of the evaluation. In each cluster, the health agent is assisted by
1-3 colvols. Ten pairs of clusters were constituted that were matched for distance to main road,
access to a dispensary, type of terrain, and access to WV’s private sponsorship program arm
(called the Area Development Program). Within each pair of clusters, one was randomly
assigned to receive the preventive program and the other one to receive the recuperative
program. A list of the 20 clusters with the type of program model they were randomized to is
provided in Annex 1.
The operations research studied one program delivery point, i.e., RP, MC, and FDP, for each
health agent (see detailed protocol in Annex 2 and summary in Table 2.1). A total of 19 RPs, 20
MCs, and 10 FDPs were observed. One FDP covers several clusters and the 10 FDPs observed
covered all 20 clusters of the project area. The MCs included in the sample reflected the
different categories of MCs, i.e., MCs for pregnant women and MCs for lactating mothers (in
both programs), MCs for mothers of children 6-23 months of age (in the preventive program),
and MCs for mothers of malnourished children (in the recuperative program).
Table 2.1. Summary of methods used in operations research and sample sizes
Research method
/stakeholder
Purpose Location and sample size
Structured observations
at main service delivery
Assess operations, flow of activities,
time involved in attending the
Unit of observation/ analysis is the service delivery point:
RPs: n = 19
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
14
points delivery point MCs: n = 20
FDPs: n = 10
In addition, at the RPs, information was collected on 38 tracked
beneficiaries (2 per RP), using both observations and interview
techniques (see below).
Structured interviews
with beneficiaries
Assess access, services used,
perceived importance of different
services and use of food
Unit of analysis is the participant/beneficiarya
RPs (n = 19):
Exit interviews with tracked respondents (n = 38)
Exit interviews with random sample of respondents (n = 59)
MCs (n = 20):
Exit interviews with random sample of respondents (n = 41)
FDPs (n = 10):
Exit interviews with random sample of respondents (n = 45)
Semi-structured
interviews with health
agents
Assess perceptions and opinions
regarding operations, quality of
services, constraints to program
implementation, and suggestions for
improvement
Unit of analysis is the health agent
RPs (n = 19)
MCs (n = 20)
FDPs (n = 20)
Semi-structured
interviews with health
coordinators and
commodity officer
Assess perceptions regarding current
responsibilities, constraints to
program implementation,
supervision, coordination, and
suggestions for improvement
Unit of analysis is the health coordinator/commodity officer
National health coordinator
Regional health coordinator (Hinche)
Regional commodity officer (Hinche)
Semi-structured
interviews with
beneficiaries in their
home
Assess perceptions regarding the
program services, their relative
importance, the constraints faced in
using the services, and suggestions
for improving the program
Unit of analysis is the beneficiary
In beneficiaries’ homes (n = 30)
Focus group discussions
with five types of
stakeholders (total of
seven focus groups)
Assess perceptions about current
responsibilities, roles, workload,
time constraints, supervision and
incentive structure, coordination and
training
Health agents (2 groups)
Colvols (2 groups)
Food monitors (1 group)
MCH supervisors (1 group)
Commodity supervisors (1 group) a
 Throughout this report, women attending the RPs are referred to as “participants,” whereas mothers and children attending the
MCs and the FDPs are referred to as “beneficiaries.” This differentiation is made because the RPs are open to all women of
reproductive age and children under 5 years in the communities served, whereas the other two program delivery points, the MCs
and the FDPs, are specifically targeted to program beneficiaries.
For the semi-structured interviews with program beneficiary women, appointments with two
women were fixed at each of 15 MCs during the exit interviews. The interviewers scheduled
these appointments to take place at home, 1-2 weeks after the MC meeting. Because the study
took place during the rainy season, appointments for home interviews could not be scheduled in
clusters with very difficult access. The total number of home interviews was 30.
The focus groups were planned with the assistance of the Assistant of the Regional Health
Coordinator and the Assistant of the Regional Commodity Officer. Two focus group sessions
were organized with health agents and colvols, and one focus group was planned with each of the
following groups: MCH supervisors, food monitors, and commodity supervisors. A total of
seven focus groups were held. Annex 3 lists the different focus groups and presents details about
the composition of each group.
The next three sections present the results of the operations research carried out at the RPs, MCs
and FDPs, respectively.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
15
3. PROGRAM OPERATIONS AT THE RALLY POSTS
This section describes the results of the operations research conducted at the RPs. First, the
expected sequence of activities at the RPs is described to provide context. This is followed by a
short summary of the methods and samples used to carry out the research, and a presentation of
the results. The results are described in relation to the three main objectives of the research, i.e.,
to assess the effectiveness of implementation of the different activities, the quality of service
delivery, and the perception of the main stakeholders of the program, i.e., the implementers and
users. As noted above, this round of operations research did not aim to assess whether
operational differences between the preventive and recuperative programs could be observed, so
most tables present the results for the sample as a whole.
3.1. Description of Rally Post operations according to the implementation plan
RPs are open to all pregnant women, mothers with children less than 5 years of age, and women
15 to 49 years old in the communities attended. Services provided include health and nutrition
education, growth monitoring and promotion of children under 5 years of age, immunization,
vitamin A supplementation, deworming, free distribution of ORS, and information about the
family planning component6
. The monthly weighing of children who are MCH beneficiaries in
either the preventive or recuperative program, and attendance at the RP by the child’s caregiver
are mandatory. Either the mother or another caregiver can take the child to the RP.
RP meetings are held on a monthly basis in each community and are managed primarily by the
health agent responsible for that community. The health agent is usually assisted by at least two
other health agents and two colvols. The sequence of activities described in the implementation
plan is as follows (also see Figure 3.1 below):
ƒ The education session is the first activity to be carried out at the RP. The sessions run for
about 10 minutes, depending on the topic. WV sets up a calendar of monthly topics, which
include the following topics: immunization, pre- and postnatal care, preparation for child
delivery, diet for pregnant and lactating mothers, weaning techniques, description of
kwashiorkor and marasmus, hygiene and environment, diarrhea and preparation of oral
rehydration salts, acute respiratory infections, family planning, and HIV prevention. It is
expected that appropriate visual materials will be used to communicate more effectively.
The health agent or colvol is also expected to conduct several sessions on the same topic at
the RP to allow all mothers to participate and to restrict the size of the group to 10-15
persons.
ƒ After the education session, the health agent registers attendance of each participant. The
health agent determines the type of vaccines to be administered, and whether the participant
is due to receive a dose of vitamin A or deworming tablets. She also updates the health cards
with the information on immunization and vitamin A supplementation, and for food aid
beneficiaries, s/he signs attendance on the ration card. This same health agent also does the
distribution of vitamin A (every six months for children less than 5 years of age), anti-
 
6
 World Vision offers hormone pills and three monthly injections. Women can receive these services administered by WV nurses
at mobile clinics, in health centers during pre- and postnatal consultations, or at Area Development Program clinics.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
16
helminths tablets (every six months to children 2-5 years of age), and oral rehydration salts
(ORS) sachets (three sachets per month per household).
ƒ The next activity at the RP is growth monitoring and promotion. Each child is weighed and
the weight is recorded on the growth chart printed in the health card kept by the caretaker. It
is expected that at least the caregivers of malnourished children, especially of moderately and
severely malnourished children (M2 or M3 for weight-for-age according to the Gomez
classification), will receive brief counseling about feeding practices and prevention of
childhood illnesses.
ƒ After growth monitoring and promotion, children are directed to receive their immunizations.
Children receive vaccinations based on their age and previous immunization history
(previously verified by the health agent in Step 2 above).
The activities at the RPs are identical for the recuperative and preventive programs, with the only
differences being the criteria for selection of food aid beneficiaries. In the recuperative program
group, children are targeted based on their nutritional status, whereas in the preventive program
group, the selection of beneficiaries is based on age.
Figure 3.1. Flow of activities at the RP
3.2. Methods used to assess the Rally Post operations
The data that provide information on the functioning of the different activities at the RPs in the
next sections were gathered using the following methods:
ƒ Observations to capture the general ambiance at the RPs (N = 19);
ƒ The tracking of caregivers through the different services provided at the RP (N = 38);
ƒ Structured exit interviews with the tracked respondents (N = 38) and with randomly chosen
caregivers who were accompanying a food aid beneficiary child, (N = 59);
ƒ Individual semi-structured interviews with health agents at the end of each observed RP
(N = 19);
ƒ Semi-structured interviews with beneficiary women in their homes (N = 30);
ƒ Focus group discussions with colvols and MCH supervisors (N = 3).
3.3. Operations and quality of services at the Rally Posts
Education
Colvol or health
agent:
¾ Conducts various
sessions in
groups of 10-15
persons for about
10 minutes on
topic of the
month
Registration
Health agent in charge:
¾ Records names,
vaccine, weight,
ORS, vitamin A,
deworming
¾ Distributes vitamin
A, anti-helminthes,
ORS
Growth monitoring
and promotion
Colvol:
¾ Weighs all
children < 5 years
¾ Records weight
in health card
¾ Does individual
counseling of
caretaker
Immunization
Colvol or health
agent:
Immunizes:
¾ children < 5
years
¾ pregnant
women
¾ women 15-49
years of age
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
17
The presentation of data on the operations of the different activities at the RPs follows the
sequence of activities as laid out in the implementation plan (see above), i.e., the education,
registration, growth monitoring and promotion, and immunization. Where available, the results
on quality of implementation are presented after results on the implementation itself. The results
on the functioning of these activities are preceded by a description of the general context of the
RP. In addition, general information on the number of participants attending RPs and the
number of health staff present, and findings on the internal organization of the RPs as well as on
time commitment needed for participants to attend the RP are included.
3.3.1. Attendance and participant/staff ratio
3.3.1.1. Number of pregnant/lactating women and children under 5 years
Table 3.1 presents the numbers of participants from each program group, as well as the total
number attending the RP. This information was collected by the health staff after the RP
meeting. The data indicate that the total number of pregnant and lactating women as well as
children under the age of five years was very similar in the two program groups. On average, 74
participants, not counting the accompanying caregivers of children and siblings, were reported to
attend a RP meeting. The range is high, indicating that RPs vary quite considerably in size and
therefore in workload for the health staff involved. Most of the participants reached the RPs on
foot. Only one of the interviewed participants used a mule to come to the RP. Nobody had to
pay for transportation. The average travel time to get to the RP was 19 minutes, varying between
1 and 90 minutes (n = 97).
The data on the tracked respondents (N = 38) and from the exit interviews (N = 59) indicate that
in general, children were accompanied by their mother (91 percent); only 3 percent (n = 3) were
brought by their father and 6 percent (n = 6) by their grandmother or another family member.
On average, the RPs observed hosted approximately 26 pregnant and lactating women and 49
children under the age of five years on the day of observation. Again, the data do not show
major differences between the two program groups, except with respect to the age of the children
attending. As expected, given the difference in targeting mechanisms used for the preventive
and the recuperative groups, the proportion of children who were under 24 months of age in RPs
from the preventive group was much higher than in the RPs from the recuperative group. Also, a
larger percentage of children attending the RPs in the recuperative area was identified as being
malnourished using the Gomez classification (35 percent) at the RPs in the recuperative program
group, compared to those in the preventive program group (9 percent). This is because the
prevalence of malnutrition tends to increase gradually with age, and thus, higher malnutrition
rates are usually found among older, compared to younger, children (Ruel 2001).
Table 3.1. Number of pregnant/lactating women and children under 5 years per RP, by
program group
Number of participants attended on day
of observation
Variable
name
Program
group
Mean
[N = 19] Range
Pregnant/lactating women q102 Preventive
Recuperative
25
27
0-72
6-61
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
18
Number of participants attended on day
of observation
Variable
name
Program
group
Mean
[N = 19] Range
Children under the age of five years
Children under the age of 24 months
Children identified as M2
Children identified as M3
Total number of pregnant/lactating women and
children under five years present
q103
q104
q105
q107
numwochi
Total
Preventive
Recuperative
Total
Preventive
Recuperative
Total
Preventive
Recuperative
Total
Preventive
Recuperative
Total
Preventive
Recuperative
Total
26
50
47
49
41
28
35
4
15
9
0.4
2
1
75
74
74
0-72
9-97
21-95
9-97
9-91
15-62
9-91
1-7
4-51
1-51
0-1
0-6
0-6
9-169
27-151
9-169
Percentage
Percent of all children identified as M2 Preventive
Recuperative
Total
8
31
19
Percent of all children identified as M3 Preventive
Recuperative
Total
1
4
2
These results do suggest that children between 24 and 59 months old in the preventive program
and younger children, i.e., under 24 months old, in the recuperative program are less likely to
benefit from the services offered at RPs (vitamin A supplementation, deworming, immunization,
ORS, etc.), even though they are eligible to receive these services. Differences in the targeting
mode apply only for eligibility to receive the food, not the remaining of the MCH package of
interventions. The program supervisors should emphasize this point with health agents and
colvols so that they reinforce the importance of the RP services for all children under 5 years old
in their motivational efforts with the community.
3.3.1.2. Number of health staff at Rally Posts
WV’s implementation plan recommends that each health agent in charge of a RP be assisted by
at least two other health agents and two colvols when they hold a RP session. The data presented
in Table 3.2 show that only at about half of the RPs observed did the health agent responsible
receive any help from another health agent, and only three of them received help from two or
more additional health agents. Nonetheless, all but one health agent received support from
colvols, and most of them had two or even three colvols to assist them. Thus, although assistance
from colvols appeared to be in conformity with WV’s requirements, help from other health
agents was generally unavailable. It is not clear why some of the health agents managed to get
more help from other health agents than others; there is no relationship between the amount of
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
19
help from other health agents and the number of people attending a RP; and there is also no
relationship between the number of helping hands and the duration of employment of the health
agent in charge.
Table 3.2. Number of health staff present at RPs
Number of health staff
Variable
name
Numbers
[N = 19]
Number of health agents aside from the health agent responsible:
- None
- 1
- 2
- 3
q108
10
6
2
1
Number of colvols available to assist the health agent responsible:
- None
- 1
- 2
- 3
q109
1
1
11
6
Mean
[N=19] Range
Participant per health agent ratio
Participant per health staff ratio (health agents + colvols)
ratioasb
ratiostb
55
22
5-151
2-59
As could be expected, the ratio of RP participant per health agent was high: on average, 55,
whereas the ratio of all health staff (health agent + colvols) was more reasonable, at 22
beneficiaries per health staff person (see Table 3.2). Note that the variation in both ratios is quite
high, and at the upper end ranges as high as 151 beneficiaries per health agent. Also note that
the ratios presented in Table 3.2 do not include nonpregnant and nonlactating caregivers or
women in the age range 15-49 years old who are also eligible to receive services at the RPs, such
as immunization.
3.3.2. General organization and sequencing of activities at the Rally Posts
Location and bottlenecks. Of the 19 RP meetings observed, about half were held in a church
(n = 10), 4 in the courtyard of a private house, 3 in a school, and 2 in a dispensary. Venues like a
church or a school, which usually have only one room, make it difficult to physically separate the
different activities of the RP and to avoid interference. The data on the 19 RPs observed indicate
that in most cases, the different services provided were indeed located in one single area, next to
each other (see Table 3.3). In 4 of the RPs, a few services, such as registration, education,
weighing, or vaccination were held in a separate room of a dispensary (n = 2), or a separate area
in a church (n = 1), or the courtyard of a private house (n = 1).
Table 3.3. Organization of activities at RPs
Location and “bottlenecks” Variable name
Number of
RPs
[N=19]
Location of individual activities (number of RPs where the individual
activity was held in a separate area):
- Registration
- Education session
q110a
q110b
1
3
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
20
Location and “bottlenecks” Variable name
Number of
RPs
[N=19]
- Vitamin A distribution
- Deworming
- ORS distribution
- Weighing
- Vaccination
q110c
q110d
q110e
q110f
q110g
0
0
0
2
3
Most important bottleneck to participant movement:
- Registration 16
- Vaccination
- None
q111
1
2
Second important bottleneck to participant movement:
- Registration 1
- Vaccination
- Weighing
- None
q112
7
4
5
Data were also gathered on the services that appeared to create the largest “bottlenecks” to
participant movements. Table 3.3 shows that registration was either the most important or the
next most important bottleneck in the majority of the RPs (n = 17). Another activity that
hindered efficient participant movement through the services was immunization (in 8 of the
observed RPs). Weighing constituted the second most important bottleneck in 4 RPs. Only 2 of
the RPs had no bottlenecks to participant movement and both of these had fewer than 40
participants.
Sequencing of activities. The implementation plan specifies that each caregiver attending a RP
meeting should be guided through the different services in the following sequence (see Section
3.1): (1) education, (2) registration, including the distribution of vitamin A and deworming
dosage, if necessary, as well as ORS packets, (3) growth monitoring and promotion, and (4)
immunization.
Our results from the tracking of two respondents per RP (for a total of 38 tracked respondents in
19 RPs) through the various program activities are presented in Table 3.4. The data indicate that
none of the respondents followed the sequence of activities as proposed in the implementation
plan. Sequence 1 (see Table 3.4), which was followed by 8 of 38 tracked respondents, is the one
that comes closest to the recommended sequence, although the vitamin A and deworming tablet
dosages were not administered along with the registration, and immunization was not provided.
Table 3.4. Sequence of program services for tracked respondents at RPs
Sequence
Numbers
[N = 38]
Sequence 1:
Education
Registration
Child weighing (+post-weighing counseling, if used)
Vitamin A dosage, if used
8
Sequence 2:
Registration
Education
14
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
21
Child weighing
If used, sequence varies:
Post-weighing counseling
Immunization of child
Vitamin A/deworming tablet dosage
Sequence 3:
Registration
Child weighing (+post-weighing counseling, if used)
Education
Immunization of child, if used
Vitamin A dosage, if used
Immunization of respondent, if used
9
Other sequences 7
For about a third of the tracked respondents, the sequence was the following: registration,
followed by education, weighing or growth monitoring and promotion, immunization, and
vitamin A dosage (see Sequence 2 in Table 3.4). A third sequence, which 9 of the tracked
respondents followed, started again with registration, followed by weighing or growth
monitoring and promotion, education, immunization, and vitamin A dosage. Slightly modified
sequences were observed for 7 caregivers.
These data show that the sequence in which participants used the various program activities
varies widely across RPs. In 4 of the RPs, the sequence even varies between caregivers during
the same RP session. Education is rarely the first activity that caregivers attend at a RP. In
addition, the vitamin A and deworming tablet dosages are not administered along with the
registration as stipulated by the implementation plan. They are either given before/after the
weighing, or before/after the immunization.
3.3.3. Education sessions
3.3.3.1. Implementation of education sessions and attendance
Observations in the RPs generated data on the number of education sessions held at the different
RPs. Information on attendance at the education sessions was obtained from the tracking of
caregivers and the exit interviews. Our formative research conducted in 2002 (Menon et al.
2002b) had revealed that a number of participants could not benefit from the education sessions
conducted at the RP because they arrived late, and the only education session provided was
usually carried out at the beginning of the meeting. Based on the discussions of these results, it
was decided that multiple education sessions should be held at the RPs to facilitate attendance by
those participants who arrive late.
Table 3.5 presents the data on the number of education sessions held at RPs, and shows that only
one education session was conducted at the majority of the RPs observed. Only four of the RPs
conducted two (n = 3) or three (n = 1) education sessions on the day of observation. The number
of adults participating in the education sessions varied from 6 to 76 in the RPs where one session
was conducted, compared to 9-29 where two sessions were held, and 4-51 in the RP where three
sessions were held.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
22
Table 3.5. Number of education sessions at RPs and number of participants per session
Education session Variable name
Numbers
[N = 19]
Number of education sessions held:
- One
- Two
- Three
q114
15
3
1
Mean
[N = 19] Range
Number of adults per session in RP where:
- 1 session was held (n = 15)
- 2 sessions were held (n = 3)
- 3 sessions were held (n = 1)
numbpartif1sess
numbpartif2sess
numbpartif3sess
30
19
20
6-76
9-29
4-51
The data presented in Table 3.6 on attendance at education sessions show that all tracked
respondents participated in an education session, whereas only 80 percent of the exit interview
respondents did so. The remaining 20 percent (n = 12) of respondents from the exit interview
sample reported that they had arrived too late at the RP to attend an education session. The
reason for this difference in attendance between the tracked respondents and the exit interview
respondents may be that the tracked respondents were usually selected right at the start of the
RP, while the exit interview respondents were selected as they left the RP. Thus, some of the
exit interview respondents could have arrived too late to attend the education session, while all
the tracked respondents had arrived at the stipulated start time and, therefore, had attended the
education session. Further, as expected, non-attendance of the education session by exit
interview respondents was more common in RPs where only one education session was
conducted because this session was also more likely to be conducted shortly after the start of the
RP. The results from the home interviews (n = 30) also confirmed that education sessions were
frequently missed when mothers arrived late at the RP. About two-thirds of the 27 home
interview respondents who had attended a RP recently reported that they had missed the
education session at the RP because they had arrived too late and no other education session was
held while they attended the RP.
Table 3.6. Attendance of education sessions at RPs and reason for non-attendance
Education session
Variable
name
Tracked
respondentsa
(n = 38)
Exit interview
respondentsb
(n = 59)
Number of caregivers who attended
education
q310+q311
q214+q215
38 47
If not, reason for non-attendance
- Arrived too late at the RP
(n=12)
12 a
 Tracked respondents were the sample of 38 beneficiaries (2/RP) observed by our fieldworkers as they went through the different
RP activities.
b The sample of exit interview respondents consisted of two-to-four randomly selected caregivers accompanying a food aid
beneficiary child in each RP [n = 59].
The majority of the education sessions observed was conducted by the health agent in charge of
the RP (see Table 3.7). As noted above, the number of participants ranged from 6 to 76. On
average, the health staff talked for about 10 minutes, but the duration ranged from as low as 1 minute
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
23
to 23 minutes. Very little use of any communication material was observed, even though a variety of
tools do exist, and WV had provided a set of these materials to each health agent and colvol.
Table 3.7. Observations of education sessions held at the RPs
Characteristics Variable name
Numbers
[N = 38] Range
Who facilitates the education session?
- Health agent in charge
- Colvol
- Other health agent
q330
28
8
2
Topics of the sessions by month:
July (n=12)
- Immunization
- Importance of health card
- Importance of weighing
August (n=20)
- Immunization
- Importance of weighing
- Treatment of drinking water
- Family Planning
September (n=6)
- Immunization
- Importance of health card
- Treatment of drinking water
q331a
8
2
2
12
2
4
2
2
2
2
Number of participants at the session q332 6 to 76
Type of visual materials used:
- World Vision album of pictures
- Images on infant and young child feeding from
IFPRI-CU-WV team
- Images from CARE Guide
- None
q333
3
1
2
32
Other reference material is used q334 6
Mean
[N = 38] Range
Average length of education sessions (minutes) lengeduc 10 1-23
The topics presented varied between the different RPs, even within the same month. Up to four
different topics were addressed in the RPs observed within a one-month period (see month of
August, Table 3.7). The most popular topics were immunization, the importance of the health
card, and of weighing the child. The results from the home interviews confirmed that
immunization was by far the predominant topic presented at the RP education sessions in the
months prior to the home interviews. None of these topics corresponded to the topics suggested
for discussion during the study months by the program. For instance, the topics for July to
September were: family planning, HIV/AIDS and Moringa olifeira. It is not clear why the
scheduled topics were not discussed, i.e., whether it was because the health agents were not
aware of these monthly topics, or whether they did not feel comfortable enough with the topics.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
24
3.3.3.2. Quality of education at the Rally Posts
Table 3.8 presents an assessment of the quality of the education provided at the RPs, from
fieldworkers’ observations of the activity. A good ambiance was reported by the fieldworkers
for 15 of the 38 sessions observed, i.e., participants were attentive to the health agent’s
presentation, there were few distractions, and the health agent/colvol spoke clearly and loud
enough so that she/he could be understood by the participants. In 9 of the 38 observed sessions,
however, the noise levels were high, which is understandable, given the level of crowding at
many of these sessions and the fact that there was no separate space available to carry out the
education.
Table 3.8. Quality of education provided at RPs
Numbers
[N = 38]
Positive aspects
 Good ambiance 15
 Practical advice 6
 Use of demonstrations 2
 Dialogue
 Animator asks questions to audience
- No questions
- Many questions
2
10
18
Negative aspects
 No animation (songs, sketch) 22
 Distracting atmosphere (side-conversations, crying children, etc.) and crowded 9
 Audience asks questions to animator
- No questions
- A few questions
34
4
A few health agents and colvols also included practical advice and demonstrations in the
sessions, or attempted to create a dialogue with the participants. In about one-half of the
observed sessions, the animators asked many questions to the audience. However, in most of the
sessions, the audience asked no questions to the animator and therefore, a real dialogue or
interaction was not established. In more than one-half of the sessions, the animator did not use
songs or sketches to animate the group and to interest the participants in the topic. Finally, the
education sessions tended to focus on messages rather than use concepts of behavior change, and
lacked practical advice and/or the discussion of potential constraints to behavior change and
solutions to address them.
The potential effectiveness of the education conducted at the RPs was assessed based on the
recall and application of the information presented at the education sessions by the home
interview respondents. The respondents were asked to recall as many of the details concerning
what they had heard and learnt and to talk about what they were able to use. The results are
presented in Table 3.9.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
25
Table 3.9. Spontaneous recall and application of advice received at RP education sessions
(home interview sample of mothers exposed to specific education topics)
Spontaneous recall of details Numbers Application/adoption Numbers
Immunization protects against illness
(N = 9)
8
Importance of giving all doses to fully
protect the child (N = 9)
3
Role of certain vaccines (N = 9) 3
Taking the child regularly to RP
for immunization (N = 9)
7
Dosage for treatment of drinking water
and role of chlorine (N = 1)
1 Treatment of drinking water, daily
(N = 1)
1
Preparation of ORS when child has
diarrhea (N = 1)
2
In general, the information given in the education sessions on immunization was considered
useful by the respondents, although it was not new to some of them. The recall of details and the
adoption of recommended practices were quite good, given the lack of depth of presentation of
topics at the RPs. For instance, most of the mothers who remembered the topic of immunization
had actively taken their child regularly to the RP for immunization. Further, most of the women
who had attended an education session on immunization also recalled that immunization protects
against illness, and a few mentioned the importance of fully immunizing the child in order to
achieve total protection. Others remembered the role of specific vaccines, although not always
correctly, and some either recalled names of vaccines or that there was an immunization
schedule to be followed, but without remembering more of the details.
The one woman who attended a session on water treatment provided detailed information on
dosage for chlorine and its role. She also reported treating the drinking water on a daily basis.
No details were recalled by respondents who had attended RP education sessions on the
importance of weighing and ORS. When asked about the application of advice, though, the one
woman who had heard about the importance of weighing said that she takes the child regularly to
the RP for weighing. Similarly, two women mentioned that they prepared ORS at home when
their child has diarrhea, even though one of these women had actually not attended a session on
ORS at the last RP.
3.3.4. Registration, vitamin A, deworming, and ORS
3.3.4.1 Implementation and use of registration, vitamin A, deworming, and ORS services
According to the implementation plan (see Section 3.1), registration is the second activity a
caregiver would attend at the RP. The same health agent who does the registration is expected to
distribute vitamin A capsules, deworming tablets, and ORS sachets. Therefore, the results for
these services are discussed in the same section, although the data presented in Section 3.3.2
show that vitamin A, deworming, and ORS are often administered along with the weighing or
the immunization activities.
The data presented in Table 3.10 on the use of services show that all tracked respondents were
registered. Information about registration was not obtained in the exit interviews. About one￾third of the children of tracked respondents received a vitamin A capsule on the day of
observation/interview. Among those who did not receive vitamin A, the reasons for not
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
26
receiving the vitamin A included: (1) the child was not due to receive vitamin A on the day of
observation (n = 14; 8 of these 14 children attended a RP where this service was not offered on
the day of observation), (2) vitamin A capsules were not available on the day of their visit to the
RP (n = 2), and (3) the caregiver had not used the service for unknown reasons, although the
child was due to receive vitamin A the day of observation (n = 7). Thus, about 38 percent of the
tracked children who were due to receive a vitamin A capsule on the day of observation did not
receive it (9/24 tracked children).
Table 3.10. Use of registration, vitamin A, and deworming tablet dosage and ORS packets
at RPs and reason for non-usage
Registration, vitamin A, deworming, ORS
Variable
name
Tracked
respondentsa
(n = 38)
Exit interview
respondentsb
(n = 59)
Number of caregivers who were registered q310+q311 38 -
Number of caregivers with child who received vitamin A
(among those due)
If not, reason for non-usage:
- Vitamin A not due
- Service not used (reason not known), but due
- Service not offered day of observation, but due
q310+q311 15 (of 24 due)
(n = 23)
14
7
2
-
Number of caregivers with child who received
deworming tablet (among those eligible > 2 years)
If not, reason for non-usage:
- Not eligible for deworming
- Service not used (reason not known), but eligible
- Service not offered day of observation, but eligible
q310+q311
q214+q215
2 (of 13 eligible)
(n = 36)
25
4
7
3 (of 20 eligible)
(n = 56)
39
8
9
Number of caregivers who received ORS packets (all
children are eligible)
If not, reason for non-usage:
- Service not used (reason not known)
- Service not offered day of observation
q310+q311
q214+q215
0 (of 38 eligible)
(n = 38)
14
24
2 (of 59 eligible)
(n = 57)
20
37 a
 Tracked respondents were the sample of 38 beneficiaries (2/RP) observed by our fieldworkers as they went through the different
RP activities.
b
 The exit interview respondent samples consisted of 2-4 randomly selected caregivers accompanying a food aid beneficiary child
in each RP [n = 59].
Similarly, very few of the children eligible to receive deworming tablets (i.e., > 24 months) on
the day of observation/interview received them. For approximately half of the eligible children,
this was due to the fact that the service was not offered on the day of observation/interview (this
was the case for 7 of the 13 eligible tracked children; and 9 of the 20 eligible children from exit
interviews). For the others, the caregivers had not used the service for unknown reasons.
Among children who did not receive the service, approximately 70 percent were not eligible (9
of the 25 non-eligible tracked children attended a RP where the service was not offered on the
day of observation; this was the case for 15 of the 39 non-eligible children from exit interviews).
None of the tracked respondents and only 2 of the exit interview respondents received ORS
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
27
packets, either because they were not offered on the day of observation (approximately two￾thirds of the children) or the respondent had not used this service for unknown reasons.
The supply of vitamin A capsules, deworming tablets, and ORS packets was inadequate in many
of the RPs observed; vitamin A was unavailable in 5 of the 19 RPs, whereas deworming tablets
and ORS were unavailable in 8 RPs and12 RPs, respectively.
In the process of registration, the health agent is also expected to update the health cards with
information on the vaccines, vitamin A supplementation, and deworming that the child is
supposed to receive if available on the day of the RP, and signs attendance on the ration card for
food aid beneficiaries. The data presented in Table 3.11 summarize the information on the status
of beneficiary cards of the child beneficiaries accompanied by the tracked respondents. The
results show that most respondents, with the exception of three, had the child beneficiary’s health
card in hand when they came to the RP. For all of those who had their health card, the
information regarding the immunizations and vitamin A supplementation they received on the
day of observation was duly recorded in their cards. For one of the two children who received
deworming tablets, however, the information was not entered in the card.
The data in Table 3.11 also show that only 27 of 38 tracked respondents had brought the ration
card to the RP. Ten respondents had forgotten to bring their ration cards and one respondent
reported that she did not yet have a ration card for her child, who was a beneficiary of the food
aid program. The date of attendance, which is mandatory in order for the beneficiary to receive
the food rations, had been entered by the health agent on the cards of about one-half of the
respondents who had the cards in their possession. The other half were probably not marked for
attendance because the system of noting beneficiary attendance at the RP on the ration card was
only recently established as a way of verifying the eligibility of the beneficiaries at the
distribution site. The other system in place to ensure that only eligible beneficiaries receive their
food ration is a list (prepared by the health staff) of all beneficiaries who attended the mandatory
RP and MCH activities. Thus, in spite of the fact that many beneficiaries did not get the date of
attendance entered on their ration card, they should be able to receive their food ration if their
name appears on the list provided by the health staff to the commodity section.
According to the birth date information on their health cards, all beneficiary children in the
preventive program group were of the required age for that category (i.e., between 6 and 23
months old). Similarly, none of the beneficiary children in the recuperative program group was
older than the upper age limit of that category (i.e., 5 years old). Two of the 13 children
classified in Table 3.11 under the category “malnourished child < 5 years” do not belong to the
recuperative program group, i.e., these were two children in the preventive group for whom
“malnourished child under 5 years” was marked in their ration cards. One of the two
misclassified children was 15 months old and therefore in the targeted age range for the
preventive group. It is likely that the misclassification in this case is a mistake made when the
information was entered in the card. The other misclassified child was malnourished (M2) and
nearly 25 months old, thus not eligible for food rations in the preventive program group. One
possible explanation for this child being a beneficiary (in spite of being older than the cutoff age
of 24 months in the preventive program) could be that this child was found to be severely
malnourished (M3) at a previous visit and was therefore eligible to receive food aid, and had
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
28
now improved and became M2, as opposed to M3.7
 We do not have any information on the
child’s previous nutritional status, however, to verify this assumption.
Table 3.11. Status of beneficiary health and ration cards
Characteristics Variable name
Numbers
[N = 38]
Number of respondents who had the child beneficiary’s
health card with them on the day of observation
Among those with health card (n = 35):
Vaccination:
- Number who received vaccination
- Number whose vaccination was entered in card
Vitamin A
- Number who received vitamin A
- Number whose vitamin A was entered in card
Deworming
- Number who received deworming
- Number whose deworming was entered in card
q338
q343
q341
q344
35
7
7
14
14
2
1
Number of respondents who own a ration card for the
child
Among those who own ration card (n = 37):
- Number who brought card at RP
- Number where date of attendance was entered
(among those with card = 27)
- Number of children in beneficiary categories:
- 6-23 months
- Malnourished < 5 years
q346
q346
q348
q347
37
27
15
14
13
Mean Range
Average age of children in each category, according to
health card (n = 27):
- 6-23 months old (n = 14)
- Malnourished child < 5 years (n = 13)
agechild&
q347
16
23
7-23
8-46
3.3.4.2. Quality of vitamin A supplementation and deworming services
One way to assess the quality of services in the case of vitamin A supplementation and the
distribution of deworming drugs is to determine whether the staff adheres to the recommended
schedule of administration/distribution of these services, which in both cases is every six months.
Data were available for 34 of the 38 tracked children concerning the vitamin A dosage and for 6 of
13 children older than 24 months and therefore eligible to receive deworming tablets (shown in Table
3.12). The fieldworkers obtained the information from the health card and the health agents’ register.
It is not clear why information could not be obtained for the other eligible children, whether these
had never received vitamin A or deworming, or whether it was due to incomplete reporting.
 
7
 Severely malnourished children between 24 and 59 months of age in the preventive program do also receive food rations for 9
months.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
29
The results show that five children had not received a vitamin A capsule the day of observation,
although their last dosage was more than 6 months ago. On average, the last dosage was 8
months ago, with a variation from 7 to 15 months. For two of the five children who had not
received their dosage on time, the service was not offered at the RP they attended. For the three
other children, the respondent did not use the service that day for unknown reasons, although
vitamin A capsules were available at the RP. It is not clear whether the health agent did not
distribute capsules to these children or whether the respondents chose not to take them. All
children for whom information on deworming tablets was available had received their dosage on
time, i.e., none of these children had received their last deworming tablets more than 6 months
ago.
Table 3.12. Timing of vitamin A and deworming tablet dosage (tracked respondents)
Characteristics Variable name
Tracked
respondents
[N=38]
Number of children who received last vitamin A dosage
(n = 34)
- ≤ 6 months ago
- > 6 months ago
Number of children who did not receive vitamin A dosage the
day of observation (n = 17) and whose last dose was > 6
months ago
q342
vitam
vitam&q341
29
5
5
Number of children who received last deworming dosage
(n = 6)
- ≤ 6 months ago
- > 6 months ago
Number of children who did not receive deworming dosage
the day of observation (n = 4) and whose last dose was > 6
months ago
q345
deworm
deworm&q344
6
0
0
3.3.5. Growth monitoring and promotion
3.3.5.1. Implementation of growth monitoring and promotion activities
As part of the activities carried out at the RP, each child should be weighed, the weight recorded
on the growth chart (weight-for-age) printed in the health card, and the child’s weight interpreted
to evaluate nutritional status (i.e., whether or not the child has an adequate weight for his age,
and if s/he is moderately or severely malnourished, i.e., M2 or M3, according to the Gomez
classification). It is expected that caregivers will be informed about the child’s weight and
receive brief counseling, especially if their child is malnourished (M2, M3).
The data from observations of tracked respondents and exit interviews (Table 3.13) indicate that
all children were indeed weighed. Data concerning the subsequent steps, i.e., informing the
caregiver of the child’s weight, recording the weight, assessing the child’s nutritional status, and
informing the caregiver, were only available for the tracked respondents. They show that
although the health staff tended to comply with the “mechanical” steps involved in growth
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
30
monitoring and promotion, namely, the weighing, recording, and nutritional status assessment,
they were much less likely to have communicated the results to the caregivers or to have engaged
in any type of discussion or counseling. Only half of the tracked respondents were informed of
the child’s weight and only 25 percent were told about the child’s nutritional status. Similarly,
only about one-third of all respondents (tracked or interviewed) received any post-weighing
counseling.
Table 3.13 also provides information on the likelihood of being counseled according to the
child's nutritional status. For both tracked respondents and exit interview respondents, having a
malnourished child (M2 or M3) seemed to increase the likelihood of receiving counseling (e.g.,
50 percent of the exit interview respondents who had a malnourished child were counseled,
compared to 30 percent among respondents of a nonmalnourished child). Thus, it seems like the
health staff did pay some attention to the nutritional status of the child in deciding whether or not
to communicate with the mother about the child’s status. However, they still did not provide
counseling to 50 percent or more of the caregivers who had a malnourished child. Similarly, the
vast majority of mothers who were accompanying a well-nourished child were not praised or
encouraged for having a well-nourished child.
These results are also confirmed by the information from the home interviews. Of the 21 home
interview respondents whose children were recently weighed at a RP, only 8 received counseling
after the weighing. One respondent explained why individual advice at RPs is difficult to get:
“There were many children; there was no time to counsel us” (Beneficiary mother).
Table 3.13. Use of weighing and other steps in growth monitoring and promotion activities
and reasons for non-usage
Weighing and other growth monitoring and
promotion activities
Variable
name
Tracked
respondentsa
(n=38)
Exit
interview
respondentsb
(n=59)
Child was weighed q310+q311
q214+q215
38 59
Caregiver was informed of child’s weight q317 16 -
Child weight was recorded in health card (n = 35) q339 29 -
Nutritional status was assessed q319 32 -
Caregiver was informed of child’s nutritional status q321 9-
Post-weighing advice was given to caregiver
If not, reason for non-usage
- Service not provided/not used
q310+q311
q214+q215
14
(n = 24)
24
22
(n = 37)
37
If child malnourished
- M2: Received counseling
- M3: Received counseling
q357&
q3103117
q209&
(n = 20)
9/20
0/0
(n = 22)
10/21
1/1
If child not malnourished q2146 (n = 18) (n = 33)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
31
- M1: Received counseling
- Normal: Received counseling
3/9
2/9
6/22
4/11 a
 Tracked respondents were the sample of 38 beneficiaries (2/RP) observed by our fieldworkers as they went through the different
RP activities.
b
 The sample of exit interview respondents consisted of two-to-four randomly selected caregivers accompanying a food aid
beneficiary child in each RP [n = 59].
Observations in the RPs revealed that most of the weighing was done by a colvol, whereas the
person who noted the weight down was more likely to have been the health agent in charge (see
Table 3.14). As noted above, only approximately half of the tracked respondents were informed
about their child’s weight and this was equally likely to have been done by the health agent in
charge or the colvol. However, the children’s nutritional status, which was assessed for
approximately 85 percent of the children, was much more likely to have been determined by the
health agent in charge (20/32) than by a colvol (11/32) or another health agent (1/32).
The results of the home interviews support the observations at the RP. Less than half of the
home interview respondents whose children were recently weighed at the RP were informed
about their child’s weight (7/21), and only two were informed of his/her nutritional status. One
respondent reported having managed to read the weight on the scale herself and three
respondents had not received any information about either the weight or the nutritional status of
the child. All respondents who received counseling (8/21), however, were told their child’s
weight and/or nutritional status.
Table 3.14. Observations of weighing sessions at RPs
Characteristics
Variable
name
Numbers
[N = 38]
Child was weighed q312 38
Person who conducted the weighing:
- Health agent in charge
- Colvol
- Other health agent
q313
7
28
3
Person who noted weight down:
- Health agent in charge
- Colvol
- Other health agent
q316
25
7
6
Person who informed the caregiver of the child weight (n = 16):
- Health agent in charge
- Colvol
- Other health agent
q318
8
6
2
Person who evaluated the child’s nutritional status (n = 32):
- Health agent in charge
- Colvol
- Other health agent
20
11
1
Person who informed the caregiver about the child’s nutritional status (n = 9):
- Health agent in charge
- Colvol
q322
8
1
Note that the evaluation of nutritional status is intended to be carried out at the RP itself, using
the growth chart printed in the health card. In three of the RPs observed, the health agents had
not done it immediately on site, but did it at home. This was apparently due to the fact that two
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
32
different health cards were used at the time of our study: the “old” health card with a growth
chart based on the percentage of the median of the weight-for-age CDC/NCHS/WHO reference
standards (WHO 1979) using the Gomez classification8
 and the “new” health card, where the
growth chart is based on weight-for-age Z-scores9
 that are derived by comparing the child’s
measurements to the same reference standards, but using the WHO classification.10 Newly
identified beneficiaries had started to receive the new health card, whereas WV was continuing
to base its reporting of children’s nutritional status on the Gomez classification.
In this transition period, the health agents had been advised to plot the weight in the growth chart
of the new card and to note down the weight in their registers, but to evaluate the nutritional
status to be included in their daily reports only at home, using the old health card. According to
the fieldworkers’ observations, this procedure was respected in three RPs. In most of the other
RPs, the evaluation with the old health card was done immediately after the weighing or before
the end of the RP. In some RPs, the recommended procedure was somewhat modified and the
nutritional status was evaluated using the new card but translated into the different degrees of the
Gomez classification for the report.
3.3.5.2. Quality of the growth monitoring and promotion activities
Weighing and plotting
Results of our assessment of the quality of the weighing procedures at the RPs are presented in
Table 3.15. Hanging (SALTER) scales are used to weigh the children. The difficulties in
reading the weight of a moving (and/or crying) child on these scales are well recognized.
Therefore, one of the observations made by the fieldworkers was the movement of the scale’s
needle at the time of measurement. In most of the observed cases, the needle was stable or only
somewhat moving (80 percent), but in 20 percent (n = 8) of the observations, the needle was
moving a lot. The children were re-weighed using an electronic scale by the fieldworkers (who
had been trained and standardized in taking anthropometric measurements) to assess the extent
of measurement error. On average, the weight measured with the hanging scale was 120 g
higher compared to the weight measured with the electronic scale, but there were large variations
in the magnitude of the measurement errors (from –2,600 to + 1,000 g).
If we consider that a difference of + or – 100 g between the two measurements is acceptable,
nearly half of the observed measurements done by the health staff are within this range (n = 18).
However, the percentage of acceptable measurements is lower when the needle of the scale
swings a lot (25 percent), compared to when the needle is stable (where 47 percent are within +/-
100 g). This finding suggests that the extent of measurement errors could be considerably
 
8
 Normal (N) corresponds to ≥ 90 percent of the median of the weight-for-age CDC/NCHS/WHO standards; mild malnutrition
(Grade M1) to 75 percent - < 90 percent; moderate malnutrition (Grade M2) to 60 percent - < 75 percent, and severe malnutrition
(Grade M3) to ≤ 60 percent (Cogill 2003).
9
 A Z-score (or standard deviation score) is defined as the deviation of the value of an individual child from the median value of
the reference population, divided by the standard deviation for the reference population (WHO 1995).
10 Normal corresponds to a Z-score of ≥-1 standard deviations; mild malnutrition corresponds to a Z-score of ≥-2 and <-1
standard deviations; moderate malnutrition (low weight-for-age) to a Z-score of ≥-3 and <-2 standard deviations, and severe
malnutrition (very low weight-for-age) to <-3 standard deviations (Cogill 2003).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
33
lowered if the health staff paid more attention to the movement of the scale’s needle at the time
of measurement. It has to be noted also that colvols were much more likely to achieve exact
measurements than health agents.
Besides the precision of the weighing, data were collected on whether the plotting was done
accurately—using the weight as measured by the health staff - and on the extent of
misclassification that resulted from incorrect plotting. Results indicate that the plotting was done
accurately in only half of the cases where the weight of the child was plotted in his/her health
card (n = 14/29). These findings were similar whether the health agent or the colvol had done
the plotting. The implications of these plotting errors for the classification of children into the
different Gomez categories were also assessed by our fieldworkers, again using the weights
obtained by the health staff. The results presented in Table 3.15 show that the plotting errors
actually resulted in relatively small misclassification errors—no differences in classification
were found in 87 percent of the cases (n = 28/32). This was probably due to the fact that the
plotting errors were not of large magnitude, and therefore, although they were found in
approximately half of the cases, they resulted in only about 13 percent (n = 4/32) of the children
being misclassified into the different Gomez malnutrition categories. Note that none of the
colvols who evaluated the nutritional status misclassified children into malnourished versus well￾nourished.
Table 3.15. Quality of weighing sessions at RPs (observations and re-weighing by our staff)
Characteristics Variable name
Mean
[N = 38] Range
Difference between weight measured by fieldworkers (electronic scale) and
by health staff (Salter scale) (g)
wt_diff
(q315&q356)
-120 -2600 - +1000
Variable name Number [N = 38]
Scale’s needle movement at the time of measurement:
- A lot
- Somewhat
- Not at all (needle is stable)
q314
8
15
15
Number of correct measurements when electronic scale measure is used as
reference (+/- 100g)
wt_diff 18
Number of correct measurements (+/-100g) when weight read when:
- Needle is stable (n = 15)
- Needle swings somewhat (n = 15)
- Needle swings a lot (n = 8)
wt_diff& q314
7
9
2
Number of correct measurements (+/-100g) when person weighing is:
- Health agent (n = 10)
- Colvol (n = 28)
wt_diff&
q313 3
15
Weight was correctly plotted into growth chart (n = 29) q340 14
Number of weights correctly plotted when person who noted down the
weight was:
- Health agent (n = 22)
- Colvol (n = 7)
q340&
q316
10
4
Difference in classification of nutritional status (Gomez) evaluated by
fieldworkers versus evaluated by health staff based on weight as
measured by the health staff (n = 32):
- No difference in classification
o Health agent (n = 21)
o Colvol (n = 11)
Misclassification of children’s nutritional status:
- “Normal” according to fieldworker but classified by health staff as
q320&
q357
28
17
11
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
34
“M1”
- “M1” according to fieldworker but classified by health staff as
“Normal”
- “M2” according to fieldworker but classified by health staff as “M1”
1
2
1
Difference in classification of nutritional status (Gomez) evaluated by
fieldworker versus evaluated by health staff based on weight as
measured by the fieldworkers (n = 32):
- No difference in classification
o Health agent (n = 21)
o Colvol (n = 11)
Misclassification of children’s nutritional status:
- “Normal” according to IFPRI team but classified by health staff as
“M1”
- “M1” according to IFPRI team but classified by health staff as
“Normal”
- “M2” according to IFPRI team but classified by health staff as “M1”
- “M1” according to IFPRI team but classified by health staff as “M2”
q320&
nutrstatverified
23
13
10
2
3
2
2
The implications of the combination of weighing and plotting errors for the classification of
children into the different Gomez categories were also assessed by comparing the classifications
obtained by the health staff with those obtained by our research staff (using their measurement of
the child’s weight and their plotting of the child’s weight). As expected, the results presented in
Table 3.15 show that the misclassification errors are slightly higher—no differences in
classification were found in 72 percent of the cases (n = 23/32), but about 28 percent (n = 9/32)
of the children were misclassified into the different Gomez malnutrition categories. This was
due to the fact that in addition to plotting errors, weighing errors occurred in about half of the
observed measurements done by the health staff. Again, misclassification errors are less
frequent when colvols evaluated the nutritional status as opposed to health agents.
Note that these results also highlight the implications of the misclassification for the program in
terms of leakage of food to non-eligible children or not reaching targeted beneficiaries (see Table
3.15). Misclassification errors in this case resulted in a similar number of eligible children being
excluded from the program (n = 2) and of non-eligible children mistakenly being included in the
program (n = 2). Although in terms of food aid amounts the numbers cancel out, the leakage of
the intervention to children who may not need it and missing children who do need it results in
lower cost-effectiveness of the program. It is therefore important that every effort be made to try
to minimize the measurement errors encountered through weighing of the child and plotting of
the child’s weight on the growth chart.
Post-weighing counseling
The quality of the post-weighing counseling received by tracked respondents (in cases where it
was offered) was also assessed during our observations in the RPs (see Table 3.16). The post￾weighing counseling was done exclusively by health agents, and mostly by the health agent in
charge. Therefore, it seems that the likelihood of receiving counseling was greater if the health
agent in charge was the person evaluating the nutritional status of the child. For the counseling,
only two respondents were taken aside to guarantee a private atmosphere, while the others
received the advice publicly in front of other participants. Of the 12 respondents who had a
malnourished child, one was reprimanded, whereas one of the two respondents whose child had
normal weight-for-age was complimented. For 65 percent (n = 9) of the respondents counseled,
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
35
the health agent checked through a few questions whether the respondent had understood the
advice given. The counseling was very brief and lasted, on average, about 3 minutes, varying
between 1 and 6 minutes.
Table 3.16. Quality of post-weighing counseling in RPs (among tracked respondents who
received counseling (n=14))
Characteristics Variable name
Numbers
(n=14)
Person who provided advise/counseling
- Health agent in charge
- Other health agent
q325
13
1
Counseling/advice was given in a private place q326 2
Caregiver of malnourished child (n = 12) was reprimanded while
advice was being given
q327
q327&q357
1
Caregiver of well-nourished child (n = 2) was complimented while
advice was being given
q328
q328&q357
1
Person giving the advice checked to ensure that the respondent
understood the advice
q329 9
Time (minutes) used in post-weighing counseling
- Mean
- Range
act1&minutes
2.8
1-6
These results suggest that counseling was offered to a relatively small proportion of participants
and generally involved little interaction time between the health agent and the respondent.
Moreover, Table 3.17 documents that the quality of interaction and of nutritional advice
provided through the individual counseling was generally poor. Overall, the advice is
characterized by a lack of detail and precision, and is generally of little practical relevance; the
health staff does not provide examples of types of foods children should be fed, nor information
on appropriate feeding frequency and portion sizes for specific children of different ages. Only
one caregiver was giving advice on specific foods to give (fruits and fruit juice), but she was not
advised on how often and in what quantity these should be given to the child.
For 3 of the 14 tracked respondents counseled, the nutritional status of the child had not been
evaluated by the health staff, so the advice given to these caregivers was not expected to be
related to the child’s nutritional status. It is unclear how the health staff decided which advice to
give. The findings for the remaining 11 tracked respondents counseled show that, with two
exceptions, the type of advice provided was generally related to the child’s nutritional status.
For two of them, however, the advice was not related to the child’s nutritional status: (1) the
mother of a well-nourished child (Normal) was told to take better care of the child; and (2) the
mother of a malnourished child (M2) was advised to continue to feed the same way, which
seemed inappropriate, given that the child was malnourished.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
36
Table 3.17. Type of individual advice given at RPs, by nutritional status
Nutritional status, as evaluated
by health staff (q320), using
Gomez classification
N
[N = 14]
Type of advice given (q324), multiple advice
possible
Nutritional status not evaluated 3 Feed the child more food (n = 1)
Encourage child to eat (n = 1)
Feed the child more food and give fruits, fruit juice (n = 1)
Take better care of your child (n = 1)
Normal 2 Continue to feed same way/compliment (n = 1)
Take better care of your childa
 (n = 1)
M1 2 Feed frequently (n = 1)
Feed the child more food (n = 2)
M2 7 Feed the child more food (n = 5)
Take the child to the hospital (n = 3)
Encourage child to eat (n = 1)
Take better care of your child (n = 1)
Feed WV ration to the child (n = 1)
Continue to feed the same way (n = 1) a
 Sentences in italic refer to a type of advice that was inappropriate given the nutritional status of the child.
Data from home interviews (n = 30) also provided information on respondent participation in the
RPs, and the quality of weighing and counseling activities. Of the 21 respondents whose
children were weighed at the RP they attended recently, only 8 reported having received
counseling after the weighing. The type of individual advice given to these respondents is
similar to the examples of advice given to the tracked respondents (see above, Table 3.17).
Again, the advice was general and only one respondent was told which specific foods to feed the
child (bouillon and enriched milk), but without providing details on how often and in what
quantity these foods should be fed. No information on the nutritional status of the children was
collected in these interviews, and therefore it is not possible to determine whether the type of
advice given was related to the child’s nutritional status.
All 8 home interview respondents who received counseling reported that they considered the
advice given useful and applied what the health staff had told them, keeping their comments as
vague as the advice was. The two respondents who were told to take the child to the hospital had
done so and their children were treated. One respondent who was advised to continue to feed
enriched meals said she could not afford the ingredients for enriched meals on a regular basis.
No other details in relation to using the advice (e.g., which foods, how often, amounts fed) were
provided by the mothers.
3.3.6. Immunization
Immunization services at the RP are offered to all children under 5 years old and to women
15-49 years of age. The results in relation to the use of immunization by children and caregivers
are presented in Table 3.18. About 25 percent of both tracked respondents and caregivers
interviewed in the exit interviews were accompanying a child who had received some
immunization on the day of observation. The majority of children who were not immunized on
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
37
the day of observation were already fully immunized.11 In 20 percent of the cases, however,
children were not immunized because the service was not available at the RP that day, although
we do not know how many of these children were actually due to receive some immunization.
Only two children were not immunized (although due) because the caregiver had not been using
the service (for unknown reasons).
According to the fieldworkers’ observations and the exit interviews, only a few respondents
received an anti-tetanus vaccine at the RP (see Table 3.18). Most of the respondents were
already immunized, i.e., had received two doses of anti-tetanus vaccine, or were attending a RP
where this vaccine was not available on the day of observation. A few respondents had not used
the service for unknown reasons, although they were not yet immunized. Four respondents were
older than 49 years (grandmother) or male (father) and thus not eligible for the anti-tetanus
vaccine.
Immunization for children was not offered in four of the RPs. In one of them, it was not offered
because an immunization campaign organized by a hospital nearby was ongoing. In two other
RPs, the health staff could not obtain the vaccines from the WV office. In the fourth one, too
few participants attended the RP and the health agent preferred not to open a vial containing 20
doses only for the 4 children who were due to receive their immunization that day. The anti￾tetanus vaccine for women was in even shorter supply and was not available in six of the RPs.
Table 3.18. Use of immunization at RPs and reason for non-usage
Immunization of child and caregiver
Variable
name
Tracked
respondentsa
(n = 38)
Exit interview
respondentsb
(n = 59)
Number of caregivers with child who was immunized
If not, reason for non-usage:
- Child fully immunized
- Service not used (reason unknown)
- Service not offered day of observation
q310+q311
q214+q215
8
(n = 30)
23
1
6
16
(n = 43)
32
1
10
Number of caregivers who were immunized
If not, reasons for non-usage:
- Service not received because post run out of vaccines
- Non applicable because respondent >49 y or male
- Respondent already immunized
- Service not used (reason unknown)
- Service not offered day of observation
1
(n = 37)
1
1
18
5
12
4
(n = 55)
1
3
25
8
18 a
 Tracked respondents were the sample of 38 beneficiaries (2/RP) observed by our fieldworkers as they went through the different
RP activities.
b
 The sample exit interview respondents consisted of 2-4 randomly selected caregivers accompanying a food aid beneficiary child
in each RP [n = 59].
 
11 A child is considered fully immunized in Haiti if s/he has received one dose of BCG, three doses of polio, three doses of DTP,
and one dose of measles vaccination.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
38
In summary, it appears that a significant percentage of children attending the RPs was fully
immunized and that half of the caregivers was already immunized. However, regular availability
of all vaccines was definitely a problem at the time of the evaluation.
3.3.7. Time commitment required for Rally Post attendance
The tracked respondents were followed through the different services provided at the RP in order
to assess the amount of time they had to invest in attending an RP. Program activities as well as
waiting time and nonprogram activities carried out during the waiting periods were recorded
during the observation, and caregivers were also asked to estimate the time it took them to get to
the RP. Results are presented in Table 3.19.
The data reveal that respondents spent, on average, 20 minutes, varying between 2 and 90
minutes, to get to the RP. In addition, they spent a total of about 2 hours at the RPs with very
high variability—between 15 minutes and 4 hours. This variation is certainly a function of the
number of other respondents already present at the time of arrival and of the internal
organization of the RP. Only 3 out of the 19 RPs observed were well organized in the sense that
health agents were giving out numbers to every arriving caregiver to ensure that caregivers
would be attended in the order in which they arrived. In the remaining RPs, the organization was
rather chaotic and sometimes the first caregivers to arrive were the last ones to leave. The total
time implication of RP attendance, including travel time to and from the RP, in addition to the
time spent there was, on average, 2.5 hours—varying between 35 minutes in the best case to 7
hours in the worst case.
Table 3.19. Time commitment required for RP attendance
Time allocation by tracked respondents Variable name
Mean
[N = 38] Range
Average time taken to get to RP (in minutes) q306 20 (18) 2-90
Total time spent at RP (in minutes) ttime 117 15-242
Total time implication of RP attendance (in minutes) tottimeimpl 156 35-422
Time per program activity spent by respondent (in minutes):
- Registration (n = 37)
- Education session (n = 38)
- Vitamin A distribution (n = 17)
- Deworming (n = 2)
- ORS distribution (n = 0)
- Weighing (n = 38)
- Counseling after weighing (n = 14)
- Vaccination child (n = 9)
- Vaccination adult (n = 1)
act1
minutes
5
10
4
3
0
4
3
3
3
1-30
1-23
1-14
2-3
1-14
1-6
1-8
3
Time spent waiting by respondent (in minutes) act1
minutes
94 4-234
% Range
Proportion of waiting time/total time at RP propwaittotaltime 74 10-97
Among the program activities provided at the RPs, the tracked respondents spent, on average,
more time for the education session than for any of the other services. However, by far the most
time-consuming activity at the RP was waiting in line for the next service. On average,
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
39
respondents spent 74 percent of their time at the RP waiting. This ranged up to 97 percent—
indicating that at least for one respondent, only 3 percent of the four hours she spent at the RP
was dedicated to receiving services, whereas the remainder was waiting. While waiting, the
respondents fed, cleaned, or played with their children, and/or talked to friends and neighbors.
For one RP where both respondents spent less than 25 percent of their time waiting, no
bottleneck had been identified; this was the RP with the lowest number of participants and the
lowest ratio of participants per health staff person.
3.4. Staff perceptions about Rally Post operations
Information about perceptions and opinions of the health staff regarding the RPs was obtained in
individual interviews with 19 health agents (one per RP) and in three focus group discussions
with colvols and MCH supervisors. This section summarizes the results on their general
perceptions about RPs, the types of problems they face, their opinions on the supply situation,
their experiences with weighing scales and bringing health and ration cards to the RP meetings,
and finally, their suggestions on how to improve the delivery of services at RPs.
3.4.1. General perception of Rally Posts
Most of the health agents interviewed felt that the RPs were working well in general. Half of
them reported high participation rates in the RPs they organized. Overall, they considered RPs
as an important activity for the communities they served because they perceived that the services
they offered (education, immunization, ORS, vitamin A and anti-helminths) were most needed in
these communities. They also felt that people enjoyed going to the RPs.
In all three focus groups, the discussants voiced that mothers liked RPs because they get to know
their child’s weight and whether their child is malnourished or not, and they receive advice from
the health staff if the child is malnourished. The discussants also expressed that more people
were interested in RPs now than in the past. The important contribution for community
development of helping children be healthier was also raised in the discussions in one of the two
focus groups with colvols and in the MCH supervisors’ group. In addition, according to the
MCH supervisors, RPs are important because the services also reach those in the population who
do not have access to the food aid program. MCH supervisors considered the RPs as the key
activity of the MCH program because this is where program beneficiaries are identified.
3.4.2. Perceived problems with Rally Post operations
This section presents findings from the interviews with WV health staff on the problems they
encountered with trying to ensure the smooth operations of the RP. Some of the problems
mentioned included issues with supplies, the late arrival of participants, participants arriving
without their health and ration cards, and a high participant/staff ratio. Other issues included the
need for better training for colvols and a better transport situation for all health staff, so that they
would be able to reach their beneficiaries more easily. These problems are summarized in this
section, combining data from the focus groups with health staff and the individual interviews
conducted with health agents at the RPs.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
40
3.4.2.1. General problems
When asked about the problems they faced in carrying out their responsibilities at the RP, the
majority of health agents mentioned the lack of medical supplies and vaccines, and the lack of
other materials, such as scales and megaphones, insulated containers and ice for the vaccines,
and chairs, benches, and tables. These problems were also voiced in all three focus groups. In
addition, the MCH supervisors expressed their concerns regarding participants who arrive late at
the RP and miss the education session.
In one of the two focus groups with colvols, discussants felt that the high participant/staff ratio
made RPs very time-consuming both for the health staff and for the participants. Some of the
health agents seemed to have this in mind when they complained about not receiving sufficient
assistance from other health agents or colvols. They also deplored the fact that some of their
assistants (i.e., colvols) were not able to enter the weight in the health card, to interpret the
growth chart, or to immunize children, which increased the burden on them. As reported earlier
(Section 3.3.1), the participant/staff ratio, and especially the participant-health agent ratio in the
19 observed RPs, was indeed quite high.
Another important problem that many health agents mentioned was the lack of availability of
transport facilities to pick up the vaccines at the dispensary or WV office, which causes delays in
the start of the RPs. They also mentioned that the large distances between the different
communities where they worked were a constraint.
3.4.2.2. Supply situation
Tables 3.20, 3.21, and 3.22 present the results from the focus group discussions and the
individual interviews regarding the availability of medical supplies, vaccines, and health cards at
RPs. The results presented in Table 3.20 confirm the data from the observations in 19 RPs (see
Section 3.3.4) and the concerns expressed by health agents in interviews that the lack of supplies
was a real constraint. The data indicate that medical supplies were rarely available in sufficient
quantities to cover all the population attending the RPs, and were sometimes not available at all.
Health agents reported that they had to give new appointments to caregivers and their children
when they ran out of supplies during the RP meetings or that they distributed during home visits.
Only one of the health agents reported teaching caregivers the use of vitamin A-rich foods when
vitamin A capsules were unavailable, and only three indicated teaching caregivers how to
prepare a sugar-salt solution at home when packets of ORS were unavailable at the RPs.
Table 3.20. Availability of medical supplies in RPs
Focus group
(X indicates that theme was present in
focus group transcript)
Individual
interviews
Health agents
Vitamin A, anti-helminths, ORS Colvols 1 Colvols 2
MCH
supervisors
[N = 19]
(Presence in
interview
transcript)
Problems
- Ran out during the RP rare
often
sometimes
sometimes
often
sometimes
12
17
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
41
Focus group
(X indicates that theme was present in
focus group transcript)
Individual
interviews
Health agents
Vitamin A, anti-helminths, ORS Colvols 1 Colvols 2
MCH
supervisors
[N = 19]
(Presence in
interview
transcript)
- Not available at all
Actions taken
- Try to get it from WV office
- Give another appointment
- Distribute during home visits
- Request to supervisor for supplies
- If vitamin A shortage, explain which foods are
rich in vitamin A
- If ORS shortage, explain how to prepare ORS at
home
X
X
X
X
X
X
X
X
2
14
4
1
1
3
Strategies for distribution if running out
- First come, first serve principle
- Priority to ones who receive it for the first time
- If ORS shortage, priority to children with diarrhea
- If vitamin A shortage, priority to mothers with
newborns
- If vitamin A shortage, priority to children with
low weight
- If anti-helminths shortage, priority to children
with symptoms of parasites
X
X
5
3
6
2
1
2
Reasons for running out/non-availability
- Shortage of stock at WV office
- Insufficient quantities provided by WV office
- Shortage of stock at Ministry level
- Insufficient quantities taken by health agent/too
many beneficiaries
- Long order time
X
X
X
X
X
X
X
X
15
13
1
The health agents indicated that they followed a variety of strategies when they had to decide
which children should receive the vitamin A supplements, deworming tablets, or ORS packets on
days of shortage. Among the most common strategies were the use of a “first come, first served”
approach, and prioritizing those who seemed to be most at need. The most common reasons
reported by the health staff for the shortage of supplies were insufficient amounts provided by
the WV office or shortage of stock at the WV office.
The availability of vaccines was also reported to be limited (see Table 3.21), again confirming
our observations and the reports from interviews with health agents; most health staff reported
having experienced a situation where certain vaccines were not available at all or where they ran
out of vaccines during the RP meeting. BCG and polio vaccines were the ones reported to be
most frequently unavailable, whereas the DTP, anti-tetanus, and measles vaccines were less
frequently reported as being unavailable. The most commonly taken action by the health staff
when they ran out of vaccines was to give participants another appointment. Because of the need
to maintain the cold-chain, the health staff did not use home visits to give immunizations, as they
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
42
sometimes did when other supplies were missing. The strategies used to decide which children
should be given priority for immunization on days of shortage and the reasons for the availability
problems were quite similar to the ones described for the medical supplies and are provided in
Table 3.21.
Table 3.21. Availability of vaccines in RPs
Focus group
(X indicates that theme was present
in focus group transcript)
Individual
interviews
Health agents
Vaccines Colvols 1 Colvols 2
MCH
supervisors
[N = 19]
(Presence in
interview
transcript)
Problems
- Ran out during the RP
- Not available at all often
often
often often
10
18
Actions taken
- Try to get it from WV office
- Give another appointment
- Request to supervisors for vaccines
- Send mother to hospital for immunization
X
X
X
X
2
6
1
Strategies for distribution if running out
- First come, first serve principle
- Priority to ones who come for the first time to
the RP
- Priority to youngest children/newborns
- Priority to MCH beneficiaries X
X
3
3
4
Reasons for running out/non-availability
- Shortage of stock at WV office
- Insufficient quantities provided by WV office
- Shortage of stock at Ministry level
- Too many beneficiaries
- No gas for refrigerator
X
X
X
X
X
X
15
7
8
3
Although the health agents reported problems with the availability of insulated containers and ice
for the conservation of the vaccines when asked about general constraints, they all reported in the
individual interviews that it was always possible for them to keep the vaccines cold throughout
the day. This was confirmed in the focus group discussions with the colvols. However, the
MCH supervisors voiced that ice was not always available or was of bad quality (not frozen
deeply enough). Similarly, they expressed concerns regarding the poor quality of some of the
insulated containers, the sometimes negligent manipulation of the vaccines by health staff, and
the fact that containers were sometimes exposed in the sun in poor RP venues.
Table 3.22 presents the results regarding the availability of health cards at RPs. No information
is available for the ration cards, as they are not distributed at the RPs. The health staff reported
frequently running out of health cards during the RP or not having any to start with.
Understandably, these problems are not perceived by the health staff as being as severe as the
lack of medical supplies and vaccines. When cards are unavailable, health staff reported that
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
43
they simply note down the services on a piece of paper and/or in the health agent’s register and
complete the cards once available.
Table 3.22. Availability of health cards in RPs
Focus group
(X indicates that theme was present in
focus group transcript)
Individual
interviews
Health agents
Health cards Colvols 1 Colvols 2
MCH
supervisors
[N = 19]
(Presence in
interview
transcript)
Problems
- Ran out during the RP
- Not available at all
X
X
X
X
X
X
16
14
Actions taken
- Try to get it from WV office
- Register services in register and/or on a
paper and give new card next month
- Register in an old small health card
XX
1
15
4
Strategies for distribution if running out
- First come, first serve principle
- Priority to ones who come for the first time
to the RP
- Priority to youngest children/newborns
- Priority to MCH beneficiaries/ malnourished
children
X
X
XX
X
7
5
4
1
Reasons for running out/non-availability
- Lack of stock at WV office
- Lack of stock at Ministry level
- Too many beneficiaries
- Too many beneficiaries who lost card
XX
X
X
X 15
2
Again, the “first come-first serve” strategy is commonly used when health cards are in short
supply. The lack of health cards at the WV office or at the Ministry level were noted as the most
frequent reasons for the unavailability problems encountered at the level of the RPs.
3.4.2.3. Weighing scales
Twelve of the 19 health agents indicated that they had never experienced any problems with the
weighing scale. The problems mentioned by the remaining 7 health agents were the following:
(1) that they did not have their own scale and had to share with other health agents (n = 5); (2)
that the needle of the scale gets blocked (n = 2); and (3) that it is difficult to fix the needle at zero
(n = 1). None of them reported that they had an extra scale, with one exception, where the health
agent owned a scale that did not belong to the project. These findings were confirmed in the
discussions with all three focus groups.
The strategies that the health staff indicated using in response to their problems with the
weighing scales included trying to get another scale from another health agent, giving a new
appointment to participants sometime during the same month while trying to get the scale
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
44
repaired in the meantime, or simply not weighing children during that month and requesting
another scale.
When asked whether they calibrated their scale, the majority of health agents (as well as the
colvols and MCH supervisors) explained that they usually checked if the needle of the scale
points to zero before starting the RP meeting and, in addition, several times during the RP. This
shows that calibrating a scale was not understood as checking the accuracy of the scale with a
standard weight, but simply as ensuring that the needle points to zero. The reported frequency of
setting the scale to zero during the RP varied considerably, from just once to 10-15 times. A few
even said that they did it each time before a new child was weighed.
3.4.2.4. Participants arriving at RPs without health cards and ration cards
One common problem encountered by health staff was that caregivers often forgot to bring their
health card and/or ration card to the RP meetings. Only six health agents indicated that
caregivers attending their RPs always had the health card of their child with them and three
indicated that they always brought the ration cards to the RP. However, half of the health agents
were unable to report experiences with the ration card because the distribution of the cards had
only started recently.
According to the health staff, the most frequently used strategy to deal with forgotten health
cards is to ask caregivers to return home to get the card, or in the case where the caregiver lived
too far away, to register the services provided to the child in the health agent’s register and/or on
a paper and to remind the caregiver to bring the child’s health card the following month. In the
case of the ration card, the caregivers are also sent back home if they do not live too far, but if
they do, they are asked to bring the card to the Mother’s Club.
3.4.3. Suggestions on how to improve Rally Post services
Most of the suggestions made by the health agents were in direct relation to the problems
identified earlier. In order of priority, they recommended that program managers take the
following actions:
ƒ Provide medical supplies and vaccines regularly and in sufficient amounts in order to serve
all RP participants.
ƒ Provide all materials needed to run the RPs effectively (megaphone, scale, insulated
containers etc.) and to improve the work environment (chairs, tables, benches).
ƒ Understand that the health staff needs transport facilities (horse/mule) in order to efficiently
carry out their work and to be able to start the RP meetings on time.
ƒ Hire more colvols to assist them in their work and provide the colvols with training in growth
monitoring and promotion, immunization, vitamin A distribution, and family planning before
they start their work.
ƒ Help them to network with other health agents in order to get more assistance in running the
RPs.
ƒ Contract ice-sellers, so that they do not need to pay themselves for the ice to preserve the
vaccines and to ensure that ice is always available nearby.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
45
ƒ Offer training in how to better organize the participants’ flow through the RP activities as
well as regular supervision in order to improve their performance.
3.5. Beneficiary perceptions about Rally Posts
In this section, beneficiary perceptions regarding the different Rally Post services are presented
as well as their suggestions to improve these services. Information was obtained from tracked
respondents (n = 38), from exit interview respondents (n = 59), and from home interview
respondents (n = 30).
3.5.1. Perceived importance of different Rally Post services
Table 3.23 shows the perceptions of respondents regarding the importance of the different
services provided at Rally Posts. About half of the tracked respondents and exit interview
respondents considered immunization as the most important service, in order to protect them
and/or their children against illnesses. Far fewer respondents judged weighing or education to be
the most important service. One-quarter could not point out a specific service, but felt that all
services were equally important. When asked about the second most important service at Rally
Posts, respondents were more likely to cite child weighing, education and immunization services
than vitamin A supplementation, deworming, ORS distribution, or post-weighing counseling.
Again, a quarter of the respondents could not identify a second most important service, but
considered all services, after the most important one they had identified, to be equally important.
None of the respondents were able to specify a service that they would consider as the least
important.
Table 3.23. Perceived importance of services at Rally Posts
Characteristics Variable name Numbersa
Service considered most important and reasons (N = 97)
- Immunization
o Protect against illness
- Weighing
o Knowing weight/growth of child
o Knowing child’s status/if healthy
- Education
o Learn how to take care of child
o Learn importance of vaccines/how to prepare
ORS/how to prepare food for child
- Post-weighing counseling
o Knowing child’s weight
- Vitamin A dosage
o Protects child
- All services equally important
q216/217
+q349/350 53
9
8
1
1
25
Next most important service and reasons (n = 72)
- Weighing
o Same reasons as above
- Education
o Same reasons as above
- Immunization
o Same reasons as above
- Vitamin A dosage
q218/219
+q351/352 23
14
13
3
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
46
o Protects the child
o Improves child’s appetite
- Post-weighing counseling
o Same reasons as above
- Deworming tablet dosage
o Cleans the child’s intestines
- ORS packets
o Protects the child
- All services equally important
1
1
1
16
Least important service (n = 72)
- None
q220/353
40 a
 Data from exit interviews with randomly chosen caregivers who were accompanying a food aid beneficiary child [N = 59] were
combined with data from the tracking of respondents through the different services provided at the RP [N = 38].
We also have information from the home interviews (n = 30) regarding the perceived importance
of the different Rally Post services. In general, Rally Posts were well appreciated by these
respondents. They felt that the immunization, weighing, vitamin A supplementation, and ORS
offered at the Rally Posts were important for their children’s health. About half of them
considered immunization as more important than all the other WV services they used; one
respondent felt that the weighing was more important for her child’s health than other services.
It is important to note, though, that many respondents had difficulties ranking different services
according to their importance, as they felt that they were all important. Very few respondents
indicated that Rally Post attendance was important as an entry point into the food aid program.
The home interviews showed that beneficiaries generally perceived immunization as being an
important service because it protects the child against different illnesses. They felt that a fully
immunized child would stay healthy, and vaccines were perceived as guarding children’s health.
In addition, they explained that immunization of pregnant women would also help protect the
children’s health.
The importance of weighing was seen as a means to check whether the child was growing well
or not. The respondents voiced that a regular check of the child’s weight was very useful in
order to identify changes in the child’s growth, since growth faltering could go unnoticed for
some time. A few respondents admitted that weighing was important in order to know the
child’s weight and to check if the weight was adequate, but felt, at the same time, that this
service did not have a direct impact on their child’s health.
Respondents expressed that the vitamin A capsules were important because they prevented their
children from having eye problems. The ORS was perceived as important when the child had
diarrhea, in order to replace lost water.
3.5.2. Ease of use
Approximately one-third of the home interview respondents felt that the services offered at the
RPs were the easiest ones to use of all the WV activities they attended. They perceived that the
RPs were convenient because they were relatively close to their homes and the time spent there
was reasonable, especially if they arrived early in the morning. One respondent also highlighted
that services at the RPs were reliable, in the sense that immunization, weighing, and vitamin A
supplementation were always offered at the monthly RP.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
47
However, one respondent felt that the time involved in attending the Rally Post made this service
package less convenient to use. She expressed that the weighing was particularly time￾consuming because there were always many children to be weighed.
3.5.3. Suggestions on how to improve the Rally Post services
Table 3.24 presents the suggestions made by the tracked respondents regarding potential
improvements to the RPs in order to make them more relevant. Many respondents had no
suggestions. However, the suggestions that were made by those who contributed can be grouped
into two types: (1) suggestions related to the internal organization of the RPs, such as better
venues for the Rally Posts, no changes in dates, repeated announcements of the dates, and timely
arrival of the health staff; and (2) suggestions requesting additional services to be offered at the
Rally Post, such as general consultations, free distribution of drugs for sick children or parents,
and regular availability of chlorine for water treatment.
Only two of the home interview respondents had suggestions on how to improve the RP services,
and both related to the weighing—one recommended having more scales at the RPs in order to
make the weighing quicker, and the other asked that health staff shows more respect when doing
the weighing.
Table 3.24. Suggestions for RP services improvement made by participants
Suggestions on how the services at the RP could be improved
(several answers possible):
Variable
name
Numbers
[N = 38]
- Have general consultations for sick persons at RPs or construct dispensary
- Have the chlorine solution for water treatment always available
- Distribute free drugs to sick children/parents
- Ensure that health agents arrive on time
- Improve the Rally Post venue
- Train health agents and colvols
- Announce the RP meeting several times
- Do not change the RP dates
- No suggestion
q355 6
1
1
1
1
1
2
1
26
3.6. Summary of findings regarding operations at the Rally Posts
3.6.1. Operations and quality of services at the Rally Posts
Attendance and participant/staff ratio. On average, 74 participants (children and pregnant or
lactating women) attended the RPs observed, with considerable variability between the sites—
from a low of 9 participants to a high of 169. Not surprisingly, the overall impression is that RPs
are generally crowded. The participant/staff ratio also varies widely, but is generally high
(average of 55 participants/health agent, and up to 151 in one RP).
Sequencing of activities and bottlenecks. Overall, it appears that the sequence of activities
described in the program implementation plan is rarely used. Our data, however, show no
evidence that the likelihood of experiencing bottlenecks is related to the type of program
sequence used. Factors responsible for bottlenecks were invariably the large numbers of
attendants and the high participant/staff ratios.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
48
Education. Our observations indicate that attendance at the education session is still a problem
because the session is usually held first thing in the morning and many participants arrive late.
The plan suggests holding one or more additional sessions to accommodate late arrivers; this
occurred in only four of the observed RPs.
Second, none of the education sessions observed corresponded to the recommended topic for the
month. While some flexibility in the choice of topics may be desirable to ensure that topics of
particular relevance are addressed in a timely fashion (for example, the use of ORS during
periods of high diarrhea incidence), the risk is that some other important topics may never be
addressed.
Insufficient use of communication materials was also observed in most education sessions, in
spite of the fact that material had been provided to the health staff. This is likely to limit the
effectiveness of the education, as is the large number of participants, the distracting environment,
and the short time allocated to this activity (between 1 and 23 minutes).
Overall, our findings suggest that in spite of some rare attempts to conduct dynamic and
interactive education sessions, the generally distracting environment due to the many activities
taking place at the same time and in the same area, the crowding, and the high participant/staff
ratio make effective education difficult in the context of the RPs. However, the RPs can still be
used as effective venues to reinforce some of the topics discussed in other more appropriate BCC
venues (like the MCs; see below) and also to present information on general topics such as
hygiene, the use of ORS, or the importance of immunization.
Registration, vitamin A supplementation, and distribution of deworming drugs and ORS. The
registration of the participants at the RPs was by far the most significant bottleneck observed
(primary bottleneck in 16 of the 19 RPs). This appears to be caused by the large numbers of
participants and insufficient staff. While it is a slow process, the registration of beneficiaries
generally seems to proceed as planned—that is, cards are usually updated with the information
on immunization, vitamin A, and deworming dosage received on the day of observation. In
contrast, bringing the ration card to the RP—as required by the program to register the date of
attendance in order to be eligible for the monthly food ration—has clearly not yet turned into a
routine.
The quality of vitamin A supplementation and distribution of deworming drugs was assessed
indirectly by verifying whether children who received it on the day of observation were actually
due, and calculating the percentage of children due who did not receive it. In general, it appears
that when given, the schedules of administration of vitamin A and deworming were followed.
However, the lack of availability of vitamin A, and particularly of deworming tablets and ORS,
was a generalized problem in the observed RPs.
Growth monitoring and promotion (GMP). In general, the mechanical aspects of GMP activities
at the RPs appear to function relatively smoothly. All the caregivers, either tracked or
interviewed, reported that their child was weighed, and in most cases the weight was plotted and
the child’s nutritional status assessed. Only about half of the caregivers, however, were
informed about their child’s weight and even fewer received some post-weighing advice. The
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
49
likelihood of receiving counseling appeared slightly higher among caregivers who had a
malnourished child, but even then, more than half of the caregivers with a child classified as M2
received no individual advice following the weighing and plotting procedures.
The quality of the weighing and plotting of the weight to assess children’s nutritional status was
less than optimal and resulted in 28 percent of the children being misclassified into the different
Gomez malnutrition categories. For the recuperative group, where children’s eligibility to
receive food rations is based on whether or not they are malnourished (M2 or M3), this is likely
to result in the misclassification of children as beneficiaries or nonbeneficiaries of the program.
In our sample of 38 tracked respondents, misclassification errors resulted in a similar number of
eligible (M2) children who were missed (misclassified as Normal or M1) and therefore excluded
from the program (n = 2), and of non-eligible children (Normal or M1) who were mistakenly
classified as M2 and thus included in the program (n = 2). Although, in this example, the errors
cancel out with regards to the amount of program resources such as food and other services
required, misclassification problems reduces the cost-effectiveness of programs. This is because
it results in the leakage of program benefits to children who are less likely to benefit, and in a
lower coverage of children who need it and would be more likely to benefit from it.
Our assessment also confirms a widely documented problem with GMP activities carried out in
clinical settings, such as the RPs, which is that the opportunity to use the child’s weight and
growth information to interact with the caregiver and to tailor individual counseling messages to
the specific needs of the child is lost. In the RPs observed, even transmitting the information to
the mother about the child’s weight and nutritional status was difficult to achieve, let alone
providing relevant and tailored individual counseling. Given the high participant/staff ratio, the
distracting atmosphere, and the numerous activities going on at the same time in the RPs, it is
probably unrealistic to expect that GMP could offer a venue for effective education and
individual counseling of caregivers.
Immunization. A large proportion of children and caregivers included in our tracking or exit
interviews at the RPs were fully immunized. For those who were not, however, close to half of
them were not immunized on the day of the RP because the service was not offered. This, in
turn, was usually due to a lack of a supply of vaccines. Clearly the lack of supplies of vaccine
and, as indicated earlier, of vitamin A, deworming tablets, and ORS packets, is a widespread
problem that needs to be addressed by the program in order to improve its coverage of these
important preventive services.
3.6.2. Staff perceptions and suggestions for improvement of the Rally Posts
General perceptions. Overall the data suggest that the health staff views the RPs as an extremely
important component of their services to the communities and particularly of their role in
fostering children’s growth and health. They also expressed that they consider the RPs important
because the services they offer also reach families who do not have access to the food aid
component of the program. Most of the health agents interviewed felt that the RPs were working
well, that attendance was high, and that participants enjoyed the RPs.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
50
Perceived problems. The problems most commonly voiced by the health staff with regards to
the RPs were the lack of supplies, both medical and general, and furniture. Other problems
mentioned were the high participant/staff ratios, the late arrival of participants, participants
forgetting their health or ration cards, and the lack of transport for them to carry out their
activities. Furthermore, the health agents felt that it was imperative that the program provide
more training for colvols so that they could better assist them in their duties.
Suggestions on how to improve the RP services. The suggestions made by the health agents to
improve the services offered at the RP were very much in line with their comments regarding the
problems they faced. In order of importance, they recommended that the program takes the
following actions: (1) provide medical supplies and vaccines regularly and in sufficient
amounts, as well as all other materials and furniture needed; (2) assist with transport facilities;
(3) hire more colvols and provide them with better training; (4) provide ice to maintain the cold￾chain for vaccines; and (5) offer training to improve the logistics at the RPs and to improve their
own performance.
3.6.3. Beneficiaries’ perceptions and suggestions for improvement of the Rally Posts
Overall, program beneficiaries indicated that they were quite satisfied with the RP services.
Immunization was perceived to be the most important service by most respondents, although, in
general, respondents had difficulties ranking them in order of importance. Few respondents had
suggestions on how to improve the RP services, but those who did mainly focused on ways to
improve the internal organization of the RPs, such as improving the venue, ensure that dates are
announced and do not change, and encouraging staff to arrive on time.
3.6.4. Conclusions on operations at Rally Posts
Overall, our assessment suggests that the RPs are generally operating in accordance with the
World Vision’s implementation plan. The problems encountered with operations relate primarily
to organizational and logistical issues such as crowding, the high participant/staff ratio, long
waiting times, bottlenecks at registration, and lack of supplies and transport for staff. In terms of
quality of services, the areas that could benefit from some improvement are the general
education sessions and the communication between health staff and caregivers in the context of
the growth monitoring and promotion activities.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
51
4. PROGRAM OPERATIONS AT THE MOTHERS’ CLUBS
This section is structured as the previous one: it first describes the activities and operations in
the MCs, as laid out in the program implementation plan; it then briefly summarizes the research
methods used for assessing the MCs; and it presents the results regarding implementation,
quality, and perceptions of staff and beneficiaries.
4.1. Description of the Mothers’ Clubs operations according to implementation
plan
Results from our initial formative research (Menon et al. 2002b) revealed that the MCs are an
ideal setting for effective BCC activities since they are located close to the mothers’ homes and
include only a small group of participants, resulting in minimal distraction (especially compared
to the RPs). Thus, the WV program decided to use the MCs as the primary venue for BCC
activities.
The formative research also showed that while the MCs were an ideal setting for BCC activities,
there were many aspects of the club sessions that needed to be modified to ensure that the BCC
would, in fact, be effective. Among other things, the clubs were reorganized in 2003 to include
groups of mothers of a particular physiological state and/or with children of a particular age. For
instance, separate clubs now exist for pregnant women, for lactating mothers up to 6 months,
and, depending on the program model, for mothers of children 6-23 months old (preventive) or
for mothers of malnourished children 6-59 months old (recuperative).
Finally, observations of MC sessions during the formative research had shown a clear need for
training the field staff in appropriate methods of communicating with adults. Thus, the WV staff
was trained in 2003 in the use of adult education techniques, in addition to receiving additional
technical training on infant and young child feeding and care practices. The staff were also
equipped with newly developed visual communication materials, which they could use as a guide
for the different learning sessions on infant and young child feeding.
4.1.1. Schedule of education sessions at the Mothers’ Clubs and duration of exposure, by
program group
The schedule of sessions for the MCs was reorganized to be age-specific and to address
behaviors that are immediately relevant for the child’s health, development, and growth at a
particular age. This was done for the pregnant and lactating MCs from both the preventive and
the recuperative groups, and for the MCs with mothers 6-23-month-old children in the preventive
group. This was not done for the MCs with mothers of malnourished children in the recuperative
group, because the objective of these MCs is to discuss the recuperation of malnourished
children, irrespective of their age.
According to the new MC schedule, women should be exposed to materials and advice about the
initiation of breastfeeding and exclusive breastfeeding during pregnancy (see Annex 4a), and the
messages should be reinforced throughout the first few months of lactation. Similarly, a session
introducing lactating women to complementary feeding should be held when infants are four
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
52
months old, and a follow-up session on nutritious complementary food when infants are five
months old. This is intended to prepare caregivers for the appropriate introduction of
complementary foods when infants are six months old. Once the child reaches 6 months of age,
mothers in the preventive group start attending the clubs for mothers of 6-23 month old children
(see Annex 4b) and again, emphasis is put on educating mothers about the current needs of their
child such as the introduction of complementary foods between 6-8 months of age, the use of
enriched porridges to improve the quality of the child’s diet (6-8 and 9-11 months), and the
gradual transition from complementary feeding to the family diet (12-23 months).
The schedule of sessions for malnourished children from the recuperative group is different, and
consists of a fixed set of nine learning sessions (see Annex 4c), which do not take the age of the
child into consideration.
The duration of attendance by mothers at the MCs (and other BCC activities) also differs
between the two program groups; mothers in the preventive program may benefit from the BCC
activities for longer than mothers in the recuperative program. For example, a mother who starts
attending the MC during pregnancy (as expected), continues to attend the same club throughout
her first six months of lactation and subsequently, for another 18 months when the child is
between 6 and 23 months of age. Thus, mothers in the preventive model may attend the MCs for
up to 30 monthly sessions, without interruption.
In the recuperative model, this continuity of attendance is not the norm. In order to be eligible
for the MCs, mothers from this group must be either pregnant or lactating, and/or have a
malnourished child 6-59 months of age. Thus, the length of participation of mothers in the MCs
depends on their physiological status and on their child’s nutritional status. Mothers are eligible
to attend the MCs for 6 months during pregnancy, 6 months during lactation (the first 6 months
postpartum), and up to 9 additional months if they have a malnourished child at any point in time
between 6 and 59 months of age. Thus, the maximum possible non-interrupted time mothers in
this program group can attend the MCs is 21 months, and this will happen only if they start
attending during pregnancy, continue through lactation, and have a malnourished child when
they reach 6 months postpartum (the malnourished child can be the 6-month-old one or an older
child, as long as s/he is less than 60 months of age).
4.1.2. Communication materials used
The communication materials currently used in the MCs for the infant and young child feeding
topics were adapted by the IFPRI-CU-World Vision team from materials developed by Freedom
from Hunger (FFH) for use with their Credit for Education programs in Haiti (see Loechl et al.
2003a). All materials are organized into learning sessions, which are designed to be completed
in about one hour. Each of these sessions covers specific key practices using a variety of
communication methods, including a presentation by the health agent/colvol, small group
discussions among participants, role-plays, songs, and theater-like activities. All the sessions
consist of a set of instructions to the health agents/colvols, accompanied by activities for them to
carry out with the group of participants in order to achieve the objectives of the learning session.
The instructions for conducting the learning session are accompanied by visual materials. The
materials are described in detail in Loechl et al. (2003a).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
53
Other communication materials are used for topics other than infant and child feeding, such as an
album of images with key messages related to other aspects of maternal and child health,
including immunization, pre- and postnatal care, preparation for child delivery, diet for pregnant
and lactating mothers, hygiene and environment, diarrhea and preparation of ORS, family
planning, and HIV prevention.
4.1.3. Final note on organization of the Mothers’ Clubs
Under the current program implementation plan for the MCs, women are brought together in
small groups (15-20 women). Health agents or colvols (or both) facilitate the meetings, which
are held once a month. Meetings take place in a variety of locations, including dispensaries,
local schools, or participants’ homes.
As noted above, the MCs are expected to be as homogenous as possible in terms of the child’s
age and/or the mother’s stage of pregnancy or lactation. However, other criteria, such as
distance from the mothers’ homes and the need to ensure a minimum of 15 participants per club,
are taken into consideration. The organization of the clubs thus reflects a compromise between
three factors, i.e., stage of pregnancy/child age, distance from the mothers’ homes, and ensuring
that each club has at least 15 members. In principle, in situations where a club consists of a
relatively heterogeneous group (e.g., where there are women with children of widely different
ages), both the health agent and the colvol should facilitate the Mother’s Club meeting and they
should split the group into two subgroups to provide relevant and timely messages to mothers
based on their specific needs.
Unlike the RPs and the FDPs, mothers are required to attend the MCs themselves and cannot
send another family member to use this service. This ensures that mothers have direct exposure
to the BCC activities, which were developed for them.
4.2. Methods used to assess the Mothers’ Club operations
The data used to assess the operations of the MCs were collected using the following methods:
ƒ Observation of general issues related to the MCs organization (N = 20);
ƒ Observation checklist to assess the quality of education (N = 20);
ƒ Structured exit interviews with randomly selected program beneficiaries at the end of each
observed MC (N = 41);
ƒ Individual semi-structured interviews with health agents at the end of each observed MC
meeting (N = 20);
ƒ Semi-structured interviews with beneficiary women in their homes (N = 30);
ƒ Focus group discussions with health agents, colvols, and MCH supervisors (N = 5).
4.3. Operations and quality of services at the Mothers’ Clubs
This section presents results regarding the organizational aspects of the MCs and the quality of
education provided at the Clubs. It also summarizes the information gathered on the
participants’ recall and adoption of recommended practices and behaviors discussed at the MCs.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
54
As in the previous section on RPs, the results are presented by topic rather than according to the
research methods used.
4.3.1. Organization of the Mothers Clubs and attendance
A total of 20 MCs in the preventive and recuperative program groups were observed. Of those, 5
were organized for pregnant women (n = 3 in the preventive program group and n = 2 in the
recuperative program group) and 5 for lactating mothers up to 6 months (n = 1 in the preventive
program group and n = 4 in the recuperative program group). In addition, 6 MCs for mothers of
children 6-23 months old in the preventive program group and 4 for mothers of malnourished
children 6-59 months old in the recuperative program group were observed. Although we had
planned to observe the MCs held by the 20 health agents working in our evaluation project area,
this was not possible in five cases because the health agents had to be replaced by a colvol on the
day of observation. Because our data collection schedule was tight, it was not possible to
reschedule the observations and therefore the MCs were observed as they were run on the day of
observation, i.e., by a colvol, as opposed to the health agent in charge.
4.3.1.1. Topics discussed in Mothers’ Clubs
Table 4.1 shows the session topics that were discussed in the observed MCs, by category of MC.
The “Session” numbers in the table refer to the location of the learning session in the
communication materials adapted by the IFPRI-CU-World Vision team (see Annex 4 for the
schedule of learning sessions for the different types of MCs). The learning session materials
were used in all the observed MCs to guide and animate the meeting.
Table 4.1. Session topics discussed in the different categories of MCs observed
Session topic
Category of MC observed Variable name
Breastfeeding
(Session #1-4)
LAM
(Session #5)
Complementary
Feeding
(Session #6-12)
Pregnant women
Lactating mothers
Mothers of children 6-23 months old
Mothers of malnourished children 6-59 months old
categori &
sessnumbrecod
4 MCs (#1-3)
2 MCs (#4)
0
0
1 MC
1 MC
0
0
0
2 MCs (#6+7)
6 MCs (#8-12)
4 MCs (#8+9)
The MCs for pregnant women focused on issues related to breastfeeding, such as early initiation
of breastfeeding, the use of colostrum, and the benefits of exclusive breastfeeding (sessions #1-
3). In one of the MCs for pregnant women, the lactational amenorrhea method (LAM) for birth
spacing was discussed (session #5). The MCs for lactating mothers addressed the following
topics: (1) detailed information and problem-solving related to breastfeeding (session #4); (2)
information on LAM (session #5); and (3) advice concerning the introduction of complementary
foods (sessions #6-7). In the two other categories of MCs (those targeting either mothers of
children 6-23 months old or mothers of malnourished children), issues related to complementary
feeding were discussed (sessions #8-12). These included a cooking and tasting trial where
mothers prepared and tasted different enriched complementary foods (Session #8).
4.3.1.2. Size of the Mothers’ Clubs and beneficiary characteristics
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
55
Most of the MCs observed were held either in the courtyard of a private house (n = 9) or in a
church (n = 8). Two used a school and one a dispensary as venues. The health staff always tried
to set up the group in a circle in order to facilitate interaction, although this was difficult in some
of the venues where the number of benches or chairs available was limited.
The data on attendance at the MC are presented in Table 4.2. Note that we differentiated in our
observations between women who the health agent had identified as regular participants (i.e.,
registered in his/her list for the observed Mother’s Club) and other people, such as
accompanying children, fathers, grandmothers or others, who were present at the meeting. On
average, 11 women who were registered for the MCs attended the meeting on the day of
observation, varying from 2 to 24. In 7 of the 20 clubs, all the registered women arrived before
the health agent started the learning session. In the remaining 13 clubs, on average, 3 women
arrived after the start of the session. For 8 of the MCs, the health agent pointed out that all
registered women were present; in the remaining clubs, on average, 3 women were absent. Most
of the MCs (16/20) included a number of participants other than the registered mothers. On
average, 9 additional persons were present. Counting both registered women and other
participants, the average number of people present at the MCs was 18, but were as many as 41
persons in one MC.
Table 4.2. Number of participants of Mothers’ Clubs
Characteristics
Variable
name
Mean
[N = 20] Range
Average number of registered women present at sessions q602 11 2-24
Average number of registered women arriving late q602A 3 1-10
Average number of absent registered women absence 3 1-10
Average number of other people present q604 9 2-17
Average number of total people present particpall 18 2-41
The data from the exit interview respondents (n = 41) indicate that the participating women had
attended the same MC an average of 4 times before the meeting on the day of observation. The
frequency varied from 2 to 10 times. All respondents reached the MCs on foot and therefore
nobody paid for transportation. The average travel time to get to the MCs was 18 minutes,
varying between 3 and 60 minutes.
According to the implementation plan, a group size of 15-20 participants is recommended with a
minimum of 5-6 participants if the health agent splits one MC in two subgroups. Our results
show that four MCs were outside the recommended range. Two of them had less than 5 women
registered and two others exceeded the upper limit of 20 participants.
4.3.1.3. Time commitments required for Mothers’ Club attendance and facilitation
On average, the MCs started 40 minutes later than scheduled (see Table 4.3). Note that the
variability is quite high, and that some Clubs even started earlier than scheduled (up to 25
minutes early). Only five MCs started within +/- 15 minutes of the scheduled starting time. The
sessions lasted for approximately one hour, with a range from 25 minutes to more than two
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
56
hours, depending on the session topic. As expected, the four cooking and tasting sessions (#8)
were by far the longest of all observed sessions.
Table 4.3. Duration of Mothers’ Clubs
Characteristics
Variable
name
Mean
[N = 20] Range
Gap between actual start time and scheduled start time of the session (minutes) gapsched 40 -25 - +135
Average session length (minutes) sessleng 66 25-150
Average total time implication for health staff (hours) (n = 18) timeimpl 2.7 1.7-5.3
Although the duration of the learning session itself was, on average, one hour, the total time
involved in chairing or attending a Mother’s Club was much higher. We did not directly
measure the time that the health agent/colvol spent at the location of the MCs, but we measured
the time between the arrival of the first participant and the departure of the last one. In general
this corresponded roughly to the time spent by the health agent/colvol at the MC venue. In two
cases, however, some of the participants arrived before our fieldworkers and their arrival time
could not be assessed, leaving these two MCs with missing information on this aspect.
On average, health staff spent approximately 3 hours for one MC, and up to 5 hours in one case
(see Table 4.3). Usually, the health agents or colvols arrived earlier than the participants in order
to prepare the venue and the communication materials; they also tended to stay longer to register
attendance in their lists, sign the mothers’ attendance in their ration cards, and answer individual
questions, if any. We also observed that in general, the health staff waited until most of the
registered women had arrived before starting the session. A special case was again the cooking
and tasting session. In two of the four observed cooking sessions, the women had forgotten to
bring along the ingredients they had promised to bring to prepare the recipes. They were sent
back home to pick up these ingredients, which significantly delayed the start of the session. The
start of meetings was also delayed in one case where the MC was scheduled on the morning of a
market day. The women came to the club meeting only after they had gone to the market.
Another difficulty with beginning at the scheduled start time is that most women do not own
watches in these rural areas. This probably explains why some women came one to two hours
early, while others arrived much later than the scheduled start time. The result is that the time
implications of running (for the health staff) or attending (for the mothers) a MC can be
considerable, in spite of the fact that the actual activity usually lasts only approximately one
hour.
4.3.1.4. Adherence to Mothers’ Club categories
Table 4.4 shows the age of the children of mothers who participated in the exit interviews at the
MCs. In general, the children’s ages matched the expected age range for each MC category,
with only a few exceptions; in the MCs for lactating mothers, three children were older than 6
months of age (they were between 6 and 9 months of age); and in the MCs for mothers of
children 6-23 months of age, one mother had a 4.5 month old infant. Thus, four of the 41 women
interviewed participated in the “wrong” type of MC.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
57
Table 4.4. Adherence to MC categories
Average age of children in each MC category, according to
respondent
Variable
name
Mean
[N = 34]a Range
Pregnant women (n = 4/11)
Lactating mothers until 6 months (n = 10/10)
Mothers of children 6-23 months old (n = 12/12)
Mothers of malnourished children 6-59 months old (n = 8/8)
agemres&
categori
2.1
4.7
11.1
31.5
1.2-2.7
0.4-9.2
4.5-22.4
16.1-52.8 a
 The sample of women interviewed in “exit interviews” consisted of 2-3 randomly selected women in each Mother’s Club
[n = 41]. Seven women in the MCs for pregnant women had not given birth and, therefore, they were excluded for the
calculations in this table [n = 34].
The MCs for pregnant women also included mothers who had already given birth, but this is
consistent with the implementation plan. This is because Clubs for pregnant women are formed
in a given community with women at different stages of pregnancy. All women continue to
attend the Club until all participating mothers have given birth. The Club then becomes a MC
for lactating women. This explains why four of the 11 women interviewed in the MCs for
pregnant women had infants between 1 and 3 months of age. They had given birth but kept
attending the same MC because other participants had not yet delivered their baby.
4.3.1.5. Adherence to planned schedule of learning sessions
According to the implementation plan, the schedule of learning sessions is set up in such a way
as to be age-specific for all categories of beneficiaries in the preventive program group. As
described above, this type of scheduling also exists for the pregnant and lactating MCs in the
recuperative program, but not for the mothers of malnourished children MCs.
We assessed the appropriateness of the scheduling of the MCs (see schedules in Annex 4a-4c) by
examining the age of the children of the women in the different MCs, focusing on the
appropriateness of the session topic and MC category for women with children in that age group.
Table 4.5 presents the age of the children of mothers who attended the different Clubs, by
session topic and MC category.
The sessions on breastfeeding (#1 and #2) are meant to be discussed when women are in their
last trimester of pregnancy. Both sessions emphasize the importance of breastfeeding—early
initiation, the benefits of exclusive breastfeeding, and the recommendation to continue
breastfeeding until the child is 2 years of age or beyond. The topics for lactating mothers with
infants 1-2 months old (#3 and #4) focus on how to ensure successful breastfeeding and
especially exclusive breastfeeding, and suggest ways to share experiences with breastfeeding
among mothers and discuss solutions for problems encountered. The presentation of the birth
spacing method LAM (#5) and its requirements is meant to take place when the infants are about
3 months old. According to the schedule, lactating women with infants 4-5 months old are
introduced to different aspects of complementary feeding (how children learn to eat, how to
breastfeed, and feed children between 6 and 12 months) to prepare them for the appropriate
introduction of complementary foods when the infants are six months old (#6 and #7).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
58
Table 4.5. Adherence to Mothers’ Club schedules: age (in months) of children who
attended the different MCs (from exit interviews; n=41)
Mothers’ Club category
Pregnant women
Still
pregnant
(n = 7)
Given
birth
(n = 4)
Lactating
mothers
(n = 10)
Mothers of
children 6-23
months old
(n = 12)
Mothers of
malnourished children
6-59 months old
(n = 8)
Topic (session number) Number
Child age
(months)
Child age
(months)
Child age
(months)
Child age
(months)
Pregnancy
 Breastfeeding (#1) 1 2.7 months
 Breastfeeding (#2) 2 1.8 and 2.6
monthsa
Lactation
 Exclusive breastfeeding (#3) 2 1.2 months
 Exclusive breastfeeding (#4) 2.1; 3.1; 3.4,
and 4.5
months
 LAM (#5) 2 0.4 and 6.7
monthsa
 Introduction of Complementary
Feeding (#6)
7.8 and 9.2
monthsa
 Learning to eat (#7) 4.9 and 5.0
monthsa
Mothers of children >6 months
 Cooking trial (#8) 6.2; 7.1; 10.9 and
11.2 months
16.1; 18.0; 18.8; 26.9; 41.7
and 52.8 months.
 Complementary Feeding for
children <12 months (#9)
11.8 and 18.2
monthsa
26.4 and 51.5 months.a
 Food variety (#10) 7.9 and 8.5
monthsa
 Prevention of diarrhea (#11) 8.6 and 22.4
monthsa
 Complementary Feeding for
children >12 months (#12)
4.5 and 15.7
monthsa
a
 In these cases, the two mothers who attended the same session topic were also frequenting the same MC.
A cooking and tasting trial (#8), where mothers prepare and taste enriched complementary foods,
is scheduled several times with different groups when children are 6-7 months old (last session
of a MC for lactating mothers; and first session for mothers of children between 6 and 23
months, or second session for mothers of malnourished children between 6 and 59 months). In
the MCs for mothers with children 6-23 months old, the next topics (scheduled when children are
between 8 and 11 months) concentrate on issues related to complementary feeding practices.
One session (#9) focuses on helping children between 6 and 12 months to eat well when healthy
as well as during and after illness; the next topic (#10) emphasizes the importance of food variety
for children 6-12 months of age, including examples of enriched meals; session #11 deals with
diarrhea prevention and hygiene; and session #12, which is meant to occur when children are
about 11 months of age, addresses issues concerning feeding children beyond 12 months of age.
In the MCs for malnourished children, these topics are presented in the sessions following the
cooking trial—one after the other in a fixed schedule that does not specifically take into account
the age of the participating children.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
59
The findings show that for the MCs for pregnant women, the schedule was generally followed,
and most women received information on breastfeeding at a very appropriate moment.
However, three of the four mothers interviewed at the MCs for pregnant women, who had
already given birth, listened to topics related to early initiation of breastfeeding and the use of
colostrum (#1 and #2) when their infants were already 2-3 months old.
In the MCs for lactating mothers, the picture was different and indicates that for many of the
mothers interviewed, the topics addressed at the MCs were discussed too late to be relevant for
immediate application with their infants (see Table 4.5). This was especially true for the
following cases: (1) one mother who attended the session on experiences with and constraints to
exclusive breastfeeding (#4) had an infant who was already approximately 5 months old; (2) one
mother who attended the session on LAM (#5) had an infant who was already 7 months old, i.e.,
beyond the period of potential usefulness of LAM; and (3) two mothers who attended the session
on the introduction of complementary foods at 6 months (#6) had infants who were already 8 and
9 months old, respectively. In contrast, the timing of the session on learning how to eat for
children 6-11 months of age (#7) was perfect for the two mothers who attended this Mother’s
Club (their children were both approximately 5 months old).
Mothers who attended the MCs intended for those with children aged 6-23 months old generally
had children who were either slightly too old or slightly too young for the topics addressed, again
precluding immediate application of the new practices. This was especially the case for a mother
of a 5-month-old infant who attended the session on feeding practices for children 12-23 months
of age (#12). Two mothers also attended the cooking session (#8) on special complementary
foods slightly too late, i.e., when their children were already 10 and 11 months old. These
complementary foods are particularly important for 6-12-month-old infants, so mothers should
be prepared in advance in order to start introducing them when the child is approximately 6
months of age. The timing of the food variety session (#10), however, was good; mothers
attending had children who were 8 and 8.5 months of age. Finally, the timing was slightly off
for two mothers with children older than 12 months who attended the session on feeding
practices for children under 12 months of age (#9). Two mothers also attended the session on
hygiene and diarrhea prevention (#11) with children that were 9 and 22 months, respectively, as
opposed to the recommended 10 months, but this is not so problematic, because the topic is more
general and can be useful for mothers of children of all ages.
The information on the children’s age by learning session number for the MCs for mothers of
malnourished children is presented for descriptive purposes, since the schedule for this category
is not age-specific. All children were between 6-59 months of age as expected.
Besides showing departures from the schedule, the results in Table 4.5 also show that the age of
the children attending the same MCs varied considerably, especially in the Clubs for mothers of
6-23-month-old children (e.g., ranging from 9 to 22 months for session #11, and from 5 to 16
months for session #12).
Thus, there are clearly some logistical difficulties in forming MCs that are homogenous in terms
of children’s age and/or the mother’s stage of pregnancy or lactation and in ensuring that all
women attend the session that is most relevant for them at a particular time. Coordination of this
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
60
aspect is particularly complex for the preventive group and will require additional attention from
the health staff and their supervisors.
4.3.2. Quality of education at the Mothers’ Clubs
Table 4.6 presents an assessment of the quality of the education provided at the MCs based on
the observations of the education sessions. This table focuses on key findings; additional details
can be found in Annex 5.
Table 4.6. Quality of education provided at Mothers’ Clubs
Numbers [N = 20]
Positive aspects
 Technical content
- 80-100 percent of technical information communicated was correct and complete 15
- No incorrect information communicated (n = 15) 8
 Session Management and organization
- Completed all session steps 19
 Facilitation and teaching skills
- Used open-ended questions to probe and encourage, in addition to open-ended questions
included in session guide
13
- Used other teaching techniques/facilitation skills as written in learning session guide 18
- Assured that all participants could see visual materials (n = 14) 13
Attitudes displayed
- Provided praise/affirmation to participants 19
- Attempted to create good dialogue and/or limit lecture style 9
Atmosphere at the learning session
- Did animations (songs, sketch) with the group, in addition to what is included in session
guide
10
Negative aspects
 Technical content
- Failed to bring back focus to promoted behavior 12
 Session Management and organization
- Failed to complete session in recommended time +/- 15 minutes 11
 Facilitation and teaching skills
- Failed to use small groups as suggested 4
 Atmosphere at the learning session
- Distracting side-conversations during the session 12
- Distracting environmental events 14
The observations suggest that most of the technical information communicated in the MCs was
correct and complete, and was consistent with the intended content of the learning sessions (on
average, 83 percent of the information was correct and complete). In 15 of the MCs observed,
between 80 and 100 percent of the technical information was presented correctly and was
complete; in 8 of them (approximately half), no incorrect information was communicated,
although the information presented was not always complete. The weakest point concerning the
technical content of the education was that about half of the health agents/colvols failed to bring
back the focus of the discussion to the behaviors being promoted when participants said
something incorrect, which means that participants in these cases may have been left with
incorrect information.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
61
The management of the sessions was generally good. Most of the observed health agents/colvols
completed all session steps and in the order indicated in the instructions. Most of them also had
no difficulties in reading the activities included in the learning session guide, i.e., they were able
to read Creole easily. However, two health agents/colvols read to the participants the specific
instructions included in the learning session, which are only meant for the health staff. Half of
the health staff had difficulties in completing the session in the recommended time +/- 15
minutes. A few went through the learning session faster than recommended, and thus spent too
little time interacting with the participants. Others spent up to one hour longer on the session,
which increased the time burden for the health staff and participants. This was especially the
case for the cooking sessions, for which the duration depends on the recipes prepared.
Concerning facilitation and teaching skills, we observed that nearly all of the health
agents/colvols used open-ended questions as suggested in the learning session guide. Thirteen of
them even asked other open-ended questions to probe and encourage participants. The large
majority (n = 18) also used the various communication methods recommended in the session
guides, such as role-plays, stories, games, and demonstrations, in addition to the presentations.
The two others tried to use them, but none of the participants actually understood what the health
agent/colvol was asking them. The use of small group discussions among participants, as
recommended in the materials, did not function very well. In four MCs, the small groups
approach was not used, although it was suggested in the learning session. Those who did use the
method seemed to experience various problems, such as difficulties in clearly defining the
question to be discussed, and logistical problems in arranging the participants to face each other
or in circulating around the room to help the groups in their discussions (see Annex 5 for details).
In general, the visual materials accompanying the learning sessions were used and the health
staff tried to ensure that all participants could see them by moving or passing them around.
The attitudes displayed during the MCs meetings were quite positive, especially when
considering that the health staff was used to a lecture-type approach before, as observed during
our previous formative research. Nearly all of them provided praise and affirmation to the
participants. However, demonstrating respect and helping participants to feel at ease when they
made interventions was less common. A real dialogue or interaction was established by eight of
the health agents/colvols observed. They were successful in returning questions from
participants to participants, in reacting to the participants’ comments, and in asking additional
open-ended questions.
In general, the atmosphere at the meetings was good. Half of the health staff did animations
(songs, sketches) with the group in addition to what was suggested in the learning session guide.
Although distracting side-conversations and environmental events (for example, a passing
motorbike, animals making noise, etc.) were reported for more than half of the MCs, the level of
distraction was definitely lower than at the RP education sessions. In half of the meetings,
laughter was observed during the session, which can be considered an indication that participants
felt at ease in the group.
These results are very encouraging and show a major improvement in the quality of education
provided in the MCs since the recent training in the new BCC strategy.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
62
4.3.3. Recall and use of the information discussed at the Mothers’ Clubs
This section summarizes the findings on the recall and use of the information discussed in the
MC education sessions by participating women. Data from 30 home interviews are used. The
home interview respondents were identified at 15 MC meetings and were interviewed
approximately 1-2 weeks after the meeting.
In general, respondents considered that the information provided at the MCs was useful. They
were asked to recall as many of the details as they could concerning what they had heard and
learnt at the MCs. They were also asked which of the recommended behaviors they were able to
adopt or try. The recall information was asked specifically with reference to the last MC
meeting they had attended, whereas information on the application and adoption of the
recommended practices was obtained for all sessions that the women had attended up to the
interview date. We first present the recall and adoption of practices in relation to the topic
addressed at the last MC meeting that each respondent had attended (see Table 4.7). The left￾hand-side column lists the topics that are addressed in the different sessions, as a reference.
Then, all recommended practices that respondents reported adopting since they had started
attending MCs are presented (see Table 4.8).
4.3.3.1. Sessions #1-4—Breastfeeding practices
The results in Table 4.7 indicate that all 12 home interview respondents spontaneously recalled
the message regarding exclusive breastfeeding up to 6 months. Two-thirds of the respondents
recalled the topic of breast milk expression and some of them also talked about how to store the
expressed breast milk. The importance of feeding colostrum to the child and the risks of bottle￾feeding were less likely to be recalled. Hardly anyone spontaneously recalled issues related to
breastfeeding frequency and duration of feedings, good positioning of the child during feeding,
or when to introduce complementary foods.
Six of the 7 lactating respondents reported that they were currently practicing exclusive
breastfeeding and that they were planning to do so until their child reached 6 months of age. For
all except one, this was their first experience with exclusive breastfeeding. Also, all 5 pregnant
respondents expressed their intention to exclusively breastfeed their infant. About half of the
lactating respondents had expressed breast milk several times since the last MC meeting and left
it for the child when they were away from home. This is a very promising result, given the fact
that our previous formative research had revealed that many women had heard about expressing
breast milk, but in most cases had not tried it. Three respondents reported drinking water while
breastfeeding at every feed as a result of having heard about it at the last MC they had attended.
Although this recommendation was presented mainly as a practice to avoid dehydration and
fatigue, respondents explained that they did it because it increased their milk production.
4.3.3.2. Session #5—Lactational Amenorrhea Method (LAM)
The two respondents who had attended the session on the lactational amenorrhea birth control
method recalled spontaneously the objective of this method (birth spacing), and one of them
remembered two of the three requirements: exclusive breastfeeding and infant less than 6
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
63
months old. None of them stated the third requirement correctly (i.e., no monthly bleeding since
birth). The reported application of the method was as incomplete as the recall. One respondent
reported that she was practicing LAM, although she talked only about one of the three
requirements. The second respondent said that she could practice LAM as long as her monthly
bleeding was not back and she breastfed exclusively (her child was already 7 months old on the
day of the interview). These results suggest that the concept of LAM and the need to meet all
three requirements was not fully understood by the respondents.
Table 4.7. Spontaneous recall and application of advice received at Mothers’ Club
education
Topics covered in sessions
Spontaneous recall of
details
Numbers who reported
adopting
Sessions #1-4 (n = 7 lactating and n = 5 pregnant women)
 Early initiation of BF
 Importance of colostrums 4
 Exclusive breastfeeding up to 6 months 12
3 (EBF protects from
pregnancy [LAM])
6 (Exclusive BF up to 6 months)
5 (pregnant women say they will
do EBF up to 6 months)
 Continued breastfeeding up to 2 years
 Risks of bottle feeding 2 (for diarrhea)
 Local beliefs and practices, compared to recommended
practices
 Good breastfeeding techniques, frequent, on demand
breastfeeding, care of nipples and breasts
 Expression of breast milk, storage of expressed milk 8
3
3
 Importance of mother drinking water while BF 3
 Introduction of foods at 6 months
Session #5 (n = 2)
 LAM method for longer spacing between births:
- EBF frequently
- No monthly bleeding since birth
- Infant < 6 months
2 (longer spacing between
children)
1 EBF up to 6 months
1 (infant < 6 months)
1 (practice of LAM, i.e., EBF up
to 6 months)
1 (practice of LAM, i.e., EBF up
to 6 months; and no monthly
bleeding)
Session #8 (n = 10)
 Enriched complementary foods/recipes/meals (recipes,
etc.)
10 (recipes for enriched
meals)
 Role of different ingredients 6
 Feasibility of preparing enriched recipes/meals
7 (gruel with egg or milk)
2 (mashed plantain with pumpkin
and fish sauce)
6 (feeding of other rich foods or
meals)
 Food texture 7 (texture of gruel)
 Feeding frequency
 Serving sizes
 Importance of encouraging child to eat 3 2
Session #9-12 (n = 6)
 Types of foods to introduce in child’s diet after 6
months of age
2
3 (EBF until 6 months) 1 (EBF up to 6 months)
 Child development steps 4
 Learning to eat (food texture, breastfeeding and
feeding)
 Feeding frequency 2
 Serving size
 Helping children to eat 2
 Helping children to eat well when ill, convalescent
 Importance of food variety and enriched meals 5 (feeding nutrient-rich foods:
leaves, beans, lemon juice, eggs)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
64
Topics covered in sessions
Spontaneous recall of
details
Numbers who reported
adopting
1 gruel with egg or milk
2 mashed plantain with pumpkin
and fish sauce
 Feeding children beyond 12 months of age:
- enriched meals
- vitamin A-rich foods
- importance of evening meal
4.3.3.3. Session #8—Cooking and tasting of enriched meals
The cooking and tasting session is scheduled for mothers with children 6-7 months old to
introduce the concept of enriched complementary foods and to provide them with concrete
recipes. Mothers prepare and taste at least two different recipes in groups (enriched gruel and
meal). In addition, the feasibility of preparing these recipes and other enriched foods or meals at
home is discussed and information on food texture, feeding frequency, and serving sizes is
provided by the health staff.
The results in Table 4.7 show that all 10 home interview respondents remembered the enriched
recipes that were prepared, although these recipes were new for all of them. About half of the
respondents also stated the roles of the different ingredients and recalled that they were
instructed to prepare a thicker gruel than the one they had usually made in order to fill the child’s
stomach with nutritious food rather than water only. In addition, one-third of the respondents
remembered that it is important to encourage the child to eat. However, most of the respondents
did not spontaneously talk about practices related to feeding frequency, serving sizes, and how a
child has to learn to eat.
The results of immediate application of the improved recipes at home are encouraging. Seven
respondents reported having prepared enriched gruel—either adding an egg or milk—at least
once since the last MC meeting. Some had prepared it as often as every day. Two respondents
of the four who had prepared the mashed plantain recipe with added pumpkin and dried fish
sauce in the last club meeting also reported having tried the recipe twice at home. Three had
prepared other rich foods or meals like rice with bean sauce and vegetables, vegetables with
dried fish and leaves, bread soup with added leaves and dried fish, bean puree, egg sauce, or
fruits a few times since the last meeting and for some, up to 2-3 times per week.
4.3.3.4. Sessions #9, 10 and 12—Complementary feeding practices
These sessions concentrate on issues related to complementary feeding practices for different age
groups: 6-8 months, 9-11 months, and beyond 12 months of age, and are scheduled for mothers
of children older than 6 months. Important information is provided on child development and
learning to eat (food texture, breastfeeding, and feeding). The sessions also cover feeding
frequency, serving size, and how to help children eat well.
The results show that more than half of the respondents recalled different child development
steps that were presented in the chart accompanying the learning sessions. Half of the
respondents remembered the importance of exclusive breastfeeding up to 6 months, although the
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
65
three sessions (#10-12) focus on practices for children older than 6 months. One-third of the
respondents talked about the frequency of feeding for children 6-8 months old. They recalled
that children in this age group should have 2 meals and 1 additional snack or eat 3 times per day.
Two respondents mentioned that they learned what types of foods to introduce at 6 months, and
two mothers remembered that it is important to encourage the children to eat. None of the
respondents talked spontaneously about feeding during and after illness, serving sizes, food
texture, food variety issues and vitamin A-rich foods, or of the importance of an evening meal.
Five respondents reported that they had fed their child rich foods such as leaves, bean sauce,
eggs, or lemon juice several times per week, and/or enriched meals such as enriched gruels
nearly daily, or mashed plantain with added pumpkin and dried fish sauce 2-3 times since the last
MC meeting. One respondent stated that she practiced exclusive breastfeeding for 6 months with
her last child, who was, at the time of the interview, 8 months old.
Table 4.8 summarizes all the recommended practices that women reported adopting since they
had started attending the MCs. Pregnant respondents were excluded from this analysis because
they had not yet had a chance to practice any of the recommendations related to infant and child
feeding and care by the time of the interviews. The practices are grouped according to the
following specific themes: (1) feeding of individual recommended foods, (2) feeding of
improved meals, (3) adding individual ingredients to enrich meals, (4) food selection, (5)
psychosocial aspects of feeding, (6) breastfeeding, and (7) hygiene and diarrhea-related
practices.
The results show that the practices most commonly reported as having been adopted were related
to feeding richer meals, especially enriched gruels or nutrient-rich foods, and to breastfeeding. A
few respondents referred to some of the psychosocial aspects of feeding, including
encouragement to eat and feeding style. The results are positive and suggest that the process of
behavior change toward the adoption of recommended child feeding practices has commenced.
This is a particularly encouraging finding, given that the full BCC strategy at the MCs had been
implemented only a few months before the interviews were carried out.
Table 4.8 also reports on some of the constraints that respondents expressed, which explained
why they were unable to adopt some of the recommendations received at the MCs. Four of the
eight respondents who reported constraints indicated that they were not able to provide nutrient￾enriched foods and meals to their child on a regular basis due to economic constraints. The
remaining four reported constraints related to the expression of breast milk. One of them faced
difficulties with the family after trying it once; the others were too concerned about the risk of
contamination of the expressed breast milk when they had to leave it behind for the child to
consume when they were away from home. It will be important to continue to focus on
assessing the constraints to using recommendations in the second round of operations research to
be carried out in 2004. By that time mothers will have had even greater exposure to the
messages and will have had a chance to ascertain the feasibility of adopting the recommended
practices and using them on a regular basis over time.
The results on recall and use of practices recommended in the MC education sessions also
provide hints for the program, such as the issues that the health staff need to emphasize more in
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
66
the learning sessions. It also helps identify the topics that should be revisited with the health
staff in their re-training in order to ensure that the health agents/colvols are fully comfortable
with these topics. In general, there was very little spontaneous recall from the part of the
mothers regarding certain recommendations related to complementary feeding practices, namely
food texture, feeding frequency and serving sizes for different age groups, and food variety. It is
not clear whether this is because the health staff did not sufficiently emphasize these aspects, or
whether mothers did not grasp them as readily as other simpler messages. Finally, the
constraints stated by some respondents in relation to adopting the feeding recommendations
emphasize the need for the program management to think about supporting program activities
that increase resource availability within households in order to support translation of knowledge
into behaviors and improved child outcomes.
Table 4.8. Application of advice received at Mothers’ Club education and reasons for not
using
Numbers
Application of recommended practices (n = 25)
 Feeding of individual recommended foods:
- Rich foods such as leaves, beans, lemon juice, fruits, eggs 8
 Feeding of improved meal composition:
- Rich meals such as vegetables/bread soup with dried fish and leaves 5
 Adding ingredients to enrich meals:
- Adding pumpkin to mashed plantain with dried fish sauce 4
- Adding an egg or milk to a gruel 8
 Food selection:
- Feeding of three groups of food 1
 Helping children to eat – the psychosocial aspects:
- Encourage child to eat 2
- Feeding of foods/meals that child likes 1
- Feeding of food for child in a separate dish 1
 Breastfeeding:
- Feeding colostrums 1
- Exclusive breastfeeding up to 6 months 9
- Expression of breast milk 4
- Drinking water while breastfeeding 3
- Good positioning of the child 1
- Wash breasts before feeding 2
- How to treat breast pain 1
- Continue breastfeeding up to 2 years 1
 Hygiene and diarrhea related aspects:
- Preparation of ORS when child has diarrhea 4
- Treatment of drinking water 3
- Washing hands before eating 1
Recommendations not practiced and reasons
 Feeding practices:
- Feeding of rich foods, adding of ingredients to enrich meals not practiced because of no
money to buy ingredients
- Feed frequently, no money
- Feed liver, no money
- Nothing practiced, no money
1
1
1
1
 Breastfeeding practices:
- Expression of breast milk, not accepted by husband (no time to feed expressed breast
milk, don’t accept expression) and not accepted by other family members (bad smell of
expressed breast milk)
- Expression of breast milk, risk of contamination with microbes
1
3
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
67
4.3.4. Overall comprehension of new concepts discussed in Mothers’ Clubs
In addition to the analysis of the recall information and adoption of recommended practices, we
also assessed the overall comprehension of the new concepts discussed at the MCs by home
interview respondents. This assessment was done after all the above-presented data were
analyzed and once we had a good understanding of the data obtained from the home interviews.
We used the approach of subjective rating with the data from the spontaneous and probed recalls
and the information on the adoption of recommended practices for each respondent.
Specifically, we examined the data to evaluate the consistency in the recall (spontaneous and
probed) and the application of the topics discussed at the MC. We then classified respondents
into the following four categories of overall comprehension: (1) very good, (2) upper middle, (3)
lower middle and (4) very poor. Subjective ratings are commonly used in clinical research; for
instance, all clinical judgments regarding how well patients respond to a treatment are subjective
ratings.
The results of the subjective rating exercise are presented in Table 4.9. One-third of the
respondents had a very good overall comprehension, i.e., they recalled information correctly and
were consistent in what they recalled and then reported translating information into practice.
Very few respondents had a very poor comprehension. Most respondents fell in the middle
categories of “upper” and “lower” middle.
Table 4.9. Overall comprehension of new concepts discussed in Mothers’ Club
Categories (n = 30) Numbers
Very good 10
Upper middle 8
Lower middle 9
Very poor 3
In order to illustrate the meaning of the subjective rating exercise, Table 4.10 shows one example
of a respondent rated as having very good overall comprehension and another one rated as
having very poor comprehension. The two respondents had attended MCs covering different
topics; the respondent with very good comprehension had attended a session on breastfeeding
(session #3), whereas the one with very poor comprehension had attended a session on LAM
(session #5).
It is clear from the first example that in addition to having a more accurate recall of the details of
the content of the messages, the respondent who was ranked as having very good comprehension
was more consistent in the behaviors/practices that she recalled and those that she applied. The
respondent who was ranked by the team as having very poor comprehension not only performed
relatively poorly on the spontaneous recall, but also reported applying very few of the
recommendations from the MC meeting (second example).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
68
Table 4.10. Examples of a very good and a very poor overall comprehension
Example of very good overall comprehension
Session #3 – How to breastfeed better
The individual: 2 children, youngest is 2 months old
Spontaneous recall
- Importance of exclusive breastfeeding up to 6 months;
- Expression of breast milk if the mother has to leave the house for some time;
- Storage of expressed breast milk in a clean cup, to be kept in a cool place next to an earthenware vessel,
so breast milk can keep for 8 hours without spoiling;
- She did a demonstration of how to express breast milk in front of the others;
Probed recall
- Good probed recall, complemented with many correct comments by mother, e.g., exclusive
breastfeeding so that child gets sick less frequently, good attachment of child (baby’s mouth wide open),
explains how to store expressed breast milk, drinking water while breastfeeding to avoid fatigue;
Application
- Breastfed exclusively so that child develops well and to prevent getting her monthly bleeding too early;
- Drank water while breastfeeding every time she feeds to avoid fatigue and to have more milk;
- Expressed breast milk so that child has something when s/he is hungry during her absence and helps to
avoid giving other foods than breast milk; she demonstrated herself the expression of breast milk at the
MC, but has not yet practiced it at home since she did not leave the house.
Example of very poor overall comprehension
Session #5 – LAM
The individual: 4 children, youngest is 7 months old
Spontaneous recall
- In the first place, she did not remember the topic of the last MC meeting; when asked about specific
ideas or pieces of information that she remembered of the last MC meeting, she talked about the
following:
- Family planning, birth spacing to have less children that could be sent to school and could get
medical treatment if sick;
- Wrong recall of requirements: monthly bleeding necessary for LAM;
Probed recall
- Good probed recall, but no further comments from the mother herself;
Application
- Could practice LAM (if: no monthly bleeding, feeding only breast milk, third requirement not
mentioned), helps child to be healthy;
- Breastfed exclusively during 6 months;
- Prepared wheat gruel and bread soup two times per month;
- Fed mangos two times and bananas every time she goes to market.
The women rated as having very good overall comprehension were, on average, slightly younger
and had slightly less children than the women with very poor overall comprehension. The
average number of MCs attended or the location where they live, however, was unrelated to their
overall comprehension.
It is important to note that the ranking into the different categories took into consideration that
the implementation of the BCC component had started only a few months before the interviews
were done. Therefore, complete acquisition of knowledge and translation into practices was not
expected. The very good overall comprehension level is relative to the other observed
comprehension levels in the sample. It was surprisingly difficult to identify respondents who
had very poor overall comprehension. The majority of them had actually done well in terms of
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
69
recall and application, in spite of the fact that the restructured MCs had been implemented only
recently.
4.4. Staff perceptions about the Mothers’ Clubs
Information about perceptions and opinions of the health staff regarding the MCs was obtained
in individual interviews with 20 health agents (one per MC) and in five focus group discussions
with health agents, colvols, and MCH supervisors. This section summarizes the results on their
general perceptions about the MCs, the types of problems they face, their experiences with
participants bringing their ration cards to the MC meetings, and finally, their suggestions on how
to improve the MCs.
4.4.1. General perception of the Mothers’ Clubs
All health agents interviewed felt that the MCs were working well in general. Thirteen
expressed that the MCs provide a forum for mothers to exchange ideas, to receive important
information, and to learn very useful things. They also felt that the MCs helped change mothers’
behaviors in a positive direction, and that mothers who attended the Clubs were in a position to
take better care of their children, to avoid certain illnesses, and to fight malnutrition. Five health
agents expressed that the MCs are the ideal venue of the MCH program for behavior change
communication. A few health agents mentioned that mothers liked the clubs and that they were
interested and motivated to participate. Only two health agents expressed that more efforts were
still necessary to improve the functioning of the MCs.
The discussants of both focus groups with colvols confirmed the health agents’ points of view, as
they, too, expressed that the MCs were working well and that mothers liked the clubs. The
important contribution of MCs to positive behavior change toward recommended child feeding
practices was raised in the discussions in one of the two focus groups with colvols.
Participants in the MCH supervisors focus group, however, felt that mothers participate in the
MCs only in order to get their ration cards signed and to be eligible to receive donated foods.
They also expressed that the MCs are the basis of the MCH program because they are an ideal
forum to initiate behavior change.
In the four focus groups with health agents and colvols, there was also discussion about how the
organization of the MCs had changed compared to the previous year. The following changes
were mentioned by the discussants:
ƒ Women and mothers were regrouped into different MCs according to their situation:
pregnant women, lactating mothers, and mothers of children (6-23 months old or 6-59
months old and malnourished);
ƒ Subgroups for MCs were established in order to address the session topics most relevant to
the mothers’ situation;
ƒ Communication materials including visual tools were developed;
ƒ The MCs were relocated in order to bring them closer to the women’s homes;
ƒ The content of the MCs was made more applied for mothers, less theoretical;
ƒ There is more supervision now than the previous year.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
70
In two individual interviews, the health agents expressed the view that they now manage to
establish a dialogue with mothers instead of holding monologues and that mothers participate
more in the discussions in the MC meetings than they did in the previous year.
4.4.2. Problems encountered in facilitating the Mothers’ Clubs
When asked about the problems they faced with the MCs, the majority of health agents
complained about late arrival of the participants and absenteeism. Other difficulties that 12
health agents mentioned were poor venues for the MCs and/or the lack of sufficient chairs and
benches for all participants. Six health agents mentioned low active participation of women
during the sessions and especially at the beginning of a new club. These difficulties were also
voiced by the discussants in the focus groups with colvols and MCH supervisors. Other
problems mentioned less frequently included the following: (1) lack of availability of transport
facilities to reach the MCs in remote areas; (2) lack of time to prepare the learning sessions for
the MCs; (3) difficulties of fixing the child development and feeding chart during the meeting;
(4) the small number of participants in some MCs; and (5) bad timing of meetings (market day,
at noon) or having several MC meetings per day. Finally, the staff also discussed how they dealt
with women who did not bring their ration cards to the MC (since they were required to mark
attendance on these cards in order for beneficiaries to receive food rations).
Dealing with late arrivals and absenteeism. The health agents voiced three consequences if
women arrive late: (1) the club meeting starts late if the health agent waits for them; (2) these
women miss certain steps of the learning session if they arrive after the start of the club; and (3)
the other participants are disturbed. Three health agents talked about potential strategies to use
to encourage women to arrive on time: either to threaten the participants arriving late with
considering them as absent so that they lose their eligibility to receive the food for that month, or
to refuse to sign their ration cards so that they cannot receive the food. The problem of late
arrivals was also voiced in the focus groups with colvols and the MCH supervisors. The most
frequent strategies used to deal with absent participants are to find out the reason for the absence
during a home visit and to encourage women to participate the next month, and to not sign their
ration cards. Four health agents reported threatening absent participants with not receiving food
if the absence is not motivated by illness, and another three reported that they made sure that
participants who were absent for other reasons than illness did not receive their food ration. One
health agent indicated that s/he checks with the supervisor to decide what to do about the
participants who miss the MCs.
Bringing ration cards to Mothers’ Clubs. The findings from the individual interviews with
health agents and the three focus group discussions with colvols and MCH supervisors indicate
that beneficiary women sometimes forgot to bring their ration card to the MC meetings. Only
one health agent indicated that women always had the ration card with them and one indicated
that the cards were not yet distributed in the communities (localités) in which s/he worked.
According to the health staff, the most frequent strategy used to deal with forgotten ration cards
is to ask the women to bring the card to the next RP meeting or to bring it later to the health
agent. Two health agents reported asking women to return home to get their card, or in the case
of beneficiaries living far away. One health agent keeps the cards for the women and gives them
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
71
back to them at the RP, before the next food distribution. Two other health agents simply do not
sign the ration cards if mothers do not bring them to the Mother’s Club.
4.4.3. Suggestions on how to improve the Mothers’ Clubs
Most of the suggestions made by the health agents were in direct relation to the problems
identified earlier. In order of priority, they recommended that program management take the
following actions:
ƒ Offer more training in general and specifically, regular retraining concerning the infant and
child feeding module.
ƒ Understand that the health staff need transport facilities (horse/mule) in order to efficiently
carry out their work and to hold MCs in remote areas.
ƒ Understand that the health staff need time to constantly encourage and motivate the
beneficiary women to attend the MCs and to arrive on time at the meeting.
ƒ Provide special and better venues for the MCs meetings equipped with chairs and benches, in
order to set up the group in a circle to facilitate interaction, and with something to hold up the
child development and feeding chart.
ƒ Hire more colvols to assist them in their work.
ƒ Provide regular supervision of the MCs in order to help motivate participants.
ƒ Review the calendars for the MC meetings with participants to take their schedules into
consideration.
ƒ Help them to revise the MC calendars regularly, at least each time when new MCs for
pregnant women are formed (every four months).
4.5. Beneficiary perceptions about Mothers’ Clubs
Information about the beneficiaries’ perceptions of the Mothers’ Clubs was obtained in home
interviews (n = 30). This section summarizes the results on the perceived importance of
Mothers’ Clubs and their ease of use.
4.5.1. Perceived importance of Mothers Clubs
The MCs were well appreciated by the home interview respondents. They felt that the MCs
were important for their children’s health. Some of them considered this service as more
important than all the other WV services they used, although many had difficulties ranking the
different services according to their importance because they felt that all services were important.
One respondent indicated that MC attendance was important in order to receive the food ration.
The majority of respondents felt that attendance at the MCs improved their knowledge about
infant and child feeding and about how to take care of their children. The examples that they
provided to illustrate what they had learned included the following: (1) exclusive breastfeeding
up to 6 months, (2) expression of breast milk when they leave the house and storage in a cold
place, (3) drinking water during breastfeeding to avoid fatigue, (4) ways to encourage children to
eat when they have no appetite, (5) what foods to introduce at 6 months, and (6) how to prepare
special meals for children.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
72
4.5.2. Ease of use
Nearly two-thirds of the home interview respondents (n = 18) felt that the MC was the easiest
service to use among the different WV activities they attended. The reasons stated for the
perceived convenience of the MCs were related to distance from home and time spent at the
club. The respondents voiced that the MCs were very close to their homes and that they usually
did not last longer than one hour, since it was a group meeting that addressed everybody at the
same time. Only the cooking session appeared to be more time-consuming, or when mothers
arrived late at the meeting. They also mentioned that they still had time for other activities on
the day of the MCs, either before and after the meeting. However, one respondent expressed that
it was difficult for her to use the MCs because she needed to arrange childcare in order to be able
to attend.
4.6. Summary of findings regarding operations at the Mothers’ Clubs
4.6.1. Organization and operations of the Mothers’ Clubs
Organization of the MCs and attendance. The MCs are meant to gather small groups of 15-20
participants. The number of registered participants attending the 20 MCs observed was within
this range, but a number of other people—children, fathers, or other relatives—were also present
at most sessions, raising the average number of participants to 18 and the upper end of the range
to 41 participants. Although it is difficult to restrict participation only to registered beneficiaries,
some advice may need to be provided to health agents/colvols on how to ensure that the size of
the MCs remains manageable.
The time involved in attending or facilitating an MC is also of concern. Although the learning
sessions lasted, on average, one hour, the time involvement for participants and health staff was
much longer, mostly because of delays in starting the session, while waiting for all participants
to join.
Adherence to the MC categories and planned schedule of learning sessions. In general, the MCs
for pregnant and lactating women respectively included the right categories of women, except for
a few mothers in the lactating groups who had children older than the maximum of 6 months
recommended by the program. The schedule of learning sessions for these MCs was also
generally as recommended. Many of the sessions targeted to mothers of children < 24 months in
the preventive group, however, did not follow the recommended schedule of learning sessions
for children of different ages, and quite a few mothers received the information at the wrong
time. Our results suggest that there are still some logistical difficulties in forming MCs that are
homogeneous, especially with respect to the small age ranges recommended for the MCs in the
preventive program group. It is important to note, however, that this new system had been
implemented only a few months prior to our research, and therefore may still have been in a
transition phase.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
73
4.6.2. Quality of education at the Mothers’ Clubs and recall and use of information
Quality of education at the MCs. The quality of education at the MCs observed was very good,
especially considering that training in the use of the new materials and adult education
techniques had been completed only a few months before our research. The technical content of
the education was correct and complete in a large majority of the MCs observed and the
management and organization of the sessions were generally good. The health staff’s facilitation
and teaching skills were also generally good; they used open-ended questions to probe and
encourage the participants, and approximately one-half of them managed to establish a real
dialogue and interaction with the group. These results are very encouraging and show a major
improvement compared to the situation observed before the training carried out in 2003. Clearly,
a behavior change process at the level of the health staff itself has commenced.
Recall and use of information discussed at the MCs. Overall, the mothers’ recall of the sessions
they had attended 1-2 weeks prior to the interviews was very good. Most could recall the main
topic of the session and at least two or three more specific concepts related to the main topic.
The rate of reported adoption of the recommended practices was also quite high, especially for
exclusive breastfeeding and the use of enriched gruels for complementary feeding of infants and
young children.
Results of the subjective rating approach also indicated that one-third of the respondents had
very good comprehension—i.e., they recalled information correctly and were consistent in what
they recalled and then reported translating into practice. Very few women had very poor
comprehension (n = 3/30), and the remaining women were in the middle two categories (upper
and lower middle). Thus overall, it appears that the learning sessions were quite effective in
transferring knowledge to women and also in initiating a process of behavior change, at least in
some of the participants.
It is important to highlight that the BCC strategy had only recently been implemented and that
the findings may be a reflection of a great momentum among the health staff about what they
had recently learned and were eager to put into practice. The same is true for the participating
women who may have been particularly enthusiastic to try some of the new recipes or other
practices they have just learned at the MCs. It will be important to reassess in the next round of
operations research, to be carried out in 2004, whether this momentum could be sustained and
whether the quality of the intervention and its impact on behavior change were maintained.
4.6.3. Staff perceptions about the Mothers’ Clubs
General perceptions. The health staff felt that the MCs were working well in general and the
colvols also indicated that participating mothers liked the clubs. Health agents, colvols, and the
MCH supervisors all referred to the MCs as an important forum for mothers to exchange ideas,
receive important information, and acquire skills that will help them take better care of their
children. There was also a general sense that MCs had improved since the previous year, both in
relation to organizational aspects as well as the content of the lessons, the availability of
communication materials, and the supervision.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
74
Perceived problems. The most common problems reported were the late arrival of participants
and absenteeism, and strategies to address these problems were discussed. Other problems
indicated by the health agents were the poor venues of the MCs and the lack of appropriate
seating arrangement for the participants, and the problem with mothers who forget their ration
cards.
Suggestions on how to improve the MCs. Again, most of the suggestions made by the health
agents were in direct relation to the problems identified. Their recommendations focused on the
following aspects: (1) staff training: ensure retraining, monitoring, and supervision to maintain
the quality; (2) resources: provide more suitable and better-equipped venues and more colvols to
assist them in their work; (3) time and workload: provide transport; (4) calendars: improve the
scheduling of the MCs.
4.6.4. Beneficiary perceptions about the Mothers’ Clubs
The MCs were well appreciated by the respondents, who indicated that the clubs were important
for their children’s health, and that their attendance at the clubs improved their knowledge about
infant and child feeding and care. Nearly two-thirds of the home interview respondents felt that
the MCs were the easiest service to use among the different WV activities that they attended,
especially because they were relatively close to their homes and not so time consuming.
4.6.5. Conclusions on operations at the Mothers’ Clubs
Overall, our results suggest that the MCs are appreciated by both the staff and the beneficiaries
and are perceived to be the least time consuming requirement of the program for beneficiaries.
MCs are also perceived as being a key element in the package of services offered by the program
and as being the primary venue for the education and BCC strategy. Ensuring the intended
composition of the clubs (especially in the preventive group) is still a challenge and will
probably require continued supervision and retraining of the staff. The same is true for
sustaining the quality of education over time and maintaining the motivation of both health staff
and beneficiaries.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
75
5. PROGRAM OPERATIONS AT FOOD DISTRIBUTION POINTS
This section is structured as the previous two. It presents a brief description of planned
operations at the Food Distribution Points (FDPs) and of the methods used to assess operations.
The findings related to operations and quality of service delivery are presented next, followed by
a summary of the perceptions of field program staff and beneficiaries regarding this service
delivery point and their suggestions for improvements.
5.1. Description of operations at Food Distribution Points according to
implementation plan
The distribution of food aid commodities to the MCH beneficiaries of the WV program occurs
on a monthly basis at special distribution points. Beneficiaries from several communities
(localités) are scheduled to receive their food rations at a central distribution point on the same
day. The distribution is done by community and starts with beneficiaries from the communities
that are far away from the distribution point. Within each community food rations are provided
first to lactating and pregnant women and then to caregivers of child beneficiaries.
There are a total of 10 central FDPs covering 50 RPs in the area included in the research
(including preventive and recuperative program communities). Unlike at the MCs, a beneficiary
can designate another family member to collect the food ration by handing over the beneficiary
card to this person.
The beneficiary households receive both direct and indirect rations. The amounts and
commodities vary with respect to the beneficiary category (see Table 5.1). Even if a household
has two direct beneficiaries participating in the program, only one indirect ration is provided.
Table 5.1. Composition of direct and indirect food rations, per beneficiary category
Children 6-23 months of age (preventive
model)
Undernourished children 6-59 months of
age (recuperative model)
Pregnant and lactating women
(both models)
Type of commodity
Direct child ration
(kg)
Indirect child ration
(kg)
Direct women ration
(kg)
Indirect women ration
(kg)
WSB 8
SFB 10 5 5
Lentils 2.5 2 2
Vegetable oil 2 1.5 1.5
The sequence of activities at the FDPs is as follows:
ƒ Eligibility of the beneficiary is verified by food monitors and health agents mainly based on
the information on the beneficiary card. This card contains information about the
beneficiaries (direct and indirect) and indicates attendance by the beneficiary at the other
MCH activities (i.e., Mothers’ Clubs, Rally Posts, and pre- and postnatal consultations) that
are required in order to receive the food rations.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
76
ƒ Once their eligibility is verified, the beneficiaries proceed to collect their rations. A team of
trained beneficiaries assists the WV staff during the distribution and is responsible for
opening food sacks, measuring out appropriate amounts of each of the commodities, and
handing over the food to beneficiaries.
ƒ Finally, once the beneficiary (or designee) has received his/her rations, the food monitor
verifies the rations received and the beneficiary card. In some cases, the food monitors
reweigh the rations to verify that the right amounts have been given to the beneficiary. Once
this final check is complete, the food monitor signs the beneficiary card to indicate that the
correct ration has been delivered to that beneficiary.
The activities at the FDPs are identical for both programs, and beneficiaries from both program
groups can attend at the same FDP.
5.2. Methods used to assess operations at the Food Distribution Points
The data that provide information on the functioning of the FDPs in the next sections were
gathered using the following methods:
ƒ Observation of the general organization and ambience at the FDPs and of the quality of food
commodities distributed (N = 10);
ƒ Structured exit interviews with randomly selected caregivers who had a food aid beneficiary
child, as they were about to leave the FDP (N = 45);
ƒ Individual semi-structured interviews with health agents at the end of the observed FDPs
(N = 20);
ƒ Semi-structured interviews with beneficiary women in their homes (N = 30);
ƒ Focus group discussions with colvols, MCH supervisors, food monitors, and commodity
supervisors (N = 5).
5.3. Operations and quality of services at the Food Distribution Points
This section presents a picture of the general operations of FDPs as compared to the
implementation plan. It also provides a brief description of the characteristics of the
beneficiaries and of the quality of the services provided, e.g., the timeliness of the services and
the quality of the food commodities. As in previous sections, the results are presented by topic
rather than by research method.
5.3.1. Attendance and beneficiary/staff ratio
5.3.1.1. Beneficiary attendance and characteristics
Beneficiaries are considered eligible to receive the food donations for the current month only if
they have attended the mandatory MCH activities for their beneficiary category in the previous
month. Table 5.2 shows that, on average, the FDPs observed hosted approximately 240
beneficiaries on the day of observation. The variability among FDPs is wide, with as many as
375 at one FDP and as few as 117 at another. On average, approximately 16 beneficiaries per
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
77
FDP were absent on the day of observation and did not pick up their food ration, although they
had met all the requirements.
Table 5.2. Number of beneficiaries at the FDPs
Number of beneficiaries
Variable
name
Mean
[N=10 FDPs] Range
Number of beneficiaries planned to receive food,
according to beneficiary list
Number of beneficiaries who actually received food
q406
q406a
256
241
122-396
117-375
Number of absent beneficiaries absentbenef 16 2-45
Table 5.3 presents data on the characteristics of the food aid beneficiaries. These data are
derived from the exit interviews at the FDPs (N = 45). The data show that it was primarily
mothers of the beneficiary children (91 percent) who came to pick up the food rations; in three
cases, fathers accompanied a beneficiary child, and in one case, another family member attended.
An examination of the ration cards showed that all beneficiary children who were in the
preventive beneficiary category according to their ration card were, in fact, in the correct age
range for that category and also lived in an area where the preventive program operates.
According to the birth date information in the ration card, the beneficiary children were between
8 and 23 months old (see Table 5.3). Similarly, beneficiary children in the recuperative
beneficiary category were also correctly identified for their group, i.e., they had been identified
as malnourished at a Rally Post and they were under 5 years of age.
However, 3 of the 22 children identified as belonging to the recuperative program did not
actually live in any area where the recuperative program operates. They were, in fact, from
preventive program communities, but had “malnourished child under 5 years” marked in their
ration cards. Two of these 3 misclassified children were less than 23 months of age (18 and 20
months) and therefore they were in the targeted age range for the preventive program group (and
thus eligible to receive the food). It is likely that the misclassification in these two cases was a
mistake made when the information was entered in the card. The other misclassified child was
malnourished (M3) and 25 months old, thus also eligible for food rations in the preventive
program group.
12 In this case, the health staff had indicated that the child belonged to the
category “malnourished child under 5 years” because the child was no longer in the 6-23 month
age range of the beneficiaries targeted by the preventive program group. A more accurate
labeling for this child would be “malnourished (M3) child 24-60 months of age.” In spite of
these slight inaccuracies in the beneficiary cards, all 45 children included in our exit interviews
were eligible to receive the food ration on the day of the interview.
According to the beneficiary ration card, the majority of exit interview respondents had one
direct beneficiary (i.e., one child in the food aid program), and, on average, four indirect
beneficiaries. Some beneficiaries had up to nine other family members, but the program still
provides only one indirect ration for the household, irrespective of household size. Four of the
households from the exit interviews had two children in the program (i.e., direct beneficiaries);
these families then receive two direct and one indirect ration.
 
12 Severely malnourished children between 24 and 59 months of age in the preventive program are also eligible to receive food
rations for 9 months.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
78
Table 5.3. Characteristics of beneficiaries at the FDPs
Characteristics of beneficiaries
Variable
name
Mean
[N = 45
beneficiaries] Range
Average age of children in each category, according to
beneficiary ration card (n = 43):
- 6-23 months old (n = 21)
- Malnourished child < 5 years (n = 22)
agembrc&
q509
15
29
8-23
15-52
Number of other family members in the household q512 4 1-9
Average duration of households’ participation in the food
distribution program (months), according to respondent (n = 44)
durpar11/
durpar22
7 1-17
Numbers
Relationship to the child:
- Mother
- Father
- Other
q501
41
3
1
Category of beneficiary child:
- 6-23 months old
- Malnourished child under 5 years
q509
23
22
Number of direct beneficiaries:
- One
- Two
q511
41
4
Duration of participation in the food distribution program,
according to respondent (n = 44)
- < 3 months
- 3-8 months
- 9-11 months
- ≥ 12 months
durpart1cat
eg
5
28
6
5
About half of the households from the exit interviews had been receiving food from the WV
program for 3 to 8 months. The overall average duration of participation was approximately 7
months (range: 1 to 17 months). Note that this duration of participation in the program does not
reflect the duration for a given beneficiary, but rather, the duration for the household as a whole.
We do not have information on the average duration of participation for individual beneficiaries
or categories of beneficiaries (i.e., pregnant, lactating women, malnourished children, or children
6-23 months of age).
Table 5.4 presents data on the time commitment required for households to come to and from the
FDPs and to use the food distribution services. Most of the respondents traveled to and from the
FDPs on foot (n = 38). Seven of the respondents used a mule to come to the distribution site.
Nobody had to pay for transportation. It took between 5 minutes and 3 hours (average of 1 hour)
to get from their homes to the FDP. In addition, they spent an average of about 4 hours at the
distribution site on the day of the observation. The respondents also reported that the time they
spent at the FDP on the day of observation was typical of what they usually spend there. Again,
the time spent at the FDP varied widely, ranging from a low of 1.5 hour to almost 7 hours.
As expected, respondents at the distribution sites where the food and food monitors arrived late
and the food distribution started only at around noon stayed at the FDP much longer than those at
distribution sites where the food and monitors arrived on time. The total time commitment of
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
79
food distribution attendance, including travel time and time spent at the FDP, averaged 6 hours.
This varied between 2.5 hours in the best case to 9 hours in the worst case.
Table 5.4. Time commitment for beneficiaries who attend FDPs
Characteristics Variable name
Mean
[N = 45] Range
Average time taken to get to FDP (minutes) q506a 58 5-180
Total time spent at the FDP on the day of
observation (minutes)
q522
(durmi522)
241 90-435
Total time usually spent at the FDP (minutes) q523
(durmi523)
240 60-600
Difference between total time usually spent and total
time spent on the day of observation (minutes)
timegaptoday_usual -0.7 -240 - +420
Total time spent on food distribution attendance
(minutes)
timeimpl_att 357 150-555
5.3.1.2. Health and commodity staff
At most of the observed distribution points, 3-4 food monitors were assisted by 1-2 health agents
and 1-2 colvols (see Table 5.5). On the day of observation, there was no MCH supervisor
present in half of the FDPs. In contrast, at least one commodity supervisor came to supervise the
activities of the food monitors and the food distribution process.
Food monitors usually stayed at the FDP for the whole day—until the food distribution was
completed. The supervisors, however, only spent a limited amount of time at the FDP. The food
monitors are the main actors at the FDP, and the pace of the food distribution depends mainly on
them. Therefore, we based our assessment of the beneficiary/staff ratio only on the food
monitors and did not include the supervisors, health agents, and colvols in the calculations.
Using this approach, we found a high average beneficiary/staff ratio of 77, ranging up to 188
beneficiaries per food monitor.
5.3.2. General implementation characteristics
This section presents results on the general implementation of the FDPs, and covers aspects such
as the venue for the food distribution, the number of communities (localités) covered, and the
amount of food distributed. Where possible, the data are compared to the implementation plan
for the FDPs.
Table 5.5. Number of MCH and commodity staff present at FDPs
Number of staff Variable name
Numbers
[N=10]
Number of food monitors present:
- 2
- 3
- 4
q402
1
4
5
Number of health agents present:
- 1
- 2
q403
2
6
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
80
Number of staff Variable name
Numbers
[N=10]
- 4
- 6
1
1
Number of colvols present:
- 1
- 2
- 4
q404
2
5
3
Number of MCH supervisors present:
- None
- 1
- 3
q405a
5
4
1
Number of commodity supervisors present:
- 1
- 2
- 3
q405b
5
3
2
Mean
[N = 10] Range
Beneficiary/food monitor ratio ratiobenefmonitor 77 29-188
5.3.2.1. Venue and communities covered
Of the 10 FDPs observed, 3 were held in the courtyard of a private house, 2 each at a church,
school or ADP office, and 1 at a dispensary. The 10 FDPs observed covered all the areas
included in the research. On average, one FDP covered 18 communities, varying between 6 and
42. The FDPs were not held separately for preventive and recuperative program communities, so
beneficiaries of both program groups received their food rations at one FDP for the group of
communities covered by that distribution point.
5.3.2.2. Amount of food received
Table 5.6 presents data on the amount of food received by the exit interview respondents and
compares this to the amount of food allocated by the program. These data were obtained by
reweighing the foods that respondents received on the day of the interview, using an electronic
scale. The amount of food a respondent should have received was calculated based on the
knowledge of the number of direct and indirect rations that they were eligible for. This
information was available for a total of 44 beneficiaries, 41 of whom were eligible for 1 direct
and 1 indirect ration, and 3 of whom were eligible for 2 direct and 1 indirect ration13. One case
of this latter group was excluded from the calculations because of uncertainty regarding the
beneficiary category of the second direct beneficiary.
Table 5.6. Amount of food received
1 direct child ration + 1 indirect child
ration [N = 41]
2 direct rations for children + 1 indirect
child ration [N = 2]
Food commodity
Amount
allocated in kg
Average amount
received in kg (range)
Amount
allocated in kg
Average amount received
in kg (range)
WSB 8.0 7.9 (5.7-9.8) 16.0 16.1 (15.6-17.0)
 
13 Even if a household has two direct beneficiaries participating in the program, only one indirect ration is provided.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
81
SFB 10.0 9.5 (8.1-13.9) 10.0 10.3 (8.1-12.8)
Lentils 2.5 2.6 (1.0-5.6) 2.5 2.6 (2.5-2.7)
Oil 2.0 2.4 (1.7-2.7) 4.0 4.3 (3.8-4.7)
The data in Table 5.6 show that on average, the amount received is close to the amount allocated.
The difference between the average amount received and the allocation was highest for the SFB,
with a difference of 0.5 kg (shown in Table 5.7). However, the variability in the gap between
amount allocated and the amount received is quite high, particularly for WSB, SFB, and lentils.
Thus, some households are likely to have received much less than their allocation, and some
other households much more. At the extremes, one respondent household received 2 kg less,
while another received 4 kg more SFB than was allocated.
Table 5.7. Differences between amount of food received and allocated
Characteristics Variable name
Mean
[N=44] Range
Gap between amount received and amount allocated (kg)
- WSB
- SFB
- Lentils
- Oil
gapwsb/gapsfb/
gaplenti/gapoil -0.07
-0.48
+0.10
+0.22
-2.3 - +1.8
-1.9 - +3.9
-1.5 - +3.1
-0.3 - +0.7
Numbers
Number of respondents who received correct amount of
food (+/- 0.20 kg)
- WSB
- SFB
- Lentils
- Oil
gapwsbcorr/
gapsfbcorr/
gaplentcorri/
gapoilcorr
17
7
30
22
Number of respondents who received less food than they
were allocated:
- WSB
- SFB
- Lentils
- Oil
gapwsb/gapsfb/
gaplenti/gapoil
28
36
10
7
Number of respondents who received more food than
they were allocated:
- WSB
- SFB
- Lentils
- Oil
gapwsb/gapsfb/
gaplenti/gapoil
14
8
24
33
We assumed an acceptable gap between allocated amount and received amount to be +/- 0.2 kg
and used this to derive the number of households who received the appropriate amount of food
rations for their beneficiary category. By this yardstick, approximately half of the respondents
received the correct amounts of lentils (n = 30) and oil (n = 22), and about 39 percent (n = 17/44)
received the correct amount of WSB. The results are worst for the SFB, where only 16 percent
(n = 7/44) received the correct amount (see Table 5.7). We also calculated the number of
respondents who received less food or more food than they were allocated. The results show that
the majority of respondents received less SFB than allocated (n = 36). For WSB, more than half
of the respondents received less (n = 28). In contrast, the number of respondents who received
more lentils (n = 24) and oil (n = 33) than allocated is much higher than the numbers who
received less of these foods.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
82
In summary, a large proportion of beneficiaries do not receive the amount of the different
commodities to which they are entitled. There seems to be a systematic bias, whereby recipients
are more likely to receive larger amounts of lentils and oil than allocated, but lower amounts of
SFB and WSB. It is not clear whether the direction of these errors is purely coincidental, but it is
unfortunate that the two commodities that seem to be generally distributed in smaller amounts
than allocated are the two fortified commodities SFB and WSB.
5.3.3. Quality of implementation
This section presents results on the overall quality of implementation of services at the FDPs.
Findings are presented on the timeliness of the operations as well as on the quality of the food
commodities that were distributed to participating beneficiaries.
5.3.3.1. Timeliness of operations
Our observations of the timeliness of the operation of the FDPs assessed whether the food
arrived on time and also whether the health and commodity staff arrived on time. We also
investigated whether all beneficiaries (from different communities) were invited to come to the
FDP at the same time. This particular aspect was included in our observations because our
research team had suggested to the WV program to invite beneficiaries from different
communities at different times in order to control the flow of beneficiaries and hopefully reduce
the waiting time. It was also felt that this could facilitate the implementation of complementary
BCC activities such as cooking demonstrations or distribution of education material to
emphasize the educational messages discussed at other venues, such as the MCs.
The results shown in Table 5.8 demonstrate that although the beneficiaries were usually told to
be at the FDP at 8:00 a.m., none of the observed food distribution activities actually started
before 9:00 a.m. At three FDPs, the distribution started between 9:00 and 10:00 a.m.; at three
other sites, between 10:00 and 11:00 a.m.; and at four sites, it started between 11:00 a.m. and
12:00 p.m. These delays can lead to substantial increases in the time commitment required for
beneficiaries.
Our observations revealed that the food arrived on time at 7 of the 10 distribution points on the
day of the observation. Delays in the arrival of the food at the remaining distribution points were
due to bad roads, a flat tire, difficulty finding transport facilities for the food, and delays with
loading the food at the warehouse.
Table 5.8. Timeliness of operations at FDPs
Characteristics
Variable
name
Numbers
[N = 10]
Start time of the food distribution:
- Between 9:00-10:00 a.m.
- Between 10:00-11:00 a.m.
- Between 11:00 a.m. - 12:00 p.m.
q414recod
3
3
4
Food arrived at the time communicated to the beneficiaries q408 7
Food monitors arrived on time q410 5
Health agents arrived on time q412 8
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
83
Characteristics
Variable
name
Numbers
[N = 10]
Proportion of beneficiaries asked to come at different times of day (percent) q407 0
The food monitors arrived on time at only half of the distribution points. Reasons reported by
those who were late included: having to stop to buy food or to eat before coming to the FDP, the
lack of fuel for their motorcycles, bad roads, or having to help load the food at the warehouse.
Food monitors are crucial to the functioning of the FDP and their late arrival often results in
severe delays for the large number of people involved in the food distribution activities.
The health staff, on the other hand, arrived on time at most distribution points, and were delayed
at only two distribution sites—one because of lack of transport and the other because s/he did not
know the start time for the food distribution.
Our recommendation to invite beneficiaries from different communities to come to the FDPs at
different times was not adopted by the program at the time of our interviews. All beneficiaries in
the observed FDPs had been asked to come at the same time. Thus, the waiting time for
beneficiaries was shorter or longer, depending on the order in which the communities were
called. We did not collect information on beneficiaries from all different communities within a
FDP, and therefore we cannot assess the magnitude of the differences in waiting time between,
say, a beneficiary from the first community served and one from the last community served on a
given day.
5.3.3.2. Quality of the food commodities
Our observations of the quality of the food at the 10 FDPs focused on visible infestation of the
food commodities with insects or worms, visible moisture, and/or any visible color change.
These observations were done before the food was distributed to the beneficiaries. In addition,
five bags each of wheat-soy blend (WSB), soy-fortified bulgur (SFB), and lentils were randomly
checked at each distribution point to see whether the bags appeared moist on the outside and to
check their expiration date. The expiration date of five containers of oil at each distribution
point was also verified.
In only one distribution site, three of the five bags of SFB checked appeared visibly humid from
the outside and the bulgur inside was lumpy. No other problems with the quality of the food
were observed in any of the other FDPs. There were no expiration dates on any of the bags or
containers examined at the 10 FDPs.
The data from the exit interviews with respondents (N = 45) also suggest that visible infections
or any other visible alterations in the quality of the food are rarely noticed. Only 2 respondents
reported having seen insects in the food commodities distributed. The data from the home
interviews also confirmed that infestation was not a widespread problem. Of the 29 home
interview respondents who had received food rations the previous month, approximately two￾thirds (n = 18) reported that they did not experience any problems with the foods. They felt that
the food was always fresh and that food monitors checked for humid bags, which were put aside.
Only four respondents mentioned occasional problems with the food quality, usually insect
infestation, and one mentioned that on one occasion the WSB tasted bitter.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
84
The results from the individual interviews with health agents (n = 20) suggest that the majority
felt that the quality of the food distributed was always good. However, two health agents talked
about their occasional experience with poor food quality, i.e., that they had observed insects in
the food, that the food was humid, or had a rancid taste. One of them reported in addition that
beneficiaries complained about very small lentil grains or SFB causing diarrhea. The discussants
of one of the two colvols’ focus groups voiced the same occasional problems with the food
quality. Members of the food monitors’ focus group explained that they usually checked for
bags with spoiled food at the warehouse. They reported that when they did find bags with
spoiled food, they immediately put those aside.
5.4. Use of food rations
This section presents the results on the use of food rations by the beneficiary families. The usual
length of time the different food commodities lasted in the home, the types of recipes usually
prepared with the food commodities and the sharing and selling of the foods received are
described. As in previous sections, the results are presented by topic, rather than by research
method.
5.4.1. Length of time food lasts in the household
Table 5.9 presents the results on the length of time each commodity lasts in beneficiary
households, by number of rations received and by number of indirect beneficiaries in the
household. In general, all the commodities were consumed within 2 to 3 weeks, depending on
the amount of food received and on the number of indirect beneficiaries in a household.
None of the commodities distributed by the program lasted for the entire month, i.e., until the
next food distribution. The SFB, lentils, and oil tended to run out before the WSB. The WSB is
the commodity with the highest fortification level of micronutrients and the one intended for the
beneficiary child; the WSB lasted, on average, approximately 3 to 3.5 weeks. Further, the data
in Table 5.9 indicate that the length of time is not strongly affected by the number of indirect
beneficiaries in the household. In cases where one direct child ration was received (n = 41), the
average duration of consumption of WSB decreased only slightly, from about 3.5 to 3 weeks
with the increase in the number of indirect beneficiaries. This suggests that a significant
proportion of the WSB may be consumed by the beneficiary child, which, if true, is consistent
with the program’s intention. The picture changes when a household received two direct rations,
i.e., when the amount of WSB doubles. With two direct rations, the food lasted about twice as
long when there were 4-6 indirect beneficiaries as opposed to 7-9 indirect beneficiaries in the
household. This finding might be purely by chance due to the small sample size, but could also
indicate a greater tendency to share or sell the food commodities in larger households, when the
amount of WSB received is larger. The same trend is observed for the donated oil.
Table 5.9. Duration of use of food in beneficiary households, by number of rations received
and by number of indirect beneficiaries
Duration Variable name
Mean
[N = 45] Range
How long the food lasted (weeks) q514a-d
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
85
- WSB
- SFB
- Lentils
- Oil
3.4
2.2
1.6
2.3
2-8
1-6
1-4
1-6
Duration of WSB in the household by number of
direct and indirect beneficiaries (weeks)
If 1 direct child + 1 indirect child rations and
- 1-3 indirect beneficiaries (n = 19)
- 4-6 indirect beneficiaries (n = 19)
- 7-9 indirect beneficiaries (n = 3)
If 2 direct child + 1 indirect child rations and
- 4-6 indirect beneficiaries (n = 3)
- 7-9 indirect beneficiaries (n = 1)
q514a/q511/
q512recod
3.5
3.3
3.3
4.0
2.0
2-5
2-8
3-4
4
2
Duration of SFB in the household by number of
direct and indirect beneficiaries (weeks)
If 1 direct child + 1 indirect child rations and
- 1-3 indirect beneficiaries (n = 19)
- 4-6 indirect beneficiaries (n = 19)
- 7-9 indirect beneficiaries (n = 3)
If 2 direct child + 1 indirect child rations and
- 4-6 indirect beneficiaries (n = 3)
- 7-9 indirect beneficiaries (n = 1)
q514b/q511/
q512recod
2.3
2.3
1.3
2.3
2.0
1-4
1-6
1-2
1-4
2
Duration of lentils in the household by number of
direct and indirect beneficiaries (weeks)
If 1 direct child + 1 indirect child rations and
- 1-3 indirect beneficiaries (n = 19)
- 4-6 indirect beneficiaries (n = 19)
- 7-9 indirect beneficiaries (n = 3)
If 2 direct child + 1 indirect child rations and
- 4-6 indirect beneficiaries (n = 3)
- 7-9 indirect beneficiaries (n = 1)
q514c/q511/
q512recod
1.9
1.5
1.0
1.0
1.0
1-4
1-4
1
1
1
Duration of oil in the household by number of direct
and indirect beneficiaries (weeks)
If 1 direct child + 1 indirect child rations and
- 1-3 indirect beneficiaries (n = 19)
- 4-6 indirect beneficiaries (n = 19)
- 7-9 indirect beneficiaries (n = 3)
If 2 direct child + 1 indirect child rations and
- 4-6 indirect beneficiaries (n = 3)
- 7-9 indirect beneficiaries (n = 1)
q514d/q511/
q512recod
2.2
2.1
2.3
4.7
2.0
1-4
1-4
2-3
4-6
2
For the SFB and the lentils, which are meant to be consumed by other household members rather
than by the beneficiary child, the data show that in households that received one direct child
ration, the food lasted for a shorter time when there were more than six indirect beneficiaries in
the household. The amounts of SFB and lentils donated by the program do not change when a
household has two direct beneficiary children. As expected, these commodities lasted for
approximately the same amount of time, regardless of the number of direct rations received.
5.4.2. Recipes prepared with the food commodities
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
86
The exit interview respondents were asked what recipes they usually prepare with the donated
foods, namely WSB, SFB, and lentils, and which family members usually consumed these
recipes. At the time of the data collection, the fieldworkers also made note of when the
respondents mentioned that they did not consume all of the food, for example, that they shared
the foods or meals prepared, using the food commodities with others outside their household.
The results are presented in Table 5.10, and show that all respondents reported having prepared
gruel with the WSB. Half of them used the donated flour to prepare dumplings and about one￾third made fried snacks. In a few households, the flour was consumed raw, and 10 of the 45
respondents indicated that they did not consume all of the flour (i.e., they shared with others
outside of their household). In nearly half of the households, the preparations made with the
WSB were consumed by all family members. Only approximately 30-40 percent of the
respondents reported that the preparations were used mainly for the beneficiary child; this
percentage was slightly higher (44 percent) for the gruel, compared to the dumplings (30
percent).
The SFB was usually cooked with lentils or beans (n = 45) in the same fashion as the traditional
Haitian dish of rice and beans. Other frequent preparations were SFB with leafy vegetables and
dried fish (n = 27) or gruel (n = 19). Half of the respondents reported having cooked it plain.
About one-third prepared the SFB with vegetables, or beans and fish, or served it with bean
sauce. Approximately 50 percent (n = 22) mentioned that they shared the SFB with others
outside the household. Compared to the WSB-based meals, the cooked SFB was even more
frequently consumed by the entire family, with the exception of the gruel, which was consumed
mainly by the beneficiary child in a third of households.
The majority of respondents cooked lentils with SFB (n = 44) or prepared lentil sauce (n = 41).
Less frequently, the lentils were cooked with other cereals than SFB (n = 33), used for a
bouillon, or prepared with SFB and fish (n = 2). Twelve respondents mentioned that they shared
some of the lentils with others outside the household. As with the SFB, the meals cooked with
lentils are consumed in most of the households by the entire family.
Table 5.10. Recipes prepared with the commodities (from exit interviews, n = 45)
Type of recipe Variable name
Frequency of
preparation
(Numbers)
Who consumed
(percent)
Mainly
beneficiary
child
All
children
Entire
family
WSB
Gruel 44 16 40
Dumplings 30 13 57
Fried snacks
Other (eat raw)
Did not consume all (sharing of food/meals
with others outside household)
q515a1-a5
gruel/
dumpling
other/dncall
congruel/
condump/
conmarin/
conother
45
23
14
5
10
36
40
21
20
43
40
SFB
q515b1-b5
sfbgruel/
plain/lentil/
fish/sfbother/sfbnall/
consfbgr/
conplain/
conlent/
confish/
consfbot
19
23
45
27
15
22
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
87
Gruel
SFB cooked plain
SFB cooked with lentils or beans
SFB with leafy vegetables and dried fish
Other (SFB with vegetables, SFB with
beans + fish, SFB with bean sauce)
Did not consume all (sharing of food/meals
with others outside household)
26
9
11
7
7
74
91
89
93
93
Lentils
Lentil sauce
SFB cooked with lentils
Other cereals cooked with lentils
Other (Lentils with SFB + fish, Bouillon
with lentils)
Do not consume all (sharing of food/meals
with others outside household)
q515c1-c5
lensauce/
lenbean/
lenothc/
lenother/
lennall/
consauce/
conbean/
conlcere/
conlenot
41
44
33
2
12
5
11
9
95
89
91
100
Table 5.11 presents information on the addition of extra ingredients when the food aid
commodities are prepared for the beneficiary children. Depending on the food, up to 80-90
percent of the caregivers reported that they added beans, meat, dried fish. or cow’s milk 2-3
times per week to the meals prepared for the children. About half of the caregivers stated that
they added eggs, liver, or other ingredients, such as vegetables, green leaves, or fish, once or
twice per week to these meals. Note that the frequency of use ranged from as low as less than
once per week to every day for certain ingredients. However, these results are encouraging,
provided that the frequencies do not only reflect what caregivers would have liked to do but
rather what they were actually able to do. Also note that we have no information on the amounts
of the different ingredients added.
The data from the home interviews about the types of recipes prepared with the different food
commodities confirm the information presented above from the exit interviews. All of the home
interview respondents who had received WSB (n = 12) declared they used the flour mainly for
gruel. Half of them used the WSB for dumplings, and about one-third made fried snacks. Most
of the respondents who had received SFB the month before (n = 29), reported cooking it with
lentils or beans. Half of the respondents mentioned that they had cooked it plain or prepared it as
gruel. Similar to the findings from the exit interviews, the lentils were mainly used for sauce or
cooked with cereals. Consistent with the exit interviews findings, the home interview
respondents reported that the meals prepared with the food commodities were generally
consumed by the entire family. The respondents also reported that young infants (under 6
months of age) were not fed meals prepared using the donated food commodities. In one case,
the respondent reported that the WSB gruel was consumed only by her and the children and that
the dumplings and fried snacks were prepared specifically for the children. Many of the
respondents mentioned that neighbors also consumed the meals prepared using the donated food
commodities, especially the ones prepared with SFB and lentils.
Table 5.11. Use of additional ingredients when preparing the food commodities for
children (from exit interviews, n = 45)
Added ingredient Variable name Percent
Frequency of
use (times/week) Range
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
88
Mean
Eggs (n = 25) q517a/eggw 56 2.2 0.2-3
Goat’s milk (n = 4) q517b/goatmw 9 2.0 1-3
Breast milk (n = 0) q517c/breastmw 0
Cow’s milk (n = 39) q517d/cowmw 87 3.4 0.2-7
Beans (n = 42) q517e/beansw 93 2.7 0.7-7
Groundnuts (n = 4) q517f/nutsw 9 1.5 0.5-3
Dried fish (n = 37) q517g/fishw 82 2.8 0.9-5
Meat (n = 39) q517h/meatw 87 1.8 0.5-7
Liver (n = 21) q517i/liverw 47 1.1 0.2-4
Other (vegetables/green leafs, fish, banana,
corn, canned milk) (n = 27)
q517j/other1w 60 2.0 0.2-5
Other (vegetables, banana, fish, canned
milk) (n = 9)
q517k/other2w 20 1.6 0.2-3
Overall, the findings confirm the well-recognized sharing of donated commodities among family
members and even, in some cases, with other families and neighbors. There is some indication
that the WSB—which is intended for the beneficiary child—may be used preferentially for this
child in approximately 40 percent of the households interviewed. However, most foods and
meals made with donated foods are consumed by all family members.
5.4.3. Sharing and selling of food commodities
We have information from home interview respondents (n = 29) who had received food the
previous month on whether they shared, sold, or exchanged the donated food. A majority
reported having shared food, either from time-to-time or regularly, with neighbors or other
family members living outside the household (see Table 5.12). The most frequently mentioned
family members with whom they had shared the food were their mother, sister, mother-in-law,
sister-in-law, and grandmother. Two respondents said that they also shared the food with
strangers, for example, people that they met on their way back home from the distribution site.
Sharing of food was in many cases motivated either by feelings of charity—a person was hungry,
in a miserable situation, or simply not in the program—or by a felt obligation vis-à-vis other
family members, hoping to receive the same favor when they would be in need. Three
respondents expressed that they sometimes compensated people who had helped them by
offering them some of the donated food (either raw commodities or prepared meals). These
people were either working on their farm or a family member who usually picked up the food at
the distribution site. A few indicated that they shared the food with neighbors to avert jealousy
or because God had said that one should share. Many of the respondents who reported not
having shared the donated foods mentioned that prepared meals using these foods were often
shared with visitors who arrived when the family was eating.
Table 5.12. Sharing and selling of food commodities (from home interviews, n = 29)
Characteristics
Frequency
(Numbers)
Number of beneficiaries who shared food 20
Shared with whom:
- Family 10
- Neighbors 11
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
89
- Strangers 2
Reasons for sharing:
- Person not in program 9
- Because person/family is hungry 8
- Because they are family 6
- Compensation for help 3
- God says so 1
- To avert jealousy 1
Number of beneficiaries who exchanged the food for other food 1
Number of beneficiaries who sold food to buy other necessities 3
Seven respondents shared only one food commodity (SFB in most cases) with others, 5 shared
SFB and/or lentils, and the remaining 8 shared three foods (SFB, WSB or lentils, oil). SFB was
by far the most frequently shared commodity (n = 19/29), followed by oil (n = 11/29), lentils
(n = 10/29), and WSB (n = 1/12). The amounts reported for the last time they had shared food
varied widely: (1) for the SFB, between 1 measuring cup and 1 local pot (about 2.5 kg); (2) for
the lentils, between ½ cup and 4 cups; (3) for the WSB, 10 cups (only one family reported
sharing the WSB); and (4) for the oil, between 3 ounces and 2 quarts. Although the amounts of
SFB and oil shared go up to 1 local pot and 2 quarts, respectively, the reported amounts for these
foods ranged in most cases between 1-12 cups for SFB and 3-18 ounces for oil.
One respondent reported having exchanged some of the SFB for corn or millet. She felt that the
food received from the program sometimes caused diarrhea, specifically the SFB.
Only three respondents confirmed that they needed to sell some of their food ration from time-to￾time. All three voiced that they had been obliged to sell about ½—1 local pot of SFB
(approximately 1.2-2.5 kg) in order to buy other necessities, such as soap, bleach to purify water,
gas, spices, or rice. Some of the respondents who reported that they did not sell any of the food
received gave the following reasons: one indicated that the amount of food—after having shared
the ration with friends, family, and neighbors—is usually just enough for her family and that
there is nothing left that could be sold; others stated that they liked the food too much, especially
the SFB, to sell it or exchange it—they also mentioned that they even buy the SFB on the market
if they run out of it; finally, others mentioned that they never sold any of the food received, in
spite of the fact that they saw others do it.
In addition to these results, our fieldworkers reported on rumors about food selling that they
heard from beneficiaries. There is a rumor that traders from Port-au-Prince come to buy the
donated foods from the beneficiaries. As reported by some beneficiaries, they either wait not far
away from the distribution site for the beneficiaries or they go from house to house to buy the
food. Other beneficiaries mentioned that they got robbed when they returned home after 6:00
p.m. The fieldworkers also reported on a number of approaches people use to try to get into the
food aid program or to stay longer in the program: (1) volunteers help unloading the food from
the trucks to be well seen by the food monitors and hopefully to benefit from possible leftover
food; (2) women try to get pregnant from whomever in order to benefit from the food; and (3)
some beneficiary families do not give the WSB to the child in order to keep the child
malnourished (the WSB is rightly seen as the most nutrient-dense food). These rumors are not
unusual in the context of food aid programs, but they are of real concern and are probably worth
following up.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
90
5.5. Staff perceptions about the Food Distribution Points
Information about perceptions and opinions of the health staff regarding the FDPs was obtained
in individual interviews with the 20 health agents and in five focus group discussions with
colvols, MCH supervisors, food monitors, and commodity supervisors. This section summarizes
the results concerning the problems they perceived in relation to the food distribution process
and their suggestions on how to improve the food distribution.
5.5.1. Perceived problems with the food distribution
The individual interviews and focus groups with the program staff revealed the following major
concerns with the food distribution process.
Delays in commencing at the FDP. Individual interviews with the health agents revealed that
they perceive delays in the start of the food distribution due to the late arrival of the food to be a
major concern (n = 17). At the same time, the health staff understood the reasons for such
delays; the colvols indicated that rain was a major obstacle for the timely arrival of the food at
the distribution site, as was the availability of food at the regional warehouse on the scheduled
day for the distribution. Most of the health agents (n = 16) also confirmed in individual
interviews that the latter was a concern, but they pointed out that it did not happen frequently.
Some of the health agents also attributed some of the delays to the bad conditions of the roads.
The challenge of trying to adhere to the scheduled distribution dates and start times was a
concern that was also voiced in the focus groups with food monitors and commodity supervisors.
They explained that bad roads, rain, breakdown of trucks, or administrative problems made
timely arrival of the food at the distribution site difficult. In addition, they explained that the
food stock at the regional warehouse was sometimes insufficient, but that they did not receive the
information on time to be able to communicate the change in the distribution date to the health
team.
Mismatched beneficiary lists. In the individual interviews with health agents, a frequent concern
was that the names of eligible beneficiaries were often missing from the distribution lists of the
commodity team (n = 15). The discussants of the focus groups with food monitors and
commodity supervisors also raised concerns about frequent differences in the number of eligible
beneficiaries listed in the monthly distribution lists compared to the number of beneficiaries
listed in the master list that includes all beneficiaries with information on entry/exit dates in the
program. They also indicated that beneficiaries did not receive their rations if their entry and
exit dates did not figure in the master list or the ration cards, or if the name of a beneficiary
appeared on two distribution lists of different zones.
Six health agents expressed that they frequently experienced problems with beneficiaries getting
angry if they did not receive their food rations. Members of the colvols’ focus group also felt
that beneficiaries blamed the health staff in situations where they had to return home without
their ration.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
91
Crowding at FDPs. Half of the health agents felt that there are too many beneficiaries per
distribution point. This concern was also mentioned by the discussants in the MCH supervisors’
focus group. Note that the commodity team (food monitors and commodity supervisors) did not
raise this point as one of their concerns.
Lack of respect for beneficiaries and health staff. In the three focus groups with the health staff
(colvols, MCH supervisors), the discussants expressed their view that food monitors lack respect
towards the health agents and colvols and towards the beneficiaries. They felt that food monitors
often treated the beneficiaries badly, even to the extent that they would not distribute the food
ration to a person who was present at the FDP if s/he did not hear his/her name called the first
time, because of the noisy environment. Also, the health staff felt that making beneficiaries wait
in the sun the whole day to receive their ration was humiliating. MCH supervisors expressed
that the food monitors often viewed the health staff with suspicion. These points were also
raised by a few health agents in the individual interviews. In addition, others indicated that there
was a lack of dialogue and collaboration between the health staff and the food monitors.
Food handling. Several health agents as well as the discussants in the colvols’ and MCH
supervisors’ focus groups expressed concerns in relation to food handling during the distribution.
They felt that the distribution process was often unhygienic—the SFB was emptied on dirty
plastic films and the oil in dirty containers, and members of the distribution team used their
hands when measuring the oil. In addition, they complained about the local containers used to
measure the amounts allocated per beneficiary. They felt that different people received different
food amounts at the same distribution point.
Bringing ration cards to FDPs. The individual interviews with health agents and the five focus
group discussions all raised the concern of beneficiaries not bringing their cards to the FDPs.
For the health staff, the main concern was that beneficiaries who were otherwise eligible were
not given their food rations, even if the health staff vouched for them. Two health agents
reported that they often asked beneficiaries to return home to get the card, but that beneficiaries
who lived far away came back too late to the FDP, when the food monitors had already prepared
their reports and the food distribution was over. These points were also raised by discussants in
the two focus groups with colvols and the MCH supervisors’ group. In addition, the colvols
expressed that whether beneficiaries who forgot their cards would receive food or not depended
mainly on the food monitor’s goodwill.
The commodity staff, on the other hand, were bound by the rules of the food distribution process,
and explained that they were not allowed to distribute food to those beneficiaries who do not
bring their ration cards, except in cases where beneficiaries had recently entered the program and
did not yet have cards. The commodity supervisors were more flexible, and expressed the view
that they would distribute food rations to a beneficiary who forgot the ration card if the health
staff confirmed the eligibility of the person and if the beneficiary were able to list the names of at
least three indirect beneficiaries in order to verify that the person is a beneficiary registered in the
master list. It could be that this potential for flexibility was not communicated well to the food
monitors.
5.5.2. Suggestions on how to improve the food distribution process
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
92
Most of the suggestions made by the health agents were in direct relation to the problems
identified earlier, and are listed below in the order of the priority in which they were expressed
by the staff. They recommended that the following actions be taken by program managers in
order to improve the functioning of the FDPs:
ƒ Address overall logistical and punctuality issues by:
- Providing the necessary support to ensure the timely arrival of the food at the FDP, the
timely start of the distribution process, and adherence to the scheduled distribution dates
for the different zones.
- Reviewing the number of beneficiaries per distribution point and establishing smaller
distribution points.
- Scheduling beneficiaries from different zones to receive food at different times on the
day of distribution.
ƒ Ensure that all eligible beneficiaries receive food by:
- Requesting the commodity team to prepare the distribution lists together with the health
agents to make sure that all eligible beneficiaries are listed.
- Preparing lists separately by zone when beneficiaries from different zones received the
food rations at the same distribution point. This would help facilitate the preparation of
reports by the health agents.
ƒ Encourage better collaboration between food monitors and health agents by:
- Encouraging greater respect on the part of the food monitors for beneficiaries and for
health agents, particularly when the latter made the case for beneficiaries who had lost or
forgotten their ration cards.
ƒ Ensure appropriate food handling and correct amount of food by:
- Requesting that the commodity team ensures hygienic food handling during the
distribution process.
- Developing standardized local containers for each food commodity in order to make sure
that all beneficiaries receive the allocated food amounts.
- Requesting that the commodity team announce the food amounts allocated for the
different beneficiary categories in local measures instead of kilograms at the information
session before the distribution starts.
ƒ Consider increasing the ration size and introducing milk powder in the ration for children.
5.6. Beneficiary perceptions about Food Distribution Points
Information about perceptions of the beneficiaries regarding the FDPs and the quality of the food
rations was obtained through the home interviews (n = 29). This section summarizes the results
on the problems they perceived in relation to the food distribution process, the perceptions of the
importance of the food distribution and the food rations, and their suggestions on how to improve
the food distribution program.
5.6.1. Perceived problems with the Food Distribution Points
Most of the home interview respondents (n = 22) expressed that the time involved in attending
the food distribution was a major obstacle to the use of this service. They explained that the food
distribution often took the whole day because of the late arrival of the food or simply because
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
93
there were so many beneficiaries at the FDP. According to them, the time implications also
depended on the beneficiary category one belonged to—pregnant and lactating women were
served before the children. In addition, the respondents indicated that distribution points were in
general not close to their homes and they had to walk long distances to get there. One-third
(n = 7) complained that they sometimes even returned home without the food, either because
they were away for a few minutes when their name was called, or else their name was not called
at all, or the food did not arrive on that day.
Only two respondents felt that the amounts of food received varied considerably. They
explained that the local measures used were not standardized and that the quantity also depended
on when a beneficiary was served. They felt that those who received the food early on during
the day got less food than those who were served toward the end of the day. One respondent also
expressed that beneficiaries were sometimes badly treated and that she felt a lack of respect from
the food commodity staff towards the beneficiaries. She expressed that one’s dignity should be
preserved even if food is donated.
5.6.2. Perceptions about the importance of the food distribution and the food rations
The food distribution was well appreciated by respondents. Some of them considered this
service as more important than other WV services they used, although many had difficulties
ranking different services because they felt that all services were important.
In general, all respondents indicated that they liked the food because it gives strength, contains
vitamins, and does not exist in the market (in the case of WSB). Many respondents expressed
that the foods, and specifically the WSB, kept their children healthy, although it did not prevent
illnesses. Lactating respondents explained that even though the child did not consume the food,
it received strength and vitamins with the breast milk. Other respondents felt that the food
received helped them to cope with food insecurity, when they had nothing to eat at home. They
expressed that thanks to the donated food, they could be sure that their children would not starve
from hunger.
There were a few exceptions, where some family members did not appreciate the food. In one
case, the respondent reported that her child (the beneficiary child) did not want to eat the WSB.
Twelve respondents voiced some concerns about problems with the food and talked about
episodes of diarrhea after the consumption of WSB and/or SFB. In some cases, this was seen
among adults as well as children only when they first started consuming the food (50 percent),
while for others, it was seen when they consumed the food frequently. Similar findings were
reported in the exit interviews at the FDPs.
5.6.3. Suggestions on how to improve the food distribution
Approximately two-thirds of the home interview respondents had no suggestions on how to
improve the food distribution. The remaining respondents recommended that program managers
take the following actions:
ƒ Address overall issues of logistics and punctuality by:
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
94
- Ensuring the early arrival of the food at the distribution site so that beneficiaries return
home before nightfall.
- Avoiding food distribution when it rains.
- Organizing FDPs closer to home communities of beneficiaries and with fewer
beneficiaries per point; for example, by organizing the food distribution by beneficiary
category, i.e., one day for pregnant women, one day for lactating women, and another day
for children.
- Informing beneficiaries in advance of changes in the distribution dates; also informing
the beneficiaries who are not eligible to receive the rations in advance, so that they do not
come to the FDP for nothing.
ƒ Ensure all eligible beneficiaries receive food.
ƒ Standardize the local measures for each food commodity, so that everybody receives the
same amount of food.
In addition, they suggested that the program staff should check the local markets after a food
distribution in order to assess the extent of food selling. Some said that it would be nice if
everyone could be in the program, so that there would be no need to sharing the food received.
Finally, some said that the food ration should be increased as the amounts are not sufficient.
5.7. Summary of findings regarding operations at the Food Distribution Points
5.7.1. Operations and quality of services at the Food Distribution Points
Attendance and beneficiary/staff ratio. Attendance at the FDPs is very high and so is the
beneficiary/food monitor ratio. Not surprisingly, the time involvement for participants in this
activity is extremely high, averaging 6 hours, including travel time. The main reasons for the
large amount of time involved in receiving the food are delays in the arrival of the food and the
staff at the FDPs.
Amount of food received. Our measurement of the amount of food received by different types of
beneficiaries showed that there were significant errors and that the large majority of beneficiaries
did not receive the allocated amount of the different products. The problem was more severe for
SFB (only 16 percent of the beneficiaries received the correct amount +/- 0.2 kg) and WSB (less
than half received the right amount). Because some received more and others less, from the
program resources perspective, these errors may cancel each other out. However, from the point
of view of the beneficiaries, these problems lead to feelings of injustice and dissatisfaction with
the program.
Quality of the food commodities. The quality of the food observed was generally good and
problems were observed only in one FDP, where three of the five bags of SFB inspected were
visibly humid from the outside and the product quality had been altered. Neither visible
infestation of food commodities with insects or worms nor any visible color changes were
observed or reported in the interviews.
5.7.2. The use of food rations
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
95
On average, the commodities received lasted 2-3 weeks, but none lasted for the entire month.
Sharing of the commodities among all family members was almost universal for SFB and for
lentils, but was less common for WSB. The large majority of beneficiaries (70 percent) also
reported sharing the food commodities, either with other family members (not living in the same
household) or with neighbors or other people.
These results confirm the well-recognized sharing of donated food commodities among family
members and in many cases also with other families and individuals. Although there is some
indication that the WSB—which is specifically targeted to the young child and has higher levels
of multi-micronutrient fortification than the other commodities—is used preferentially for the
young child, more than half of the households (60 percent) reported sharing the WSB among
other family members.
The practice of sharing donated commodities reflects the reality of rural Haiti, where most
families are extremely poor and rely heavily on social networks, and where sharing of received
goods is an obligation in order to benefit in turn from goods that others may receive. A few
families also admitted to selling some of the donated commodities at times, in order to purchase
other basic needs. It will be difficult for the program to change these practices unless other
supporting program activities can be put in place to significantly increase the resource base
available to these poor households.
5.7.3. Staff perceptions about the Food Distribution Points
Perceived problems. The program staff identified several concerns regarding the functioning of
the FDPs, most of which related to logistical and organizational aspects. Delays in the start of
the food distribution due to the late arrival of the food was perceived by the health agents as a
major operational problem. The food monitors and their supervisors also expressed their
concerns regarding the start time of the food distribution, and mentioned that adherence to the
schedule of food distribution was also an issue. Problems with the lists of beneficiaries and with
beneficiaries forgetting their ration cards were also commonly raised. These problems are real
concerns, because they result in some beneficiaries returning home without their food ration.
This causes a lot of frustration among beneficiaries and dissatisfaction with the program, and
also results in problems for the staff.
Other concerns raised mostly by the health agents included crowding at the FDPs, lack of respect
from the food monitors towards the beneficiaries and the health staff, and the non-hygienic
handling of foods in some cases.
Suggestions for improvements. The health agents recommended that the program address the
overall logistical and punctuality issues by providing support to ensure the timely arrival of the
food at the FDPs and to enable adherence to the scheduled distribution dates. They also made
two specific suggestions on how to reduce crowding: establish smaller distribution points; and
schedule beneficiaries from different zones to arrive at different times. They recommended that
additional efforts be made to ensure that all eligible beneficiaries receive their food. The health
agents also recommended that hygiene in food handling be improved and that local containers be
standardized to ensure that beneficiaries receive the correct amount of each commodity.
5.7.4. Beneficiaries’ perceptions regarding the Food Distribution Points
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
96
As expected, the food distribution was well appreciated by respondents. Many indicated that
they particularly appreciated the WSB because it contains vitamins, which keeps children
healthy. Some also felt that the food distribution helped them to cope with food insecurity.
Perceived problems. As expected, most beneficiaries considered that the time involved in
attending a FDP was a major obstacle to the use of the service, and it was due to a combination
of the long distance to the FDP and delays in the start of the food distribution. A significant
number of respondents expressed concerns about the food, and especially the WSB and the SFB,
which they believed caused diarrhea. It is difficult, however, to tell whether the donated foods
are really responsible for the reported diarrhea.
Suggestions for improvement. Only a few beneficiaries proposed suggestions on how to improve
food distribution. Their recommendations were very similar to those made by the health staff
and focused mainly on ways to improve the logistics and punctuality issues. They also
recommended that the program staff visit the local markets after a food distribution day to assess
the extent of food selling. Finally, they indicated their wish that everybody should receive food
donations so that there is no need for sharing with non-recipients; they also recommended that
the ration sizes be increased.
5.7.5. Conclusions on operations at the Food Distribution Points
In summary, our data confirm the existence of logistical constraints related to the transport and
distribution of food commodities in the study area. Transport problems, which affect the timely
arrival of both food and staff, are difficult to overcome in areas such as the Central Plateau,
which are characterized by extremely poor road conditions, and especially so during the rainy
season. Other problems, such as excessive crowding and long waiting times, which are also
affected by logistical constraints inherent to the environment, continue to be a challenge, but
need to be addressed. Solving the measurement problems to ensure that beneficiaries receive the
amount of food they are entitled to is also important for fairness as well as to ensure efficient use
of resources. The sharing of food commodities is probably unavoidable, but the program should
continue to emphasize the importance of prioritizing the targeted beneficiaries within the family,
and the differential use of selected food commodities such as the WSB for young children who
have high nutrient requirements. The other commodities, which are provided specifically to
complement the diet of other family members, can continue to be used for this purpose.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
97
6. STAFF PERCEPTIONS OF PROGRAM ORGANIZATION AND
MANAGEMENT
This section describes the perceptions of the different WV stakeholders regarding various aspects
of the program’s organization and management. First, the management structure and the key
stakeholders in the program are described to provide context. This is followed by a short
summary of the methods and samples used to assess the staff perceptions, and then by a
presentation of the results related to work roles, motivation, constraints to general satisfaction,
supervision, and coordination between the health and food components of the program.
6.1. Management structure and key stakeholders in the program
For the purpose of this evaluation, the main stakeholders in the program included management
and field staff at different levels of the program. The roles of these different stakeholders are
described in detail in Section 2.3. A brief overview of the organizational structure, including the
roles of the key stakeholders in the program, is presented below.
Health agents and colvols (program volunteers) are the direct implementers of the MCH program
in the field and thus, are the frontline staff in contact with the program participants. They are in
charge of the interventions being delivered at the RPs and MCs and assist the food monitors at
the FDPs. Health agents are WV employees and receive a monthly salary. Colvols are
community volunteers who assist the health agents in their duties. They receive a small monthly
incentive from WV. Both health agents and colvols are supervised by nurses who work under
the supervision of the Assistant of the Regional Health Coordinator in Hinche. This assistant is
supervised by the Regional Health Coordinator. The National Health Coordinator for WV is
based in Port-au-Prince and oversees the activities in all the program areas of WV in Haiti. The
organizational structure for the health component of the program is presented in Annex 6.
Food monitors are responsible for the distribution of the food rations at the FDPs. They pertain
to the Commodity Section of the program, and are supervised by the field supervisors, who work
under the supervision of the Assistant of the Regional Commodity Officer. This Assistant is, in
turn, supervised by the Regional Commodity Officer in Hinche. The organizational structure for
the food component of the program is outlined in Annex 7.
6.2. Methods used to understand the program organization and management
The data that provided information on the program organization and management were gathered
using the following methods:
ƒ Focus group discussions with colvols, health agents, and MCH supervisors (N = 5);
ƒ Focus group discussions with food monitors and commodity supervisors (N = 2).
The data from the focus groups were supplemented by semi-structured interviews with the
national and regional health coordinators and the regional commodity officer.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
98
As described in Section 2.4.5, the field notes and audiotapes from the focus group discussions
were transcribed and then analyzed to identify specific themes that reflected staff responses to
the broad categories of issues discussed in the focus groups. We examined the data to identify
theme-based statements that reflected ideas, concepts, values, feelings, or emotional states. The
presentation of the results in this chapter discusses each of the themes identified under the broad
categories of (1) staff perceptions related to their responsibilities; (2) motivation and perceived
value of the program; (3) constraints to job satisfaction; (4) supervision; and (5) coordination
between health and commodity sectors.
6.3. Perception of responsibilities of World Vision staff
The three major themes that emerged from interviews with health and commodity staff in
relation to their current responsibilities were: (1) the feeling that their jobs entailed serious
responsibility; (2) their sense of increasing confidence in their abilities to handle their
responsibilities; and (3) their role in contributing to behavior change of both individuals and
communities. In Table 6.1, an X indicates that the theme was present in the focus group
transcript. The results indicate that feelings of serious responsibility as well as confidence in
their ability to do the job were seen across all focus groups, while sentiments about changing
lives of the communities they worked with were primarily seen in discussions with health staff.
Table 6.1. Themes with respect to current responsibilities
Focus groups: Themes
Work entails serious
responsibility
Perception of changing
lives and communities
Confidence with their
ability to fulfill their
responsibilities
Colvols XX X
Colvols XX
Health agents X X X
Health agents X X
MCH supervisors X X X
Food monitors X X
Commodity supervisors X X
Serious responsibility and changing lives and communities. In all seven focus groups, the
discussants voiced the view that they were engaged in serious activities and perceived their
responsibilities as large. In addition, the members of the health teams, and specifically the
colvols and MCH supervisors, expressed the view that they were contributing to changing
peoples’ lives and pointed this out with pride. They also expressed their felt need or obligation
to help mothers obtain more knowledge and thus have healthier children. Finally, the sense that
the program had changed from the way it was implemented in the past was also expressed,
particularly with the view that the current implementation was more effective. The following
statements illustrate some of the ways in which the health staff have internalized the goals of the
program:
“Facing my responsibilities in the program, I feel proud. I have the information on how
a child develops, how the child should eat, and what a pregnant woman should do.”
(colvol)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
99
“I am feeling obliged, although the work is difficult. When one realizes that there are a
number of mothers who introduced in the past foods very early to their children and who
know now how to feed their children, e.g., when to introduce foods.” (colvol)
“The program is not a game; it is a program that targets young children and pregnant
women; the program combats infant mortality, risk pregnancies, and malnutrition.”
(colvol)
“For us, it is a mission. In the past, mothers lost their children because of lack of
education, because they were ignorant.” (colvol)
“Our responsibility is not small, it is big; we help mothers to keep their children healthy
so that they develop adequately; this is especially true for the MCs, we help the mothers
and they also help each other.” (colvol)
“Looking at how the work is done currently compared to the past, one can see that there
is a big difference; today people learn better.” (health agent)
“When the implementation of the program started, the work was a little bit difficult. But
today, with the testimonies I receive from mothers and beneficiaries, I feel that we play a
role in the changes in the communities that are about to happen. That motivates me even
more to continue so that we reach what we are looking for.” (MCH supervisor)
“The program has a component of recuperation of malnourished children, which is of
utmost importance but which adds stress to the work and makes our responsibilities
heavier, but, given the engagement of the staff, the way we are motivated to do the work
and seek perfection, although our responsibilities are big, but our love for our work helps
us to ignore them, this is what makes me say that everything works well.” (MCH
supervisor)
Another theme in relation to responsibilities that emerged in the discussions concerned increased
workload. In response to a question about changes in their responsibilities compared to the
previous year, all of the groups identified an increased workload in connection with their
increased responsibilities, but many of them did not make negative references to their workload.
The discussion in both colvols’ and health agents’ focus groups described this workload increase
in relation to the larger number of MCs they needed to organize, and also in relation to the
increase in the number of caregivers coming to the RPs. Colvols expressed clearly their views
that they now have as much work as the health agents. For the MCH supervisors, the increased
workload was described in relation to an extension of their supervision areas. Below are some
examples of the ways in which this theme was expressed:
“In the past, colvols did not have much work; now, this is different. We have as much
work as the health agents, if not even more. As they can hold a session of a MC, we can
do it now as well, thanks to the training recently received.” (colvol)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
100
“The work time has nearly doubled since last year. Last year, one could have between 2
and 3 MCs; now it’s between 8 and 10. Today, we have more people coming to the RPs.
For this year, I feel that we have to invest more time in each of the services.” (health
agent)
The health staff also expressed their views about the value of the recent training sessions for
achieving project goals and changes in the communities. The members of the health team
described the training as informative and practical and highlighted the idea that they had gained
not only a lot of knowledge but also new communication skills, which they felt they needed in
order to do community work. Overall, a positive attitude toward the program emerged in the
discussions about current responsibilities and training. This was particularly true for the health
staff, who were also much more open to talking about perceptions and feelings.
The following quotes from the data provide insights into how the staff perceived the training
sessions and the positive implications for their work:
“If we had an idea about health, the training helped us to become good; we are
professors now, the first time I felt like someone misplaced, abandoned, but after that I
said to myself that it is through practicing that one improves.” (colvol)
“This year the training was not a theoretical training, it was an applied training, and
topics were different. We have listened, we have seen, and we have applied.” (colvol)
“After the training received, today, there is no longer a monologue but a dialogue
between the mothers and us others when we animate a MC, which is very much
appreciated by the mothers.” (colvol)
“When we transmit the training received to the mothers, if they could apply things as we
teach them, the objective of USAID and WV would be achieved.” (colvol)
“Thanks to the training, we have more knowledge; there is one thing that we were taught
related to adult learning, that is named respect, we can’t find it though with the
managers.” (health agent)
“The training made us more dynamic in our work; it allows us to play a role in the
development of the community.” (health agent)
“The way the training was done put us in front of ourselves; it made us think about our
own knowledge; it showed us that we too know some things.” (MCH supervisor)
“I am for nearly 10 years with WV. This training that I have received in November and
December, it is the first time that we have received such a training. I was used to talk
about behavior change; with the training I know now what this is.” (MCH supervisor)
“One can talk about changes since we have received the two training sessions, because
before we were like abandoned in the field. One worked with what one knew; now, after
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
101
the training received, we notice that there are big changes that occurred at our level, at
the health agents level, and also at the beneficiaries level.” (MCH supervisor)
Confidence. The theme of confidence in their preparation and capacity to meet their
responsibilities was also present in all seven focus groups. Health agents and colvols voiced the
perceptions that they had the necessary information on child development and feeding and
special needs of pregnant women in a number of ways. They also mentioned the importance of
the recent training sessions in giving them confidence. The following examples provide insights
into how they perceived their own competence (self-efficacy):
“The workload increases from day to day. Luckily we have more training now, what
makes me strong enough to do the work. In my opinion, the program is a gift offered by
WV to the mothers. Thus, the supervisors and trainers train us.” (colvol)
“We feel prepared for the responsibilities that we have now in the program, especially
after the training we have received recently.” (colvol)
“We have already answered, we feel comfortable with our current responsibilities in the
program, once we have given our word, we don’t think again about problems.” (health
agent)
“Compared to the way the program had started, we feel that there is an evolution. I
think the program will attend its objectives because we got it on the right track. In
addition, with all the support we receive and will continue to receive, although the work
is huge, we will keep on going.” (MCH supervisor)
“Our personal satisfaction is that we like our work; we feel confident with our work.”
(Commodity supervisor)
6.4. Motivation and perceived value of the program
From the previous section, it is evident that, in general, the health staff have a positive attitude
about their responsibilities in the program. In this section, which describes the themes related to
job motivation and perceived value of the program, we also see the predominance of positive
views. Some of the themes related to job motivation that emerged from the data were (1)
enjoyment of work, (2) the contribution to the development of the community, and (3) the
contribution to behavior change in the community. Less frequently, the discussants voiced that
their professional status or the training they received was a motivating factor for them. A final
theme of motivation relates to the perceived value of the program by the health staff. In Table
6.2, an X indicates that the theme was present in the focus group transcript.
Table 6.2. Themes with respect to job motivation
Focus groups: Themes
Enjoyment
of work
Community
development
Behavior
change
Professional
status Training
Perceived
value of the
program
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
102
Colvols X XX XX
Colvols X XX X
Health agents X X X X
Health agents X X X
MCH supervisors X X X X
Enjoyment of work. Enjoyment of the work is an important dimension of job motivation, and
this was a theme that was reflected across all the focus groups with health staff. All the health
staff group discussants voiced positive sentiments about enjoying their work. Their statements
related to the positive aspects of their work included features such as enjoying working with
people, getting to know other communities, or receiving training. MCH supervisors noted that
community work was very demanding and required a strong commitment from their side. Below
are some examples of the ways they expressed these views:
“Yes, we enjoy our work a lot. Thanks to the program, there are a lot of areas and
people that I did not know before, and that I know now.” (colvol)
“If we would not enjoy our work, we would not have reached our current level in the
program.” (health agent)
“When I say that I am fine, this is because I enjoy my work, although my zone is a very
difficult one. It is not the money that motivates me, but the people.” (MCH supervisor)
“In general, this is not a work that one can do without enjoying it, because community
work is not limited to time; there is no specific time to start or to stop; one comes back
from the office and has to continue working at home in the evening. I repeat, if I am here
today, this is because I enjoy my work.” (MCH supervisor)
“I think there are several things that go together; one needs engagement, good will, and
love in order to be able to do community work because it’s not an easy task. If you don’t
like it, if you don’t enjoy your work, when people understand only with difficulties, if you
don’t be patient with them, you will never have a positive result, and you will fail. In
order to succeed, you need to enjoy what you are doing.” (MCH supervisor)
Community development and behavior change. The focus groups revealed that the contributions
they made with their work to the communities and to individual behavior change were
substantial motivating factors for the staff. In addition, this source of motivation provides us
with a sense of the commitment of the health staff to the well-being of the communities that they
are involved with, and is a very positive sign for the program. The following quotes from the
data illustrate some of the ways in which the contribution of community development and
behavior change to job motivation were expressed:
“Our motivation is the community development.” (colvol)
“When we see that malnutrition rates decrease, then this is extremely motivating for us to
do our work.” (colvol)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
103
“Help the people in my community changing their behaviors, their mentality, this is my
motivation.” (health agent)
“We like our community and it is our task to develop it.” (health agent)
“The children are the future of the community and it’s for them that we work.” (health
agent)
“The fact that I start observing changes in the behavior of people is a motivation source
for me.” (MCH supervisor)
Professional status. In the focus group discussions with health agents and MCH supervisors,
professional status was revealed as another factor for job motivation. The health agents
specifically voiced that their own conscience and their commitment motivated them to be correct
and honest in their work. Below are some examples of the ways this theme was expressed:
“It’s my professional conscience that motivates me.” (health agent)
“What motivates me are my will and my conscience. My conscience tells me that one
should not lie to people, that one should not give false information. If I give an
appointment, I must be the first to respect it.” (health agent)
Training. This theme appeared as a motivating factor in the discussions with the two colvols’
focus groups. The following quotes illustrate the ways in which the contribution of the received
training to job motivation was voiced:
“It is the training that we have received that motivates us.” (colvol)
“The fact that we have received training motivates us to do our job in order to achieve a
behavior change of the people in the community.” (colvol)
These and other statements, quoted earlier in this section, convey high levels of work motivation
of health staff, primarily derived from their desire to help others, their sense of making a
difference through their work, and their enjoyment of the job.
Perceived value of the program. A final theme that also should be highlighted relates to the
perceptions of the health staff concerning the value of the program for the community. Some of
the views about the benefits of the program for the communities describe the satisfaction that the
health staff got from seeing behavior change in their communities as well as improvement in
malnourished children. The following excerpts from focus group discussions provide
illustrations of their views that their work and the program are worthwhile and beneficial for
participants and the community:
“These past years, we were sad seeing certain mothers who had behind them
malnourished children. Today, we can’t say any longer that this is the case; I only can
love the work I do because we played an important role in these changes.” (colvol)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
104
“Given the responsibilities we have now in the program, we can say that a lot of efforts
are undertaken; there are changes in the community compared to how things were
before; one step forward has been made. That gives us energy; it motivates and
stimulates us to do the work because we feel that it is not for nothing that we work; we
will be able to find the expected results. I am investing myself voluntarily, given my
observations because before, more precisely in January when I just had started with the
program, I felt an enormous need of the people. Now I see changes; I understand that
the work I do is not for nothing; I am satisfied even if it is not at 100 percent, because I
feel that things move.” (MCH supervisor)
“When the program makes one step, I have to do two, because when I arrived in my area,
I realized that the mothers [in the program] didn’t want to do exclusive breastfeeding;
today, these mothers are encouraging other mothers in practicing exclusive
breastfeeding. They compliment us because their children don’t get sick any more. In
one word, the program has done a lot and I am happy and proud because thanks to our
work in the communities, one can feel a huge engagement of the mothers to exclusive
breastfeeding. In one word, the program works well.” (MCH supervisor)
The significance of these views should not be underestimated. It has been shown in another
context that when first-level workers perceive the program they are working for to be of value in
their communities, they are more highly motivated and gain more satisfaction from their work
(Dickin 2003).
6.5. Constraints to job satisfaction
The picture we draw of the WV staff based on the operations research data presented in the two
previous sections is the following: the staff in general, and especially the health staff, are well
motivated, have a positive attitude toward the program, and take their responsibilities seriously.
However, the research also identified some features that have negative influences on general
satisfaction, and in this section of the report we will outline these findings.
In the transcripts of discussions with colvols and MCH supervisors, a theme of occasional
feelings of discouragement and dissatisfaction appeared. The following statements are
examples:
“Sometimes I feel discouraged; I don’t find any more the power to do the work. From
the moment you work for WV, it is inevitable that a moment comes when you want to
leave. You do the work with passion, but sometimes it is just too much.” (MCH
supervisor)
“When you do work that you are not supposed to do, and you then realize that no effort is
made to facilitate your tasks, this is frustrating.” (colvol)
“When we started with the work, our supervisors gave us the guarantee that work
conditions will change, that things will become much easier; so far, nothing has
happened.” (colvol)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
105
“We work at least as much as the health agents; why do they always benefit when there is
a little improvement.” (colvol)
The themes we identified that led to this type of discouragement or dissatisfaction can be
characterized under six headings: (1) lack of respect from supervisors and higher management
levels; (2) low salaries and problems with payment arrangements; (3) lack of transport and
supplies to facilitate their work; (4) responsibility for too many activities; (5) weak planning and
coordination between the different levels in the program; and (6) problems due to poor roads and
transport. In Table 6.3, an X indicates that the theme was present in the focus group transcripts.
Another theme related to discouragement appeared only in the discussions with the MCH and
commodity supervisors and higher management level staff. These are concerned issues of
administration and the constraints to planning that administrative problems presented.
Table 6.3. Themes with respect of external constraints to satisfaction
Focus groups: Themes
Lack
of
respect
Low salary and
problems with
payment
arrangements
Lack of
transport/
supplies
Too many
activities/
workload
Weak
planning and
coordination
Roads and
transport
problems
Colvols X X XX X
Colvols X X XX
Health agents X X X X X
Health agents X X X
MCH supervisors X X
Food monitors X
Commodity supervisors X X
Lack of respect. In the discussions in both health agents’ and colvols’ focus groups, it emerged
that the staff often felt that they were not respected by the WV management, including
supervisors. This theme was voiced especially in one of the colvols’ focus group discussions and
one of the health agents’ focus groups. The feeling that they were not adequately respected even
led to occasional thoughts about leaving their jobs. The following statements illustrate the ways
in which the health agents and colvols expressed their frustrations related to respect:
“Each time a colvol leaves the program, you ask yourself why you don’t do the same,
since you are not considered.” (colvol)
“Lack of respect—the other day we had to go to Hinche and the office had sent a dirty
car to pick us up, and we had to sit on the ground.” (colvol)
“When we consider the way we are treated, as WV employees this upsets us. For
instance, when a WV employee passes in a WV car, finds us on his way and does not take
us, then this is really frustrating.” (health agent)
“We like our work and this is true, but when we consider the way we are treated, it is
frustrating. Educating people is an enormous work; nevertheless, we are considered less
important than employees at higher levels.” (health agent)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
106
Low salary and problems with payment. Adequate compensation for work done is a key
component of job satisfaction, and as shown in Table 6.3, the theme of low salary and problems
with payment appeared in all the transcripts of sessions with health agents and colvols. Overall,
it appears that this group of staff, in particular, feels under-compensated for their efforts. In
addition to the low salary, however, a lack of timeliness in payments was also an issue that
emerged as a source of dissatisfaction among these staff members. Some examples of their
frustrations related to their salaries are provided below:
“The salary is too low. I have a family with 3 children and I don’t manage to fulfill all
my responsibilities; that’s my biggest problem.” (colvol)
“There is no fixed day for the payment of our salary, but we have the dates of all
activities in our agenda.” (colvol)
“My problem is that I was working in the commercial sector and things were not too
difficult economically; since I am working as colvol, I don’t manage any longer to satisfy
the needs of my family; the amount that we receive as salary is clearly way below our
expenditures and the worst is that we are not paid in a timely manner. There can be
differences up to 15 days in the payment dates and sometimes even one month.
Sometimes I feel very discouraged; I even consider to quit the work.” (colvol)
“When I see that I have to pay about $40 in order to come and pick up the small salary,
then this is hard. There will be only $100 left; you can’t do anything with this amount.”
(colvol)
“The work increases from day to day, but the salary is not adjusted accordingly.” (health
agent)
“The work takes all our time and we are nearly not paid for it.” (health agent)
“When you work for an organization and you don’t have a contract, this is like you were
not working, because you never know exactly for how long you are going to be there.”
(health agent)
Lack of transport and supplies. The theme of problems with transportation and facilities
emerged in six of the seven focus groups. Only food monitors did not express this concern.
Health agents and colvols mentioned the transport problem, especially in relation to the distances
between the areas they were covering, the organization of RPs when they need to pick up
vaccines at the WV office or at a dispensary before starting the RP, and the home visits in remote
areas. The MCH and commodity supervisors expressed their concerns about the limited number
of cars available that constrained their supervision activities. Below are some examples of ways
the transport constraints were described:
“The problem that bothers us most is when you have to organize a RP in a very remote
area compared to the area where you live and you have to walk the whole day with the
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
107
thermos on your back before you arrive. It is indispensable to have transport facility
(mule).” (colvol)
“We don’t have transport facilities and the areas are very distant from each other; it
happens that we leave our house at 5:00 a.m. and come back late in the evening.” (health
agent)
“Our major problem is the lack of transport facilities. We have three cars and sometimes
all three break down the same day, which paralyses all activities that day.” (MCH
supervisor)
“There are also logistical problems. You want to supervise a food monitor and you have
to go in the same car with him to the distribution point; you can’t find a car and you are
supposed to be at all the distribution points.” (commodity supervisor)
In the discussions with the members of the health and commodity teams, the problem of supplies
was also a theme. Some supplies were not available at all or in insufficient amounts. The
supplies that were mentioned as insufficient in amount included: ice for conservation of
vaccines, health cards, vaccines, scales, material for pre- and post-natal consultations, and
electricity. Supplies that were identified as not available at all were bags for documents,
raincoats, megaphones, and communication equipment for the cars.
Workload. Another theme related to constraints to satisfaction was workload. As described
above in Section 6.3, high workload, in and of itself, was not necessarily viewed negatively.
However, ways in which workload had negative effects also emerged in the analysis of the
transcripts. Health agents and MCH supervisors complained about being responsible for too
many areas and too many activities. The MCH supervisors were concerned that their supervision
responsibilities with health agents and colvols was suffering because of their many other
activities, particularly for mobile clinics and pre- and post-natal consultations.
The negative sentiments related to staff workload can be characterized as discouragement due to
the increased workload, mainly because the increase left no time for personal activities. Initially,
colvols as community volunteers were only supposed to help the program for 15 working days
per month. They were compensated with a small remuneration for these services, attaining 30
percent of the salary of a health agent. Typically, they had other income-generating activities
during the remaining days of the months. The following statements testify to their view that they
are currently working full-time for the program and no longer see themselves as volunteers:
“Now we sacrifice all our time to the work; we don’t have time any longer for our
families, our agricultural activities, and our animals. Because of that, we suffer a lot;
this is true for several people. We only have the Saturday for us, Sunday is for God.”
(colvol)
“Last year, colvols were asked to be available only for 12 working days per month. I
attended MCs meetings but just to listen; now, it is rare that I have one day per month for
my personal activities, given my full agenda. I have two communities under my
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
108
responsibility; there a MCs to organize, home visits, I have four vaccination posts.”
(colvol)
“The colvol works as much as the health agent; if yesterday one could talk about
community volunteers, I think that today one has to change wording, given the amount of
work that we have. The supervisors don’t even think that we could get sick one day; even
the Sunday we work sometimes. One cannot talk about volunteers today; we have too
much work.” (colvol)
Health agents also complained about less time for their own activities:
“The work has become much harder, but also more interesting, especially with the new
methods. We now have more responsibilities and less time for our own activities.”
(health agent)
“Work has become so demanding that we don’t find time for our personal activities.”
(health agent)
Planning and coordination. Problems in planning and coordination between levels in the
program emerged as a theme in two focus groups. The health agents and colvols felt that
supervisors and higher management levels were not considerate of their own planning of
activities in the field, even though they routinely submitted their work calendars to their
supervisors. A further problem was that the regional office often planned activities for days on
which they already had a fixed agenda. They felt that there were weaknesses in program
planning due to improvised meetings and frequent changes in the MCH program itself. The
following examples illustrate these concerns:
“This happened to me once. I had a meeting with the mothers in order to cook one
recipe; the mothers had brought all the ingredients and in the moment we wanted to start
with the cooking, the office in Hinche asked me to come. I was embarrassed and didn’t
know what to do. Finally, I decided to go to Hinche.” (colvol)
“Many say that there are always improvised meetings; this proves that there is no
planning.” (colvol)
“We have submitted our monthly calendar to the office in Hinche, but it is not
respected.” (health agent)
“We can have a MC scheduled for a certain day and the office calls in order to give us
another work to do. That puts us sometimes in a difficult situation in front of the
mothers; we then have to go later and apologize to the mothers about that.” (health
agent)
“The program changes from day to day; it seems that even the people who designed the
program don’t understand what they do.” (health agent)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
109
Roads and transport problems. In contrast to other staff, the themes described above did not
emerge from the interviews with food monitors. Their concerns related primarily to late arrival
of food or non-availability of food at the distribution points, due to the general road and transport
conditions. Their description of these supply problems was sometimes accompanied by
expressions of discouragement, as in the following statement:
“The problem that we have is a problem which does not depend on us—natural disaster
and road infrastructure (rain, the state of the road); sometimes the truck can break down,
stick in mud, but we always create means to solve this kind of problem. One can be ready
to go to work and not be able to go there because of the bad road.” (Food monitor)
6.5.1. Suggestions from staff for ways to increase support
In all of the focus groups there was discussion about what could be done to increase support and
address problems they had identified. The following ideas and suggestions are presented with
the source of suggestion indicated in brackets:
ƒ Ensure respect from World Vision management:
- Official recognition of field staff as WV staff: badges, contract (health agent).
- More respect in general for staff in the field (colvols).
ƒ Provide adequate salaries and payment arrangements: Higher salaries paid on time (health
agents, colvols).
ƒ Provide transport facilities and supplies:
- Provision of transport facilities: mule, bicycle (health agents and colvols); motorbikes,
cars (MCH supervisors); trucks (food monitors).
- Provision of supplies: bags, megaphones, raincoats, communication equipment,
refrigerator for vaccines, scales, tensiometer, thermos, etc. (health agents, colvols, MCH
supervisors, food monitors).
ƒ Reduce workload: More staff (MCH supervisors).
ƒ Improve planning and coordination:
- Better coordination and information flow between different levels in the program; more
respect for field calendars (health agents, MCH supervisors).
- Good administration supporting program operations (Commodity supervisors).
ƒ Provide more technical support: More supervision, exchange with other organizations
implementing similar programs (MCH supervisors).
ƒ Provide more training and ongoing training on:
- Job responsibilities: reporting (health agents, colvols); organization of RPs (colvols);
organization of MC: (a) modules on infant and young child feeding (health agents,
colvols), (b) animation style (colvols), and (c) elaboration of learning sessions (MCH
supervisors); animation of community meetings (health agents); organization of pre- and
postnatal consultations (health agents, MCH supervisors).
- Technical issues: nutrition (health agents, colvols, MCH supervisors); immunization
(health agents, colvols); family planning (health agents, colvols, MCH supervisors); risk
pregnancies/techniques of childbirth (MCH supervisors); HIV/AIDS (MCH supervisors);
use of Moringa olifeira (MCH supervisors).
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
110
6.6. Supervision
The themes we identified in the interviews with health and commodity staff in relation to
supervision by direct superiors were very diverse and ranged from very positive comments to a
rather negative view of supervision. We classified the comments into several specific themes.
The first two reflect general features of supervision: (1) the feeling that supervision is
stimulating and motivating; and (2) the view that one received adequate technical guidance for
problem-solving, planning, and other technical issues. Descriptions of perceived level of support
were classified into (a) moderate (the feeling that one received some support from supervisors
but not to the extent expected), (b) the perception that no support was forthcoming from one’s
direct supervisor, and (c) the view that supervision was negative. In Table 6.4, an X indicates
that the specific theme or value was expressed in the transcript.
Table 6.4. Themes with respect to supervision
Focus groups: Themes
Motivating
Adequate
technical
guidance
Moderate
support
Non￾support Demoralizing
Colvols XX X
Colvols XX X X X
Health agents X X X
Health agents X X X
MCH supervisors X X
Food monitors X X
Commodity supervisors X X X X
Motivating supervision: In all seven focus groups, some of the discussants voiced the view that
supervision and the supervisors’ attitude were stimulating and motivating for them to take their
work more serious and thus, to do a better job. In addition, the health agents and colvols
expressed the feeling that their work was valued when they received supervision visits. Another
aspect was the effect that supervision visits had on the beneficiaries. Health agents and colvols
mentioned that supervision encouraged the participating mothers and noted that this resulted in a
higher participation in the program activities. The following statements illustrate the ways in
which the staff perceived the stimulating and motivating character of supervision:
“I feel safe when there is supervision; this allows me to take the work more serious; I feel
more responsible. When there is no supervision, one is tempted to let oneself go and to
look for easy ways, but if there is supervision, one feels more motivated.” (colvol)
“Our work is valued; the supervisors come to the field to visit us; they don’t stay in their
offices. That makes work important.” (colvol)
“The supervision does not only motivate us; it also motivates the mothers, because they
understand that what we do is not an isolated effort, but that there is a whole
organization behind our efforts to work for a better community.” (colvol)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
111
“The supervision allows us to make more efforts; the supervision motivates us. When
there is supervision, in case of a mistake, one knows it immediately and can correct it.”
(health agent)
“The way in which one makes the supervision is very good; one respects the principles.
The supervision is like a serum.” (health agent)
“The supervision is always a good thing, because it motivates us to do a better job.”
(health agent)
“The supervisors act with us like brothers.” (health agent)
“If I know that I will be supervised, I will prepare myself consequently.” (MCH
supervisor)
“For our part the supervision is very good; we feel at ease when one comes to supervise
us.” (food monitor)
“Without puffing ourselves up, we feel good with the methodology applied for the
supervision.” (Commodity supervisor)
Adequate technical guidance. Discussants in all groups except for the MCH supervisors brought
up the value of the technical support features of supervision. Health agents and colvols
expressed the view that the feedback provided during supervision helped them to improve their
performance. The staff expressed their appreciation for such supervisory support as helping
them when they forgot something in the education sessions, help in planning their work,
spending enough time with them, organizing meetings for all staff if they felt that a problem
experienced by one person was relevant to the whole group, etc. Below are quotes from the data
that reflect the ways in which this theme was expressed:
“Yes, I find enough support. Sometimes, one can forget something and then the
supervisor helps us; I think that this is a good thing.” (colvol)
“Yes, I receive support of the supervisor when I have a problem, because at each time I
am visited by the supervisor, he stays always very long with me in order to give me
advice in case I would have done a mistake.” (colvol)
“Yes, we receive support when we need to plan. When we need to plan, we contact our
supervisor, and together we do the planning.” (colvol)
“We receive support for the technical advice because when we receive the visit of a
supervisor, this one asks us a lot of questions and if we did not know some of the things,
the supervisor explains them to us.” (health agent)
“I feel always at ease when there is supervision because it helps me to make progress in
my work.” (health agent)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
112
“When there is supervision, we receive feedback on our mistakes.” (health agent)
“When we are facing difficulties, the supervisor can organize individual meetings or
group meetings if these concern the whole group; in these meetings one finds useful
advice being able to help us to accomplish our tasks, because one expects from us an
effective work; as soon as one needs an advice, one obtains it.” (food monitor)
“Concerning the support when one needs to plan or needs technical advice, we find
enough of it, even from the more senior food monitors. The support is not only provided
by supervisors, but also by mentor food monitors.” (food monitor)
“Yes, when we have problems, we receive enough support. For instance, I was at a food
distribution in Saltadere and I suddenly saw a car with my supervisor in it. As the
beneficiaries were not in order, the supervisor came and helped me to arrange them
carefully. I had difficulties to start with the distribution but with his arrival, everything
was solved.” (food monitor)
“I can say that in 99 percent of the cases, we find all support expected; sometimes he [the
supervisor] receives us even at his house.” (Commodity supervisor)
“The supervisor is a guide, a compass. I can also say that the supervisors are here to
give us technical guidance, to correct what is not correct. The supervisor knows our
weaknesses; he knows also our strengths. Although we function as friends, we respect
the hierarchy.” (Commodity supervisor)
Moderate support. Some statements were less positive about the support received from
supervisors. These statements indicated that individuals felt they were receiving some support
but not always when they needed it most. However, there was also recognition of circumstances
that interfered with the ability of supervisors to provide the level of support that individuals
desired:
“One has to say that it is not always possible to have the support from the supervisor
when one needs to plan, despite of the goodwill of this last because one can want to
prepare a learning session for a MC meeting and one solicits the assistance of the
supervisor, this one can give us an appointment for one day and this day coincides with
another planning.” (colvol)
“No, sometimes we pose the problem ourselves; we do not always find the support when
we need it most, but one has to recognize that sometimes we do find it.” (colvol)
“We receive support from supervision; we do not say much, but, nevertheless, we receive
some.” (health agent)
“The supervisor is limited, he has not the final decision. Example: The supervisor can
allocate transport facilities to each Commodity supervisor, but we have only three cars,
he cannot satisfy everyone.” (commodity supervisor)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
113
Nonsupport and demoralizing supervision. A positive appreciation of supervisory support is
implicit the preceding comments. In contrast, some of the comments clearly had a negative
dimension. This dimension was described as lack of support in case of problems, when staff
needed to plan, needed technical advice or wanted to have constructive feedback on their
performance. In one of the two colvols’ focus groups frustration was expressed about the
supervisor’s behavior, and supervision was described as demoralizing. The following examples
are illustrative:
“When one is sick one cannot say that one finds enough support because there are times,
as sick as one can be it is necessary to go to work.” (colvol)
“According to me, when a supervisor does a supervision visit, then this is in the intention
to demolish us, for only criticizing and degrading us, but not to correct our mistakes.”
(colvol)
“No, we do not find any support from the supervisors when we need to plan. If we do not
find any support when we need to plan, one should not even speak about technical
advice. We do not find any support when we need technical advice.” (health agent)
“I do not feel at ease because since I am working in the field I cannot remember having
received the visit of a supervisor who would have given me feedback, which would have
enabled me to know in what I have to improve.” (MCH supervisor)
“I feel that there is a deficit at the level of supervision.” (MCH supervisor)
“I am with WV since 1 year and 6 months, I received supervision visits when I was
working in the ADP, but since I am with the MCH program, I never received any
supervision.” (MCH supervisor)
“I never receive any support. Even if I receive a supervision visit, I never receive any
feedback. By lack of time, even when one needs technical support one does not find it.”
(MCH supervisor)
6.6.1. Suggestions from staff on ways to improve supervision
In all seven focus groups a number of suggestions were made about how to improve supervision.
Their suggestions are listed below. The focus groups in which the suggestion was present are
indicated in brackets:
ƒ Improve technical support:
(a) initiate monthly planning together with supervisor (health agents, colvols), (b) initiative
planning and preparation of MC activities as joint or collaborative effort with supervisor or
with other health agents (health agents, colvols), (c) designate a permanent person assigned
to give advice in their working areas (health agents, colvols), (d) institute on-going, recurrent
training activities (health agents, MCH supervisors, food monitors, Commodity supervisors);
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
114
ƒ Strengthen coordination and collaboration between different levels in the program:
(a) institute more dialogue and joint planning among colvols, health agents, and supervisors
(colvols, health agents), (b) create an atmosphere of respect for planning (health agents,
MCH supervisors), (c) assign two days in monthly calendars for unforeseen activities (health
agents), (d) initiate regular meetings of the health team (MCH supervisors);
ƒ Increase administrative support and logistics:
(a) insure that fuel is provided in a timely fashion (Commodity supervisors), (b) provide
laptop computers for report writing (Commodity supervisors), (c) obtain more trucks/cars for
the program (food monitors, Commodity supervisors); and
ƒ Reduce workload that interferes with supervision or establish a special team that is
dedicated exclusively to supervision (MCH supervisors).
6.7. Coordination between the health and food components of the program
In the discussions with the health and commodity staff, issues of coordination and collaboration
were raised. A number of statements emphasized the need for the two teams to work together for
the well-being of the beneficiaries. The addition of the food component was particularly singled
out as a development that was positive for the beneficiaries and had the additional effect of
encouraging participation in other program components. Both health and commodity staff
pointed out that the health and the food components of the program were complementary. Some
members of the commodity staff felt that there was perfect collaboration between the two
groups. However, this sentiment was not echoed by the health staff, who felt that the
coordination between the two groups was, in fact, quite poor.
Below are a few examples of the ways these views were expressed:
“It is necessary to work together for the wellbeing of the beneficiaries.” (colvol)
“It is necessary for the two teams to work in close collaboration because they are
complementary.” (health agent)
“I find that it [food distribution] is a good thing, because when there was no food
program people were less interested.” (health agent)
“The [food] monitors need to know that they have to work in collaboration with us.”
(health agent)
“There is a lack of collaboration between the two groups.” (health agent)
“MCH cannot do the work without acting in concert with Commodity.” (MCH
supervisor)
“There is no coordination between the [food] monitors and the health agents, between
the monitors and the beneficiaries.” (MCH supervisor)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
115
“Everything is good, I think that one [team] cannot work without the other, it is as if one
asks me which part in my body is the best?” (food monitor)
“These two components go together, one supplements the other.” (commodity
supervisor)
“The work is based on both teams, without the health team, the commodity team cannot
make any progress, one cannot function without the other. We could consider WV as a
tree, and we (Commodity and MCH) are the branches.” (commodity supervisor)
“I think that we are in perfect collaboration, because we find our objective in MCH.”
(commodity supervisor)
The more specific themes we identified in the interviews concerned both positive aspects of the
coordination and those negative aspects that, in fact, constrained good coordination. We
categorized the positive statements into three groups: (1) the value of exchange of ideas,
(2) joint preparation of the food distribution, and (3) joint transportation of staff. The themes
representing aspects of the coordination that were seen as negatively affecting the program were:
(1) insufficient communication and dialogue between the two teams, and (2) insufficient
collaboration in preparing the food distribution and during the distribution process. Finally, an
overarching theme that emerged in all interviews with health staff in relation to their work with
the commodity staff related to a perceived lack of respect from the food team for the health staff
and also for the beneficiaries. In Table 6.5, an X indicates in which focus group transcripts each
of these themes was present.
Table 6.5. Themes related to coordination and interactions between health and food staff
Themes
Positive aspects of the coordination Constraints to good coordination
Focus groups:
Exchange
of ideas
Joint
preparation of
food
distribution
Joint
transport
Insufficient
communication/
dialogue
Insufficient
collaboration in
distribution
preparation and
process
Lack of
respect
Colvols XX
Colvols XX
Health agents X X X
Health agents X X
MCH supervisors X X X X
Food monitors X
Commodity supervisors X
Positive aspects of coordination. Expressions of positive themes related to coordination were
exclusively voiced by members of the health team. The members of the food team did not note
any specific positive aspects of the coordination. Health agents expressed the view that the
exchange of ideas between the health and the food team was positive for the program. Members
of two focus groups (health agents and MCH supervisors) mentioned collaboration in preparing
the beneficiary lists for the food distribution and the joint planning of the distribution dates as
positive points of the coordination. In addition, colvols and MCH supervisors noted
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
116
collaboration between the two teams in helping each other out with transportation. The
following statements illustrate the ways the health staff expressed positive aspects of the
coordination between the health and food team:
“What is good for the program is the exchange of ideas that exists sometimes between the
MCH and commodity staff.” (health agent)
“The fact that it is MCH which prepare the lists and which Commodity is obliged to
respect.” (health agent)
“The fact that MCH and Commodity sit together to prepare the [distribution]
calendars.” (MCH supervisor)
“There is an improvement in coordination because now we can sit down with the people
of Commodity to plan the [distribution] calendars.” (MCH supervisor)
“The collaboration that exists between the Commodity and MCH staff is for example if
one must go somewhere, the [food] monitors can transport us to this place and bring us
back later.” (colvol)
“When there is a transport problem, one or the other [team] is always ready to help.”
(MCH supervisor)
Negative aspects of coordination. In all seven focus groups, at least one of the specific negative
themes concerning coordination surfaced during the discussion. Colvols and MCH supervisors
expressed views that a lack in communication and dialogue between the two teams had negative
effects on the program. Health agents and commodity staff both mentioned insufficient
collaboration during preparation for the food distribution and during the distribution process as a
problem. Health agents were concerned that there was insufficient collaboration in planning the
distribution dates. Commodity staff suggested that the health team often did not prepare the
beneficiary cards correctly, i.e., that they sometimes did not indicate the entry and exit dates of
beneficiaries on the cards, which is a mandatory prerequisite for receiving the food ration. Other
issues mentioned by the members of the commodity team in relation to insufficient collaboration
during the food distribution concerned absent beneficiaries, which sometimes meant that the
commodity team had to return large food quantities to the warehouse at the end of a distribution
day, and lack of understanding from the health staff when beneficiaries arrived late at the
distribution site, after the distribution is finished, and did not receive their ration. Below are
some examples of the ways in which these themes were expressed:
“There is no communication in this coordination, there should be a meeting between the
two groups so that they get to know each other better, one should know the other, if the
person does not know us, she/he can say anything to us, it is a business of big chief and
small chief, and I know that the small chiefs work in the same way than the big chiefs.”
(colvol)
“There is no communication between MCH and Commodity.” (MCH supervisor)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
117
“I don’t feel that there is enough collaboration, the [food] monitors don’t ask us when
they are in the field and when they need an advice and when we are there.” (MCH
supervisor)
“Commodity does not sit together with MCH for the planning of the distribution dates.”
(health agent)
“I have a doubt, I have the impression that these two groups do not get along too much,
like if there was no coordination. For example, we have an appointment for the food
distribution each second Tuesday of a month, everyone met and waited for the arrival of
the food—in vain—it is only the following morning while I was organizing a RP, without
informing us, the monitors arrive in order to distribute the food without even explaining
us why they did not come the day before; the beneficiaries were very unhappy.” (health
agent)
“It is when a beneficiary should receive his ration, and then, on her/his [beneficiary]
card neither the entry nor the exit dates are marked.” (food monitor)
“It is also when a smaller number [of beneficiaries] was served than what was planned,
and one has to return with an important food quantity.” (food monitor)
“Administrative problems, the information on the [beneficiary] cards are incomplete
(entry date, exit date), cases of absence.” (commodity supervisor)
“Sometimes, one can finish with the report, be in the process of packing everything
together when a person of the health staff comes with a beneficiary and asks to serve this
person, if one refuses, both call you a black animal.” (commodity supervisor)
Lack of respect. A key theme that that emerged in the discussions with the health team
concerned lack of respect. In contrast to Section 6.5, where lack of respect from the WV
management side was described, lack of respect in this section refers to the food team and
specifically to the food monitors. It should be noted that this theme was apparent in all the focus
group interviews, and was discussed at some length by the health staff, suggesting that it is, in
fact, a substantial source of distress. Some members of the health team openly expressed their
feelings related to the food monitors’ behavior and the lack of respect they exhibited in their
treatment of health agents and colvols. In addition, health agents suggested that the food team
did not handle foods hygienically, which they saw as a sign of lack of respect for the
beneficiaries. The following statements illustrate these sentiments:
“The colvols are there to serve the beneficiaries, and it is similar for the food monitors.
However, the monitors consider the colvols like dogs.” (colvol)
“No respect, we do not help the [food] monitors with unloading the food nor with
arranging the food because we feel that there is no respect. They are not all the same,
some are nice.” (colvol)
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
118
“There is sometimes a lack of respect from certain [food] monitors for health agents and
colvols, certain monitors believe that they are lords.” (colvol)
“The [food] monitors are gangsters, people who simply do a ‘job’ in order to make
money.” (health agent)
“People of Commodity behave like chiefs in front of the MCH people.” (health agent)
“Nowadays, the [food] monitors show slightly more respect for the health agents than in
former times.” (health agent)
“The way they [the food monitors] do the distribution, they put their hands in the flour
when distributing it, this is a health program, there needs to be a little bit more hygiene.”
(health agent)
“If one takes into account the way people of Commodity behave, one would not be there,
because they do not have any consideration for the beneficiaries. They deposit the food
on very dirty covers. In addition, we know the beneficiaries better than they do, if there
is a beneficiary who forgets her card, it is completely normal that one presents oneself to
explain to them that the latter meets the requirements and that she only has forgotten her
card, but that she can receive her ration; they always think that we do all this so that we
can share the food with the beneficiary.” (health agent)
“The [food] monitors behave very badly towards the health agents.” (MCH supervisor)
6.7.1. Suggestions for improving coordination between the health and food components of
the program
In all seven focus groups, a number of suggestions were made on how to alleviate the constraints
for good coordination between the health and the food team. These suggestions are listed below.
The focus groups in which the suggestion was present are indicated in brackets:
ƒ Increase dialogue between the two teams:
(a) schedule regular meetings between the two teams (health agents); (b) hold
meetings/workshops on activities, roles, and responsibilities to acquaint members of each
team with the work of the other component (health agents, MCH supervisors, food monitors,
Commodity supervisors); (c) hold meetings between the two teams after each food
distribution (Commodity supervisor), (d) communicate the information on the quantity of
food that is planned for each area in advance of the distribution (health agents); (e) institute
joint planning sessions (health agents, food monitors); (f) establish a system of verification of
the distribution lists by food monitors, colvols, and health agents on site before distribution
starts (colvols);
ƒ Encourage the two teams to make greater efforts to collaborate and appreciate their
complementarity (colvols, health agents, MCH supervisors, food monitors);
ƒ Establish a method for the food monitors to meet the beneficiaries more than once a month
(MCH supervisors);
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
119
ƒ Improve the organization of food distribution:
(a) establish separate distribution points for beneficiaries of different communities (health
agents, colvols), (b) construct small warehouses closer to the distribution sites (health
agents), (c) start distribution earlier in the morning (health agents), (d) provide the two direct
rations for one household at the same time to avoid long waiting times for certain
beneficiaries (health agents), (e) request help of the community for the unloading of trucks
(Commodity supervisors), (f) increase motivation of beneficiaries in order to minimize
absent cases (Commodity supervisor), (g) harmonize the distribution procedures between
different WV intervention areas (Commodity supervisor);
ƒ Encourage the teams to place program interests above the interests of their own respective
units (MCH supervisor);
ƒ Support the development of greater respect between the two teams:
(a) encourage greater respect on the part of the food monitors for health agents and colvols
(colvols, health agents), (b) institute training in adult learning for food monitors with a focus
on respect (colvols, MCH supervisors), (c) create badges for the health team (colvols, health
agents).
6.8. Summary of issues related to program organization and management
This section summarizes the findings with respect to organization and management of the
program.
Staff perceptions related to their responsibilities. A paramount finding of the operations
research on program organization and management is that, at all levels, staff feel that they are
engaged in important activities that are leading to improvement in the lives of the people the
program serves. They believe that their jobs entail serious responsibilities, and they express
confidence in their preparation and ability to meet these responsibilities.
Motivation and perceived value of the program. The conviction that they are contributing to the
development of the communities in which they work, and facilitating behavior change in
individuals and families, are important sources of motivation. Many also find the work itself
enjoyable. The acquisition of professional status and the value of training are also highlighted by
community-level staff as a source of job satisfaction and a factor in the development of their
capacities to carry out their jobs. Some of the respondents contrasted the current training
orientation with their previous experiences, emphasizing not only the new technical knowledge
they have acquired, but also that they have developed improved modes of interacting with their
clients (e.g., using adult learning principles in the design of their education activities).
At every level of the program delivery system, staff called attention to expansion in their
workloads. Various perspectives were expressed concerning the significance of this fact. While
some felt it to be a source of discouragement, this was not universally the case. One
consequence is that expanded work time puts greater pressure on the availability and
management of time for other personal obligations (e.g., in agriculture and other domestic
activities). For community volunteers (colvols), a significant source of tension vis-à-vis health
agents is that they see themselves as working as hard as the agents without appropriate financial
compensation.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
120
Constraints to job satisfaction. The operations research identified five issues that negatively
affect the satisfaction people experience in their jobs. For health agents and community
volunteers (colvols), these are: (1) lack of respect from supervisors and higher management
levels; (2) low salaries and problems with payment arrangements; (3) lack of transport and
supplies to facilitate their work; (4) responsibility for too many activities; and (5) weak planning
and coordination between the different levels in the program. MCH supervisors expressed
concern only about lack of transport and supplies to facilitate their work and responsibility for
too many activities. Commodity supervisors noted only lack of transport and supplies to
facilitate their work as a problem that affects their job satisfaction.
Supervision. At each level, staff were asked to reflect on their relationships with their direct
supervisors. At every level, from colvols to supervisors, positive comments on supervision
reflected two general features: (1) the feeling that supervision is stimulating and motivating; and
(2) the view that they received adequate technical guidance for problem-solving, planning, and
other technical issues. The value of technical guidance and appreciation for it was articulated in
all the focus groups, with the exception of the MCH supervisors, where this perception was not
brought up in discussion. On the other hand, some concerns were expressed about how
supportive supervisors were. At the opposite extreme from “stimulating and motivating” was the
view that supervision was demoralizing (expressed in one of the colvol focus group discussions).
“Moderate support” and “nonsupport,” intermediate positions on this dimension, were also
expressed.
Coordination between health and commodity sectors. With respect to coordination between the
health and commodity sectors, positive features of the relationship noted by respondents
emphasized the value of exchange of ideas, and the fact that there is joint preparation of the food
distribution, and joint transportation of staff. The themes representing aspects of the
coordination that were seen as negatively affecting the program were: (1) insufficient
communication and dialogue between the two teams, (2) insufficient collaboration in preparing
the food distribution and during the distribution process, and (3) lack of respect from the food
team for the health staff and also for the beneficiaries.
Staff suggestions to improve program support and supervision. In all of the focus groups,
respondents were asked to provide their ideas about how to increase support and address
problems. Section 6.5.1 (above) lists a variety of suggestions, ranging from providing field staff
with WV badges and transportation facilities to more training on job responsibilities and
technical health and nutrition issues.
In all of the focus groups, a number of suggestions were made about how to improve
supervision. As detailed in Section 6.6.1 (above), suggestions covered a range of matters under
the general topics of: (1) improving technical support, (2) strengthening coordination and
collaboration between different levels in the program, (3) improving logistics and administrative
support, and (4) shifting workloads to provide increased opportunities for supervision.
Finally, in all seven focus groups, a number of suggestions were made about how to alleviate the
constraints for good coordination between the health and the food team. As detailed in Section
6.7.1 (above), these range from activities to increase dialogue and collaboration between the two
teams to suggestions about how to improve mutual understanding and respect.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
121
7. FEEDBACK TO PROGRAM MANAGEMENT AND PLAN OF ACTION
DEVELOPED BY WORLD VISION TO IMPROVE SELECTED ASPECTS
OF PROGRAM OPERATIONS AND SERVICE DELIVERY
The results of this first round of operations research were presented by the IFPRI-CU team in
Haiti in February 2004. The general presentation of the results to PVOs and USAID staff was
followed by a 1.5 day meeting of the IFPRI-CU team and the WV-Haiti management staff. The
main objectives of the meeting were:
ƒ To review and discuss the findings of the operations research;
ƒ To discuss the constraints to implementation that were identified in our assessment and to
identify potential solutions to address these constraints and to strengthen program operations
and quality of service delivery;
ƒ To prioritize the constraints to be addressed and the potential solutions to be implemented,
and to develop an “action plan” for implementing the selected actions to strengthen the
program.
A matrix was used to guide the discussions and to facilitate prioritization of the constraints to be
addressed by the program in the short term (see Table 7.1). The matrix consists of five columns;
starting from the left-hand side, the first two columns list the operational constraints identified at
the different service delivery points and for different activities; the next column lists the potential
solutions identified by the management staff present at the meeting to address the different
constraints; the fourth column summarizes discussions regarding the feasibility of adopting the
proposed solution(s) in the short-term; and the fifth column provides an assessment of the
potential impact of addressing the different constraints identified on program operations as well
as on the overall impact of the program on its targeted beneficiaries.
The matrix summarizes the process utilized by the IFPRI-CU-WV team to review and prioritize
potential solutions to the constraints identified and to develop an action plan to implement future
corrective actions. The three criteria used for the prioritization of constraints to be addressed
were the following: (1) the possibility of identifying a solution (corrective measure) that was
within the scope of current program activities; (2) the feasibility of implementing these
corrective measures, given the program’s current financial and human resources; and (3) the
estimated impact of implementing these corrective measures to strengthen a particular aspect of
program operations on the overall program’s effectiveness and impact. This latter criterion is
important because it may be that for some aspects of the program, very simple and low-cost
solutions exist, which if implemented could have a major impact on operations and on the
effectiveness of the program. By contrast, efforts to improve other aspects may not warrant the
level of investment necessary to implement corrective actions because their overall potential to
improve the programs’ effectiveness is small. These aspects were carefully gauged in the
discussions held in Haiti.
A summary of agreements on the corrective actions to be implemented by the program at the
different delivery points is provided below and summarized in Table 7.1.
7.1. At the Rally Posts
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
122
The logistical constraints related to crowding, high participant/staff ratio, and time involvement
for both staff and participants at the RPs were discussed in length, but few solutions were
identified to solve these problems in the short term. Given that the program is already perceived
as having a large number of health agents and colvols, it was not considered feasible to increase
the number of staff at this time. It was also considered inappropriate to recommend a major
effort to split existing RPs into smaller ones, because the staff is already stretched to their
maximum, with health agents being responsible for up to nine RPs per month. It was agreed,
however, that the management would continue to examine the situation on a case-by-case basis
and that RPs that are clearly unmanageably large would be re-examined and split if deemed
possible.
A suggestion was made by some of the health agents interviewed during the operations research
to improve the training of colvols in order to increase the support they provide to the program.
The management staff indicated that this measure had already been implemented and that colvols
were now being included in all field staff training. The colvols had also recently been
“promoted” and their salary was raised from 30 percent of the health agents’ salary to 50 percent.
This change was accompanied by a change in title, whereby the colvols are now referred to as
“agents promoteurs de santé,” or health promoter agents.
It was suggested that improving the venue of the RPs with some basic furniture, such as chairs,
benches, and tables, could facilitate the logistics and organization of the RPs. The management
staff agreed to follow up on this aspect and felt that it was feasible to mobilize community
resources to improve the venue of the RPs.
Finally, the group revisited the program’s proposed sequence of activities at the RPs, which,
according to our operations research, was not particularly effective in preventing bottlenecks at
registration and long waiting times for participants. The group agreed on a revised sequence of
activities to be tested and implemented if effective (see Table S2, second row, third column, for
the new sequence). The main changes in the proposed sequence of activities included: (1)
giving a number to each participant as they arrive at the RP, (2) weighing the children before
they are registered, (3) evaluating the nutritional status and individual counseling of the mother
along with registration rather than with weighing, and (4) providing the deworming, vitamin A,
and ORS tablets as the last activity for participants (along with immunization), rather than
providing these at the time of registration.
It is hoped that these relatively simple modifications in the RP operations will help improve the
flow of participants through the different activities, and reduce the time burden for both health
staff and participants, making the RP generally more efficient and pleasant. Avoiding
bottlenecks may also help improve the quality of services if it allows the health staff to dedicate
slightly more time to each participant and if the overall environment is less crowded.
Education. The group agreed that it was worth pursuing efforts to increase the coverage of the
education sessions by offering more than one education session at the RPs. It was felt that this
could easily be achieved by ensuring that supervisors reinforce the importance of providing
additional education sessions to accommodate late arriving participants. There was a general
consensus, however, that the RPs were not the most suitable venue for effective education, given
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
123
the generally crowded, busy, and noisy environment and the number of activities taking place
simultaneously. It was therefore agreed that the education sessions at the RPs should be used to
deliver simple messages on topics of general interest, such as hygiene, immunization, or family
planning. The RPs were to be considered as a complementary opportunity to cover additional
topics and to reinforce some of the messages provided at the MCs (the main venue for the
education and BCC strategy). The team also agreed that supervisors needed to be more active in
promoting the use of the 12-month calendar of education topics to ensure that participants were
exposed to the complete set of topics planned by the program, which ultimately should help
increase their general knowledge of health, nutrition, and hygiene.
Supplies. The lack of supply of vitamins, ORS, deworming tablets, and vaccines at the RPs
appeared to be mostly due to a problem with the main providers (Ministry of Health, UNICEF,
etc.). It was therefore considered important that the program management continues to pursue
every effort to ensure constant supply of these important preventive health services, but no action
was recommended for field staff, because it was considered to be outside of their control. High
coverage of these preventive services is extremely important in the context of MCH programs as
they are key complementary strategies to improve child growth, health, and survival, and should
therefore continue to be prioritized by the program management.
Growth monitoring and promotion. Growth monitoring and promotion in the context of the WV
MCH program has two main purposes: (1) to identify malnourished children in the recuperative
program communities; and (2) to communicate with mothers (from both preventive and
recuperative groups) about their child’s nutritional status and growth and provide individual
counseling. For the first purpose, it is important to minimize measurement error in order to
reduce misclassification of children during the screening process. For the second purpose, i.e., to
communicate with the mother, it is important that the health staff spend some time with the
mother to inform her of the results of the weighing and nutritional status assessment, and to
provide her with specific advice regarding her child’s progress. Problems with both of these
aspects were identified through our operations research and the solutions proposed at the meeting
to address them are summarized below.
With respect to the measurement problems, again the group agreed that given the environment at
the RPs and the equipment used to measure children, a certain level of error is to be expected. It
was agreed, however, that retraining and increased supervision of the health staff will be used to
reduce measurement errors and misclassification problems. One simple recommendation, which,
according to the results of our operations research, could significantly reduce measurement error,
is to make sure that the child’s weight is recorded when the needle of the scale is stable (as
opposed to when it is moving significantly). Plans to retrain and improve supervision of the
plotting of children’s weight on the growth card were also agreed upon.
The discussions on process of GMP and the communication with mothers led to an agreement
that all mothers should be informed about the child’s weight, nutritional status, progress since
last weighing, and that a brief individual advice should be provided. The group was very clear
about the limitations of conducting GMP in the context of the RPs, and especially with regards to
the time that the health staff can allocate to individually counseling each mother about her
child’s growth. Given the crowded and noisy environment and the severe time constraints faced
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
124
by the health staff, it was agreed that the brief advice would consist of praising and encouraging
the mother if her child is growing well. If her child was not growing well, the advice would
consist primarily of a recommendation to follow up with other program activities and to adopt
the recommended practices discussed at the MCs (e.g., exclusive breastfeeding, use of enriched
porridges, etc.).
7.2. At the Mothers’ Clubs
Implementation of the new BCC strategy in 2003 required significant reorganization of the MCs,
especially in the preventive program group. This is because the strategy emphasizes the
importance of the timely delivery of education messages to mothers, based on the specific age
and developmental stage of their child. This approach is based on the premise that a much
greater impact on behavior change is expected if the intervention reaches the beneficiaries at
their best learning moment. Our findings showed that there were still some problems with the
reorganization of the clubs and that some mothers were in the wrong groups relative to the age of
their child, and thus likely to receive education messages that were not particularly timely and
relevant. Management staff agreed that improving this aspect of the MCs was feasible, since so
much had been achieved already, but that it would require strengthening supervision in the field.
The management staff emphasized the need to strengthen the training of the supervisors
themselves in both the rationale for the recommended structure of the clubs and how to
operationalize it. It was also agreed that one modification to the current system would be made
in the preventive group, which now uses two age groupings for the MCs for beneficiary children:
mothers of children 6-8 months and mothers of children 9-23 months. The change recommended
was to replace these two groups with the two groups that were originally planned: 6-11 months
and 12-23 months. These two groups are better balanced and include children with more similar
needs relative to child feeding practices. It was also agreed that the MCs with children 6-11
months would emphasize the differences in recommended practices for children 6-8 and 9-11
months of age regarding the number of meals, and the consistency and quantity of food.
Remarkable improvements in the quality of education delivered at the MCs were achieved since
health staff were trained in the implementation of the new BCC strategy in 2003. The
importance of effective supervision and feedback were emphasized during the meetings as being
key to maintaining the quality of the intervention, and even more importantly, to maintaining the
enthusiasm and motivation of the staff, and the interest of the beneficiaries.
7.3. At the Food Distribution Points
The logistical problems related to the distribution of the food, and especially the transport of
food and staff to the point of distribution, and the time burden of this activity for staff and
beneficiaries were discussed at length in the meeting. The commodity team is fully aware of the
problems highlighted by our operations research and many of the constraints identified have
already been addressed since our data were collected. The commodity staff have been actively
working on improving the logistics of the food distribution, including taking measures to
improve local storage facilities. Transport and other logistical problems, however, can never be
fully overcome in areas such as the Central Plateau, where road conditions are so bad and the
availability of fuel continues to be a major constraint. These problems are systemic to the
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
125
context in which this program operates. It is worth pursuing efforts to try to improve the
situation, but flexibility and patience will always be required by staff at all levels and by the
beneficiaries themselves.
Errors in the amount of food received by the beneficiaries are also difficult to avoid, but the
management staff recommended that the existing supervision system be strengthened. This
system consists of having the supervisors systematically verify the ration received by a
subsample of beneficiaries as they leave the RP. This system is clearly worth reinforcing to
minimize errors in food allocation, and consequently improve beneficiary satisfaction.
The communication difficulties that had been reported between the commodity staff and the
health staff and beneficiaries, respectively, were also well understood by the management staff,
who indicated that these cases had been addressed individually by the supervisors or regional
coordinators on a case-by-case basis. Continued supervision will be required to maintain good
communication among all staff and with the beneficiaries.
Finally, the problems of hygiene in the distribution of food commodities, which had been raised
in our operations research, had also been addressed in part by the program. Plastic sheets are
now being used and the suggestion to provide gloves to food handlers was perceived as feasible.
An additional suggestion was to provide a small compensation to the food handlers to motivate
them to do their work more carefully and more hygienically. The possibility of acquiring larger
containers with a tap to distribute the oil more hygienically was also discussed. It is important to
recognize that these proposed approaches to increasing hygiene in the food distribution process
will require identifying the financial means to acquire the materials proposed.
7.4. Final comments
The action plan described above to address some of the operational constraints identified through
our operations research relies almost entirely on a strong and effective supervision system. At
the time of the meeting, the management staff indicated that they had already been taking steps
to strengthen the supervision of the program at all levels, and the health team presented their
newly developed supervision plan for the region.
Our next round of operations research will be conducted in 2004, starting in June. This will
allow some time for the WV team to experiment with the implementation of the corrective
measures agreed upon. Our operations research will focus on assessing the effectiveness of
implementation of these corrective measures and on studying the supervision structure. An
additional objective of the next round of operations research will be to assess differences in
implementation between the two program intervention groups; understanding differences will
facilitate interpretation of the final impact evaluation findings.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
Table 7.1. Summary of discussions held in Haiti with World Vision staff on the operations research findings and follow-up
actions
A - RALLY POSTS (RPs)
Aspects/activities Constraint(s) identified
Potential corrective
action(s)
Feasibility of
implementing
corrective action(s)
Potential impact of
improving this aspect on
effectiveness and impact
of program
Increase number of RPs
Increase staff (AS/colvols)
Continue to work on a case￾by-case basis to split the RPs
that are really too big and can
be split
Not possible at this point;
health agents already have
too many RPs/month
Not feasible in short-term
because of program criteria
and lack of resources
Feasible; already being
done; should be continued
- Will reduce staff and
participants time burden
and frustrations
- May increase participation
at RPs, which, in turn, may
increase impact of program
- May increase quality of
services at RPs
Train colvols better so that
they can help the health agents
more
This is already being done;
colvols have been included
in all recent training; salary
has also been increased and
they are now referred to as
“health promoter agents”
Will increase assistance that
health agents receive and
reduce their time burden
Organization Too crowded, too many
beneficiaries; ratio of
participants/staff is too high
Improve venue of the RPs by
providing basic furniture
(chairs, benches, tables, etc.)
This will be done through
mobilization of community
resources
Could make the RPs more
efficient and more pleasant
for staff and participants
122
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
Aspects/activities Constraint(s) identified
Potential corrective
action(s)
Feasibility of
implementing
corrective action(s)
Potential impact of
improving this aspect on
effectiveness and impact
of program
Registration is a major
bottleneck; sequence of
activities varies a lot between
RPs and does not usually
follow the implementation plan
The following sequence was
recommended for the future:
1) give a number to each
participant
2) provide general education
session
3) weigh child, plot weight on
growth chart in health card
4) do general registration
(including register weight in
registry)
5) evaluate nutritional status;
6) inform and advise mother
(see below in GMP)
7) give required immunization,
deworming, Vitamin A, and
ORS
Feasible; the team has
discussed several possible
sequences and concluded
that this one was probably
the best choice; supervisors
will assist health agents/
colvols in implementing
and testing this sequence
See above
Specific activities (organization and quality)
1. Education Still mostly one session, first
thing in the morning and
therefore many participants
miss the session
Give at least two education
sessions and if needed, up to
three
Feasible; was done in some
of the RPs observed; the
supervisors will be key in
motivating the health staff to
comply with this
recommendation
Greater coverage of
participating mothers; this
may increase maternal
knowledge and reinforce
certain topics discussed at
the MCs, thereby
strengthening the BCC
strategy
123
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
Aspects/activities Constraint(s) identified
Potential corrective
action(s)
Feasibility of
implementing
corrective action(s)
Potential impact of
improving this aspect on
effectiveness and impact
of program
Quality of education is non￾optimal (messages tend to be
vague, little use of visual
materials, limited interaction
between staff and participants)
Monthly education topics
recommended by the program
are rarely used
May be difficult to improve,
given crowded, busy, noisy
environment; may be best to
focus on simple topics and
messages of general interest
(e.g., ORS, immunization,
family planning, etc.)
Ensure that the calendar of
topics is distributed to all
supervisors; also ensure that
supervisors work closely with
health staff during their
monthly meetings to reinforce
adherence to monthly schedule
of topics.
Feasible to focus on simple
topics, ensuring that
information is accurate and
messages are clear and
simple; supervisors will
have to monitor this.
Feasible; requires greater
communication between
health staff and their
supervisors and monitoring
of implementation of the
calendar of topics
Following the 12-month
calendar of education topic
will allow mothers to
receive education on the
full set of topics considered
important and of general
interest to mothers by the
program. This can increase
their general knowledge of
health, nutrition and
hygiene.
2. Vitamin A/deworming/ORS Many participants do not
receive the services and this
seems to be due largely to a
lack of supply of the products
This problem is due to lack of
availability from the main
providers (Ministry of Health,
UNICEF, etc.)
This problem is outside of
the program’s control, but
efforts should be pursued to
ensure a constant supply
These preventive health
services are very important
for child’s health, growth,
development and survival.
Therefore it is important to
achieve high coverage.
124
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
Aspects/activities Constraint(s) identified
Potential corrective
action(s)
Feasibility of
implementing
corrective action(s)
Potential impact of
improving this aspect on
effectiveness and impact
of program
Children are weighed, their
weight is recorded, and their
nutritional status assessed, but
there is little use of the
information to communicate
with the mother and to give her
tailored individual advice
regarding her child’s growth
and progress
Retrain personnel to ensure
that each mother receives the
following information:
1) Child’s weight
2) Child’s nutritional status
3) Child’s progress (gained or
lost weight)
4) Brief message of
encouragement (if child is
growing well) and
recommendations for
follow-up with other
program activities (e.g.,
MCs) if child is not growing
well. If child is M3, make
sure s/he is scheduled for a
home visit.
Feasible; will require that
supervisors motivate staff
and monitor that these
actions are implemented
The information provided
to mothers about their
child’s nutritional status
and growth can help raise
awareness among mothers
and motivate them to
engage more actively in the
BCC activities and to adopt
the recommended practices.
3. Growth monitoring and
promotion
The quality of the weighing and
plotting varies; errors result in
the misclassification of some
children into the different
Gomez categories, and thus in
errors in screening beneficiary
children (true only for the
recuperative group, which uses
children’s nutritional status as a
screening criteria)
Retrain health staff to improve
the quality of the weighing
(making sure that needle of the
hanging scale is still before
recording the weight will
help).
Retrain staff to improve their
skills in plotting the child’s
weight in the growth chart.
Very feasible; can be
achieved through good
training and supervision
Improving the weighing
and plotting of the weights
in the growth chart will
reduce misclassification
errors, thereby improving
the cost-effectiveness of the
program.
4. Immunization Lack of supplies (similar to the
problems of vitamin A, ORS,
and deworming tablets)
See above (vitamin A, ORS,
and deworming)
See above (vitamin A, ORS,
and deworming)
See above (vitamin A,
ORS, and deworming)
125
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
B- MOTHERS’ CLUBS (MCs)
Aspects/activities Constraint(s) identified
Potential corrective
action(s)
Feasibility of
implementing corrective
action(s)
Potential impact of
improving this aspect on
effectiveness and impact
of program
Organization Problems with reorganization
of MCs for different subgroups
of mothers/children (especially
in preventive group)
Ensure that supervisors
understand the rationale for the
reorganization of the clubs, and
continue to reinforce importance
of implementing this
reorganization as planned.
Slightly modify current system,
which is to form two groups
with 6-23 month old (in
preventive group): 6-8 and 9-23.
Replace with the following two
groups: 6-11 and 12-23.
Very feasible, but requires
close supervision to ensure
that groups are formed as
recommended by the
program and that mothers
attend the groups they are
assigned to.
The importance of the timely
delivery of the education for
the effectiveness of the BCC
cannot be overemphasized. A
much greater impact on
behavior change is expected if
the intervention reaches the
beneficiaries at their best
learning moment.
Achievement of behavior
change may, in turn, reduce
childhood malnutrition.
Quality of the education and
BCC intervention
Very good quality overall;
marked improvements since
development of BCC strategy
and training of staff;
performance varies, however,
and continued improvements
can still be achieved
Strengthen supervision and
ongoing training of staff;
strengthen the communication of
concepts and technical content
of the sessions, as well as the
utilization of the communication
material and the adult education
techniques. Continue to use the
supervision checklists to provide
feedback to the staff and to
monitor progress.
Very feasible; is already
being done; important to
continue to strengthen the
supervision and feedback
provided to the staff.
Supervisors also need to
monitor that enthusiasm and
motivation of the staff, and
interest of beneficiaries are
maintained over time.
The greater the quality of the
BCC in MCs, the greater the
potential impact of the
program on improved child
feeding and care and
consequently on children’s
nutritional status, health, and
development.
126
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
C - FOOD DISTRIBUTION POINTS (FDPs)
Aspects/activities Constraint(s) identified
Potential corrective
action(s)
Feasibility of
implementing corrective
action(s)
Potential impact of
improving this aspect on
effectiveness and impact
of program
Very large number of
beneficiaries per distribution
point and high
beneficiary/staff ratios
Improving the logistics of
food distribution could reduce
the staff and beneficiary time
burden and increase
satisfaction.
Important delays in arrival of
food and staff at FDP
Organization
Very high time burden for
beneficiaries (including travel
time and time spent at the
FDP) and staff
Some organizational aspects
have already been addressed
by the program and some are
in the process of being
resolved; the staff makes
every effort to improve the
logistics of the food
distribution process, including
improving local storage
facilities.
Difficult to solve completely,
because most logistical
problems are inherent to the
rough environmental
conditions of the area.
Amount of food received by
beneficiaries is not always the
amount allocated by the
program
A system is already in place
to verify that the quantity of
food received by the
beneficiaries corresponds to
the amount allocated.
Feasible; system is already in
place. Need to ensure that
check is done systematically
on a sample of beneficiaries
as they exit the FDP.
Could increase beneficiary
satisfaction and reduce food
losses.
Quality
Communication difficulties
have been reported between
commodity and health staff
and commodity staff and
beneficiaries
These problems are addressed
on an ongoing basis, as they
occur, by the supervisors or
regional coordinators.
Very possible, already being
addressed.
Could improve staff
satisfaction with their work
and beneficiary satisfaction
with the program.
Certain problems of hygiene
in the manipulation of the
food during the distribution
process have been identified
Suggestions made to provide
gloves to food handlers and
small compensation to
motivate them to adopt more
hygienic practices. Use of
plastic sheets and cleaning of
measurement tools have
already been adopted.
Possibility of acquiring large
containers with a tap for the
distribution of oil was also
recommended.
Feasible, but requires
financial means to acquire the
materials proposed.
Would reduce the risk of
contamination of the food.
127
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
132
REFERENCES
Cogill, B. Anthropometric Indicators Measurement Guide. Washington, D.C.: FANTA, AED,
2003.
Dickin, K. L. “The Work Context of Community Nutrition Educators: Relevance to Work
Attitudes and Program Outcomes.” Doctoral dissertation. Ithaca, N.Y.: Faculty of the Graduate
School, Cornell University, May 2003.
Loechl, C., P. Menon, G. Pelto, and M. Ruel. Behavior Change Communication to Improve
Infant and Young Child Feeding Practices in Rural Haiti: Training and Communication
Materials. Washington, D.C.: FANTA, AED, 2003.
Loechl, C., P. Menon, M. T. Ruel, and G. Pelto. Process Used for the Design of an Integrated
Health and Nutrition Program to Prevent Child Malnutrition in Rural Haiti. Washington, D.C.:
FANTA, AED, 2003.
Menon, P. and M. T. Ruel. Childcare, Nutrition, and Health in the Central Plateau of Haiti:
The Role of Community, Household, and Caregiver Resources. Report of the IFPRI-Cornell
World Vision Baseline Survey, Haiti 2002. Washington, D.C.: FANTA, AED, 2003.
Menon, P., M. Ruel, G. Pelto, Y.-F. Pierre, E. Metellus, and A. Ferrus. A Qualitative Study of
the Patterns of Infant Feeding and Care in the Hinche Area of Plateau Central. Washington,
D.C.: FANTA, AED, 2002.
Menon, P., C. Loechl, G. Pelto, and M. Ruel. Development of a Behavior Change
Communications Program to Prevent Malnutrition in the Central Plateau of Haiti: Results and
Challenges from a Formative Research Study. Washington, D.C.: FANTA, AED, 2002.
Ruel, M. T. The Natural History of Growth Failure: Importance of Intrauterine and Postnatal
Periods. Martorell, R. and F. Haschke, editors. Philadelphia, PA: Nestec Ltd., Vevey/Lippincott
Williams & Wilkins, 2001, p. 123-158.
WHO. Measurement of Nutritional Impact. Geneva, Switzerland: WHO, 1979.
WHO. WHO Technical Report Series 854 Physical Status: The Use and Interpretation of
Anthropometry. Geneva, Switzerland: WHO, 1995.
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
133
Annex 1. List of Clusters
Cluster # Zone Commune Preventive (P) or Recuperative (R)
1 Marmont Hinche R
2 Madame Brun Hinche P
3 Bassin Zim Hinche P
4 Marialapa Hinche R
5 Cherival Hinche P
6 Pablocal Hinche P
7 Bintourib Hinche R
8 Carrefour Ledans Hinche R
9 Casse Thomonde and Lascahobas P
10 Pareidon I Lascahobas R
11 Pareidon II Lascahobas P
12 Salmadere Lascahobas R
13 Tierra I Thomonde R
14 Laloimassouse Thomonde P
15 Locaret Thomonde R
16 Locorbe Thomonde R
17 Tierra II Thomonde P
18 Ananas Thomonde and Lascahobas P
19 Rode/Beganabe Hinche R
20 Wanniqueter/Moruque Hinche P
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
Annex 2. Protocol for the Operations Research
No. Sample Module Method
PLANNED
sample size
ACHIEVED
sample size Sampling
General Data Collection
Form
Observation + Consulting
health agents daily
reports
20 19
General Exit Interview Structured individual
interview
2-3 per RP
Total: 40-60
59
Respondent Tracking Form Observation + structured
individual interview
2 per RP
Total: 40
38
1 Rally Posts (RP)
Interview with Health
Agent
Semi-structured
individual interview
20 19
1 RP per health agent in 19 clusters of
the evaluation project area
Observation of Mothers’
Club
Observation 20 20
General Exit Interview Structured individual
interview
2-3 per MC
Total: 40-60
41 2 Mothers’ Clubs (MC)
Interview with Health
Agent
Semi-structured
individual interview
20 20
1 Mothers Club per health agent in all
20 clusters: 5 MCs with pregnant
women, 5 MCs with lactating
mothers, 6 MCs with mothers of 6-24
months old children, 4 MCs with
mothers of malnourished children)
General Data Collection
Form
Observation + Structured
individual interview of
health agents/food
monitors
10 10
General Exit Interview Structured individual
interview
4-5 per FDP
Total: 40-50
45 3 Food distribution points (FDP)
Interview with Health
Agent
Semi-structured
individual interview
20 20
The 10 distribution points are
covering all 20 clusters of the
evaluation project area
4 Program beneficiary
women
Interviews with program
beneficiary women
Semi-structured
individual interview
30 Total of 30:
5 pregnant women
10 lactating mothers
9 mothers of 6-23 month
children
6 mothers of malnourished
children
Selection of women at Mothers’
Clubs in 15 clusters
5
Colvols/health
agents/health
supervisors/
food monitors/
commodity
supervisors
Focus group discussions
with program staff
Focus groups 2 groups with health
agents
2 groups with colvols
1 group with
supervisors
2 groups with food
monitors
2 groups with health agents
2 groups with colvols
1 group with MCH
supervisors
1 group with food monitors
1 group with commodity
supervisors
The program staff involved is
working in the 20 clusters of the
evaluation project area
130
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
No. Sample Module Method
PLANNED
sample size
ACHIEVED
sample size Sampling
6
Health coordinators
and commodity
officer
Interview with the regional
and national health
coordinators and regional
commodity officer
Semi-structured
individual interview
3 3 The program staff involved is
covering either the Central Plateau
(regional) or all World Vision
intervention areas in Haiti (national)
131
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
136
Annex 3. Details of the Different Focus Groups
Participants
Location of
focus group
Number of
participants Remarks
Health agents Casse 7 Health agents of the evaluation project area
in the communes of Thomonde and
Lascahobas
Health agents Hinche 9 Health agents of the evaluation project area
in the commune of Hinche
Colvols Casse 11 Colvols of the evaluation project area in the
communes of Thomonde and Lascahobas
Colvols Hinche 9 Colvols of the evaluation project area in the
commune of Hinche
MCH
supervisors
Hinche 6 MCH supervisor nurses of the evaluation
project area and the assistant of the regional
health coordinator
Food monitors Hinche 9 Food monitors covering the evaluation
project area
Commodity
supervisors
Hinche 6 Commodity field supervisors, warehouse
coordinator, CTS coordinator, and assistant
of the regional commodity officer
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
137
Annex 4a. Schedule of Learning Sessions at Mothers’ Clubs for Pregnant and Lactating
Women (Preventive and Recuperative Program Groups)
Month of
pregnancy Mothers’ Clubs for pregnant women
5 Other topics: Diet for pregnant women
6 Other topics: Dangerous signs during pregnancy
7 Other topics: Preparation of child delivery
8 Session 1
Importance of breastfeeding (initiation of breastfeeding, exclusive breastfeeding, continue
breastfeeding until 2 years of age or beyond)
Discouragement of bottle use
Comparison of recommendations with local beliefs and practices
9 Session 2
Initiation of breastfeeding, importance of colostrum
Exclusive breastfeeding until 6 months
Position and attachment of the baby during feeding, frequency of breastfeeding
Child age
(months) Mothers’ Clubs for lactating women
1 Session 3
Sharing experience with exclusive breastfeeding
Review of exclusive breastfeeding
Review of position and attachment of the child during feeding, frequency of breastfeeding and care of
nipples and breasts
Expression of breast milk
Drinking water while breastfeeding
2 Session 4
Sharing experiences related to drinking water while breastfeeding and expression of breast milk
Sharing experience related to exclusive breastfeeding
Discussing constraints/problems related to exclusive breastfeeding and offering solutions
3 Session 5
Exclusive breastfeeding and Lactational Amenorrhea Method (LAM)
4 Session 6
Introduction of complementary foods when children are about 6 months old
Importance of continued breastfeeding until 2 years of age or beyond
5 Session 7
Overview on child development and feeding chart (for children 6-11 months of age: food consistency,
participating in feeding, frequency, quantity of food)
Learning how to eat
Important information about the first food (in addition to breast milk) given to children
Preparation of the next session: preparing nutritious foods
6 Session 8
Preparing nutritious foods/cooking session
Tasting and discussion
Repetition of learning how to eat
Session 7
Preparation of the next session: preparing nutritious foods
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
138
Annex 4b. Schedule of Learning Sessions at Mothers’ Clubs for Mothers of 6 to 23 Month of
Age Children (Preventive Program Group)
Month in the
program Mothers of 6-to-23-month-old children
1 Session 8
Preparing nutritious foods/cooking session
Tasting and discussion
Repetition of learning how to eat
2 Session 9
Repetition on child development and feeding chart (for children 0 to 11 months of age)
Helping children to eat
Feeding during and after illness
Preparation of the next session: variety of food
3 Session 10
Sharing experience with one new feeding practice (related to helping children to eat)
Variety of food
Sharing experience with preparing nutritious foods at home
4 Session 11
Hygiene in food preparation, handling and storage—Diarrhea prevention
Feeding during and after illness
5 Session 12
Child development and feeding chart (for children 12-23 months of age)
Discussing food variety issues (special complementary foods, fruits and vegetables, vitamin A-rich
foods, animal foods, evening meal)
Preparing a creative way to communicate one feeding recommendation of the child development and
feeding chart
6 Session 13
Causes of malnutrition
Different types of malnutrition
Recuperation of moderately malnourished children
7 Other topics: Diarrhea
8 Other topics: Immunization
9 Other topics: Hygiene
10 Other topics: Use of Moringa olifeira
11 Other topics: HIV/AIDS
12 Other topics: Family Planning
13 Other topics: Home Gardening
14 Session 12
Child development and feeding chart (for children 12-23 months of age)
Discussing food variety issues (special complementary foods, fruits and vegetables, vitamin A-rich
foods, animal foods, evening meal)
Preparing a creative way to communicate one feeding recommendation of the child development and
feeding chart
15 Other topics: HIV/AIDS
16 Other topics: Family Planning
17 Other topics, placement in schedule to be determined
18 Other topics, placement in schedule to be determined
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
139
Annex 4c. Schedule of Learning Sessions at Mothers’ Clubs for Mothers of Malnourished
Children 6 to 59 Months of Age (Recuperative Program Group)
Month in the
program Mothers of malnourished children
1 Session 13
Causes of malnutrition
Different types of malnutrition
Recuperation of moderately malnourished children
Introduction to the child development and feeding chart
Preparation for the next session: preparing nutritious foods
2 Session 8
Preparing nutritious foods/cooking session
Tasting and discussion
Repetition of learning how to eat
3 Session 9
Repetition on child development and feeding chart (for children 0 to 11 months of age)
Helping children to eat
Feeding during and after illness
Preparation of the next session: variety of food
4 Session 10
Sharing experience with one new feeding practice (related to helping children to eat)
Variety of food
Sharing experience with preparing nutritious foods at home
5 Session 12
Child development and feeding chart (for children 12 months to 5 years of age)
Discussing food variety issues (special complementary foods, fruits and vegetables, vitamin A-rich
foods, animal foods, evening meal)
Preparing a creative way to communicate one feeding recommendation of the child development and
feeding chart
6 Session 1
Importance of breastfeeding (initiation of breastfeeding, exclusive breastfeeding, continue
breastfeeding until 2 years of age or beyond)
Discouraging bottle use
Comparison of recommendations with local beliefs and practices
Other topics: Immunization
7 Session 11
Hygiene in food preparation, handling and storage—Diarrhea prevention
Feeding during and after illness
Other topics: Diarrhea
8 Other topics: HIV/AIDS
9 Other topics: Family Planning
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
140
Annex 5. Quality of Education Provided at Mothers’ Clubs
Observations of the quality of the education provided at Mothers’ Clubs (n = 20)
Variable
Variable
name
Mean
or
%/numbers Range
Technical content
Communication of accurate technical information
- Information correct and complete (average %)
- 80-100% of information correct and complete
- Information presented but not correct (average %)
- No incorrect information
q609a
83%
15
14%
8
64-100%
7-33%
Accurate responses to all/most of the questions from audience (n = 3) q609b 3
Brought back focus to promoted behavior q609c 8
Session Management and organization
Completed all session steps q610a 19
Completed all session steps in order q610b 20
Did not read instructions q610c 18
Had no difficulties in reading the session q610d 19
Did not complete session in recommended time q610e 19
Gap between actual and suggested length of session (minutes) gapsugge 10 -20 - +55
Had materials ready and organized q610f 18
Facilitation and teaching skills
Used small groups as suggested q611a 16
- Defined question to be discussed
- Helped arrange participants to face each other
- Circulated around the room to help
- Asked for group reports
q611b
q611c
q611d
q611e
15
7
10
13
Used open-ended questions as in session guide q611f 18
Used open-ended questions to probe and encourage at other times q611g 13
Spoke loudly and clearly q611h 20
Showed all/most of the visuals of the session (n = 16) q611i 14
Assured that all participants could see visuals (n = 14) q611j 13
Used other teaching techniques/facilitation skills as indicated in
session guide
q611k 18
Attitudes displayed
Provided praise/affirmation to participants q612a 19
Demonstrated respect for participants q612b 17
Helped participants feel at ease with participating q612c 14
Attempted to create good dialogue and/or limit lecture style
Atmosphere at the learning session
q612d 9
Any distracting side-conversations during session? q613a 12
Any disruptive children? q613b 5
Any other distraction? q613c 14
Laughter during session? q613d 11
Did animations with the group q613e 10
Participants appeared bored q613f 7
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
141
Annex 6. Organizational Structure of the Health Component of the Program
Headquarters (Port-au-Prince)
National Health Coordinator
Regional Office (Central Plateau)
Regional Health Coordinator
MCH Supervisors
(Nurses)
Health Agents Community volunteers (colvols)
Assistant of the Regional Health
Coordinator
Assistant Training
Assistant Supervision
Assistant ADPs
An Operations Evaluation of World Vision’s Maternal and Child Health and Nutrition Program in Central Plateau, Haiti
142
Annex 7. Organizational Structure of the Food Component of the Program
Regional Office (Central Plateau)
Regional Commodity Officer
Commodity Field
Supervisors
Food Monitors
Assistant of the Regional
Commodity Officer
Warehouse CTS Coordinator
Coordinator
Assistant of CTS
Coordinator
Warehouse Manager
Assistants of CTS Data Entry Clerk
Warehouse Manager
Warehouse Tally Clerks
Warehouse Loaders