Integrating Child Survival and IMCI Activities into Six Target Communities in the North-East Department of Haiti CS-XVII Cooperative Agreement No. HPF-A-00-01-00031-00 Final Evaluation Report Project Location: Department of the North-East, Haiti Project Duration: September 28, 2001 – March 31, 2006 Submitted to: U.S. Agency for International Development GH/HIDN/NUT/CSHGP Room 3.7-72B Washington, DC 20523-3700 Attn: Susan Youll Submitted by: Project HOPE – The People-to-People Health Foundation, Inc. Millwood, Virginia 22646 Tel: (540) 837-2100 Fax: (540) 837-1813 March 31, 2006 Prepared by: Linda Morales, Consultant Contact persons: Shaun W. Walsh Bonnie L. Kittle Regional Director, Africa and Haiti Director, Health of Women and Children TABLE OF CONTENTS EXECUTIVE SUMMARY ........................................................................................1 ASSESSMENT OF RESULTS AND IMPACT OF THE PROGRAM ......................3 Summary Chart of Results...............................................................................................................3 Results - Technical Approach .........................................................................................................4 Project Overview ............................................................................................................................5 Nutrition and Immunizations...........................................................................................................6 Diarrheal Disease ...........................................................................................................................9 Maternal and Newborn Care ........................................................................................................12 Pneumonia case management.....................................................................................................14 CHILD SPACING/ HIV/AIDS/STI..................................................................................................15 Results - Cross-Cutting Approaches............................................................................................17 Community Mobilization ...............................................................................................................17 Communication / BCC ..................................................................................................................22 Capacity Building Approach .........................................................................................................25 Sustainability Strategy ..................................................................................................................36 Planning........................................................................................................................................39 Staff Development ........................................................................................................................41 Supervision...................................................................................................................................43 Human Resource Management ...................................................................................................45 Financial Management .................................................................................................................46 Logistics........................................................................................................................................47 Information Management..............................................................................................................47 Technical and Administrative Support..........................................................................................47 Collaboration with USAID Mission................................................................................................48 Program Management - Lessons Learned...................................................................................48 CONCLUSIONS & RECOMMENDATIONS..........................................................49 Annex 1 Evaluation Team Annex 2 List of Persons Interviewed Annex 3 Evaluation Process Annex 4 Project Goal, Objectives and Indicators i List of Abbreviations ARI Acute Respiratory Infection BCC Behavior Change and Communication BCG Bacillus Calmette-Guerin Vaccine CBD Community Based Distributor CBO Community Based Organization CDS Centers for Health and Development {Centres pour le Developpement et la Sante} C/HH Community HouseHold CS Child Survival CSSA Child Survival Sustainability Assessment DHNE Department of Health for the North-East DIP Detailed Implementation Plan EBF Exclusive Breastfeeding FP Family Planning GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome IEC Information, Education and Communication IMCI Integrated Management of Childhood Illness KPC Knowledge, Attitudes and Practices (survey) LAM Lactational Amenorrhea LQAS Lot Quality Assurance Sampling (technique) MOH Memorandum of Understanding MOU Ministry of Health MSPP Ministry of Public Health and Population NGO Non-Governmental Organization ORS Oral Rehydration Solution ORT Oral Rehydration Therapy PEPFAR The President’s Emergency Plan for AIDS Relief PM Project Manager PVO Private Voluntary Organization RP Rally Post SDMA Service Delivery and Management Assessment STI Sexually Transmitted Infection TBA Traditional Birth Attendant(s) TT Tetanus Toxoid Vaccine UCS Central Health Unit {Unité Communales de Santé } USAID United States Agency for International Development VCT Voluntary Counselling and Testing VHC Village Health Committee WFP World Food Program ii CHAPTER ONE Executive Summary The Project HOPE Haiti Child Survival Project was implemented from October 1, 2001 to March 31, 2006 and spent over $2.6 million dollars ($1,300,000 USAID grant money and more than $1,341,717 Project HOPE match - which includes the values of the Gifts in Kind contributions). Building on lessons-learned by HOPE’s seven years of Child Survival experience in the country’s North Department, the project’s interventions focused on reducing infant and child mortality rates and improving reproductive health in the adjacent underserved North-East Department of Haiti. Six communes in the most underserved western section of the North-East Department – with a total population of 98,907 – were initially selected for the program. However, in 2004 as a result of financial problems and with approval from USAID, the target area was reduced to two communes: Trou du Nord and Terrier Rouge with a total population of 56,778. The project worked with the six health facilities that service the two communes. The project’s interventions included: nutrition, including breastfeeding and micronutrient supplementation; control of diarrheal disease; maternal and newborn care; pneumonia case management; child spacing; and HIV/AIDS/STIs. Technical objectives included: • Improved family hygiene • Increased use of ORT • Increase immunization coverage • Improved pneumonia case management • Improved coverage of Vitamin A supplementation for mothers and children • Increased immediate and exclusive breastfeeding • Increased contraceptive prevalence • Increased knowledge of HIV/AIDS • Increased numbers of condoms distributed • The objectives related to “Increased care-seeking and treatment for STIs” were eliminated when the project reduced its scope. Program strategies utilized the innovations and methods of HOPE’s prior CS project in the North Department, with the addition of some specific new approaches. Major operational approaches included: • Training and support of village-based health promoters in technical skills; • Training and support of MOH auxiliaries and supervisors; • Training of representatives of community-based organizations to facilitate replication of health education and provide peer support for new behaviors; • Training and support of CBD post representatives for improved community access to health products; • Assessment and improvement of MOH health facilities in the target communes; • Capacity development at the village, health facility, and UCS levels through training and technical assistance and strengthened public-private sector coordination. (Beyond the scope of material improvements, these last two objectives were reduced when project activities were diminished due to budget difficulties). 1 The project accomplished these objectives through the training and support of community￾based health activities provided by 36 community health promoters, 8 health agents, 8 auxiliary nurses, a network of 48 community-based distributors (CBDs), and the training of 400 members of community-based organizations (CBOs). The project also provided approximately $670,000 in material support to health facilities and enhanced collaboration between trained traditional birth attendants (TBAs) and promoters. Promoters and their supervisors met weekly at their respective health centers. During these meetings, health center auxiliary nurses provided supervision, refresher training, and review of monthly report data. This project was implemented by CDS, a Haitian NGO, with considerable experience implementing health programs in Haiti. Initially, CDS was to provide technical assistance to strengthen the quality of and expand services provided through the Ministry of Health (MOH). Unfortunately, misunderstanding caused by initial early discussions between Departmental MOH partners and the first Project Manager (who had to be let go after 5 months with the project) caused the relationship between CDS and the Departmental MOH to be strained from the beginning. When it became clear that DHNE was not interested in collaborating through a stronger role, CDS became the primary implementer of program activities (operating through MOH structures). CDS, however, has been a continuous player in the North-East Department and it appears will continue to play a strong role due to its continued viability (through GFATM and PEPFAR funding). The model for increasing access to and demand for community level services is one of the priority conclusions drawn from this project: • The project model for reinforcing community level services maximizes community level impact while taking sustainability and replicability into account: o The promoters provide access to a range of preventive services at the community level. 2 CHAPTER TWO Assessment of Results and Impact of the Program A. Summary Chart of Results Indicator Baseline Target % KPC Final % Child Health % children 12-23 months completely vaccinated NA 80 80.4 % children who received a dose of Vitamin A in the last 6 months 19.3 30 85.2 % of mothers who exclusively breastfed their babies during the first 6 months 44.3 20 67 % of children U2 weighed in previous four months NA 50 70 Diarrheal Disease Control % of mothers who keep Sel Lavi at home to prevent dehydration NA 30 51.9 % increase in mothers who used ORT in last diarrheal episode 56 60 74 % know signs of dehydration 38.8 80 85 % increase of mothers who know how to prepare ORS correctly 26.3 80 89.3 % of mothers who provided the same or more liquids/solids during a disease episode 14.81 60 liquid 40 solids 44 liquids 39 solids Child Spacing % of women who do not desire a pregnancy in the next 2 years are using a modern contraceptive method 18.7 35 55.4 % of women can cite 2 or more FP methods NA 60 96 % of children 0-23 months born at least 24 months after previous surviving child2 61.1 Increases by 20 49 STIs/HIV/AIDS % of women that can name at least 2 ways to protect themselves from HIV/AIDS/ STIs 52.9 80 943 % of women whose partner used a condom during their last sexual intercourse NA 15 Not collected % of women and men can name at least 2 signs indicative of an STI 35.9 30 85 Pregnancy and Delivery Management % of mothers cite 3 or more danger signs during pregnancy & 2 or more signs during delivery 6.4 60 87 % of mothers who receive 3 or more prenatal exams 45.6* 60 41 1 for the question those who provided both fluids and solids 2 Only mothers who already had two children were considered in the final KPC 3 Reflects one known preventive method instead of two 3 Indicator Baseline Target % KPC Final % from health professional % of mothers with at least 2 TT vaccinations 11 70 62 % of mothers who can cite the signs of illness that require care seeking NA not stipulated 98 % of home deliveries use clean delivery kit 41 85 73.7 % of mothers who receive Vitamin A within 7 days post-natal 5.9 60 92.2 % of births who receive home visits by promoter within 7 days 7.7 75 87.5 % of mothers with children 0-23 months who wash their hands before eating, before feeding a child, after defecating and after attending a child who has defecated4 . 6.4 Rapid catch only 56 *card confirmed NA= no answer was available from the baseline Not amongst the project’s objectives, but worthy of mention were significant improvements in the following indicators: Indicators Baseline % Final KPC % BCG vaccination coverage 34.7 93 Measles vaccination coverage - 85 Immediate breastfeeding 56.8 88.3 Knowledge of an STD other than HIV/AIDS 62.6 95 B. Results - Technical Approach The principal results of this project are: • Improved access to health services, and • Improved behavior changes and health status, especially: -Improved coverage for Vitamin A and immunizations for mothers and children -Increased access to safe maternity care -Increased immediate and exclusive breastfeeding It is important to note that these accomplishments were achieved within a very short time frame (community level activities did not begin until July 2003 and the final KPC was conducted in December 2005) and despite a series of challenges and setbacks (including, political turmoil, difficulties in getting qualified staff to live in the project area and high staff turnover, no existing physical infrastructure from which to administer project activities in the target area, the non￾existence of a PVO country office, natural disasters, poor roads and extremely poor communications systems). 4 Actual question was related to knowledge of four time when to wash hands. 4 The project made impressive strides regarding two of the components of C/HH-IMCI – increasing services available in the community and strengthening the links between community and health facilities. Improving the third pillar, the information available to community members, was more mixed due to problems conveying clear messages. Where objectives were not achieved, it was due to limited focus on competency based training, lack of clarity and prioritization of messages, and misperceptions regarding costs or difficulty in accessing facilities (for prenatal visits for example). Project Overview The Project HOPE Haiti Child Survival XVII project covered two of the 13 communes of the Department of the North-East in Haiti. It worked with six health centers and their catchment areas, with an estimated 16,517 children under five and 15,000 women of childbearing age. Due to financial problems (notably difficulties in raising the $1.3M cash match required for organizations with two previous cycles of CS projects in the same country), the original target area of six communes was reduced to two and some of the original objectives identified in the DIP were modified or eliminated. These changes, such as elimination of the objectives enhancing the capacity of the departmental health office, were identified towards the end of 2004 and were included in the Third Annual Report. In addition, the project suffered from a number of setbacks/challenges including: a delayed start-up of project activities, a consistently strained relationship with the MOH, periods of political instability in which a project vehicle was stolen, staff had to leave the country, and project activities were stalled; and further delays caused by hurricane damage (during September 2004, the road to the North and the North-East was blocked, making it impossible to get medicine, vaccines and other supplies from Port-au-Prince to the project area). In early January 2005, further changes were made to address budgetary deficiencies: reduction of the project period from 5 years to 4.5, elimination of the midterm evaluation, and a reduction of specific activities. However, the project remained focused on seven technical areas, including immunizations, nutrition/micronutrient deficiencies (including breastfeeding), diarrheal disease; maternal and newborn care; pneumonia case management; and child spacing. These areas were selected based on epidemiologic data, showing high incidence of infant and child mortality due to diarrhea and respiratory infections, high incidence of malnutrition, and a high estimate of maternal mortality. The primary project planning matrix included a total of 18 objectives as stated in Annex 4. The project also had six capacity building “objectives” as part of its sustainability plan; these were reduced to two: o Do mothers demonstrate appropriate knowledge and practices at target levels? o What percentage of the promoters use correct basic health education messages? In order to achieve these results, the CDS successfully recruited, trained and supported a network of community health promoters. 5 Nutrition and Immunizations Indicator Baseline % Target % Final KPC % % children who received a dose of Vitamin A in the last 6 months 19.3 30 85.2 % of mothers who receive Vitamin A within 7 days post-natal 5.90 60 92.2 Increase in the % of children U2 weighed in previous four months 50 70 % of mothers who exclusively breastfed their babies during the first 6 months 44.3 20 67 Indicator Baseline Target Final % children 12-23 months completely vaccinated 15 80 80 The primary activities to improve nutritional status and increase vaccination coverage took place through rally posts (RP) offering community-based growth monitoring and immunization services. During Rally Posts, children’s weights are monitored, they are vaccinated and provided Vitamin A, and mothers of underweight children are referred to the World Food Program (WFP) recuperation center where rations are provided. These sessions are held once each month in each community according to a predetermined schedule. The posts also allowed promoters to gather data for surveillance through their monthly facility-based reporting system, and offered a mechanism for follow-up through targeted home visits. While significant coverage gains have been made in Vitamin A and immunizations through this approach, these sessions are not as effective as they could be for improving nutritional status due to the fact that limited counseling occurs, thus calling into question their ability to achieve real change in feeding practices. With respect to exclusive breastfeeding, as part of their training, promoters received messages regarding exclusive breastfeeding promotion; however, upon close scrutiny, these messages seemed to apply to breastfeeding in general and no specific exclusive breastfeeding promotion strategy (beyond promoting exclusive breastfeeding during home visits to new mothers) was implemented. The project had originally planned to utilize existing mothers and fathers clubs to further promote exclusive breastfeeding, but due to his previous experience and perceived ineffectiveness of these groups in other communes, the project manager (PM) (after discussion and approval from the CDS Executive director) decided not to pursue this approach. Since Vitamin A for mothers was distributed primarily during the post-partum home visits, the substantial increase in coverage for this indicator can be attributed to the project’s emphasis and resulting significant increase in the number of mothers who received post-partum home visits within seven days of giving birth. Original project plans for improving nutritional status also included piloting of a positive deviance/Hearth intervention; unfortunately, this was eliminated when the project was required to reduce its budget. Upon scrutiny of the baseline malnutrition rates, it appears that the project 6 zone did not have rates high enough (30% or greater rates of moderate malnutrition) to support Hearth according to the Hearth guidelines). Results The project’s intervention, to provide Vitamin A through community based growth monitoring “Rally Posts” appears to have significantly increased coverage despite frequent stock-outs of Vitamin A from the national level MOH. This graph represents the increase in Vitamin A distribution from baseline rates of 19.3% in 2002 to 85.3% in 2005 for children and approximately 15%* (baseline figure not available for women) to 92% for women. The project surpassed its target of 30% for children and 60% for women. 1 2 0% 20% 40% 60% 80% 100% Baseline vs. Final Vitamin A Coverage for Women & Children % children who received a dose of Vitamin A in the last 6 months % of mothers who receive Vitamin A within 7 days post￾natal Vaccination Coverage for Children 0% 20% 40% 60% 80% 100% 12 2002 2005 % children 12-23 months completely vaccinated Rally posts and home visit follow-up by the promoters also contributed to the substantial increase in coverage rates for children which jumped from 15% in 2002 to 80% in 2005. The project met its target of 80% coverage for children 12-23 months who have been completely vaccinated. In addition to the results for the project indicators in the summary table and the above graphs, the project achieved these additional outputs: • Established 70 monthly rally posts (where immunization, growth monitoring, health education etc are provided); • Immediate breastfeeding increased from 56.8% to 88.3% • BCG immunization coverage increased from 34.7% – 84.1% • Increased demand for micronutrient supplementation and vaccinations at the community level; 7 • 36 trained health promoters, eight health agents and approximately 400 educated community organization members – again, many of whom are expected to replicate the education sessions/messages for their family, friends, and colleagues. Strengths/Accomplishments: • Despite frequent stock-outs of Vitamin A, the project was able to show an impressive increase in the proportion of women and children receiving the vitamin. This is perhaps due to the fact that on various occasions when stock-outs from the MOH were eminent, the project purchased Vitamin A. • The integration of growth monitoring, vaccinations, Vitamin A, and ORS distribution at community based Rally Posts (RP) increases access to these services for mothers and their children who would not go to the health facilities due to distance/cost of transportation. • The Rally Posts also provide an opportunity for increased involvement in health activities by the community members. • Rally Posts empower the promoters to take greater responsibility for their community’s health and expands the function of growth monitoring to also serve as an integrated service delivery point for preventive health at the community level. • Rally Posts also provide a source of data for what is happening in the community and enable promoters and their supervisors to identify families requiring home visit follow-up. • While recognizing that RPs alone are not likely to lead to significant decreases in malnutrition, project staff, promoters, and health center supervisors feel they were useful in increasing access to vaccination services and identifying candidates for inclusion in the WFP feeding program. Challenges: • Growth monitoring sessions are less effective than they should be because little individual counseling occurs; promoters are not providing feedback about the weight of the child to mothers; some promoters do not remove children’s clothing before weighing them; at some posts, the cold chain is not respected since promoters have only one thermos that they must continually open for each vaccine dosage; education sessions are not consistently providing clear IEC messages and promoters are not using visual aids. • Promoters are reluctant to counsel mothers when children are gaining weight because they fear that mothers will no longer bring children for weighing or will withhold food to ensure that the child weighs in the red or yellow growth ranges; this is due to the promoters’ perception that mothers understand (incorrectly) that they will no longer qualify to receive family rations from the World Food Program’s food supplementation program if their children are considered to be growing adequately. • Neither promoters nor their supervisors/auxiliaries at the health center understand that data is collected in order to gauge progress and to inform decisions (all promoters collect data on numbers of mothers in attendance, and numbers of children underweight and severely underweight, but they do not compare these numbers with any denominators and are therefore unaware of the success of their approaches, nor whether progress is made from one month to the next). They also need to understand the process of diagnosing causes for lack of weight gain and tailoring counseling to the identified need. • Common shortages of Vitamin A and vaccinations at the RP’s sometimes discourage mothers who come to the RP expecting these services (the shortages are due to frequent 8 and at times, lengthy stock-outs at the health center level resulting from nationwide shortages). • The community members are not involved in the organization or logistics of the Rally Posts, thus representing a missed opportunity for their participation in community mobilization and preventive health activities. • Supervisors and promoters lack appropriate checklists with which the supervisors could evaluate the quality of the RP. Future Implications / Recommendations / Lessons Learned • The use of the growth monitoring sessions as distribution points for increasing access to preventive health services at the community level was very effective at increasing levels of coverage for immunizations and Vitamin A. By integrating vaccinations, Vitamin A distribution for mothers and children, and the distribution of ORS, an array of preventive services with known impact can reach the community. • Faced with the problem of disincentives for promoters to counsel women, the promoters in collaboration with community members should create a greater incentive for mothers of well nourished children, such as a simple wooden plaque or certificate with their names added for each month that the child shows adequate weight gain. • Given the level of resources needed to make growth monitoring sessions more effective, the MOH at the district and health center level will need to reflect on the level of support they can provide to improve these activities (better support for supervision of the activity and ways to ensure fewer input shortages, cold chain maintenance, etc., given their common shortages of vaccinations and Vitamin A). • PLAN/Haiti supervisors may be able to provide limited supervision from a different project. Diarrheal Disease Indicators Baseline % Target % Final KPC % % of mothers who keep Sel Lavi at home to prevent dehydration 30 51.9 % of mothers who exclusively breastfed their babies during the first 6 months 20 67 % increase in mothers who used ORT in last diarrheal episode 56 60 74 % know signs of dehydration 38.8 80 85 % increase of mothers who know how to prepare ORS correctly 26.3 80 89.3 % of mothers who provided the same or more liquids/solids during a disease episode 60 40 44 39 To accomplish the project’s objectives in this area, promoters were trained in diarrheal disease prevention and management (as part of their initial 47 day training--see capacity building below) and promoted correct utilization of ORS, distribution of ORS sachets at the RPs (during education sessions) and during home visits and proper care and feeding of sick children. In 9 addition, promoters were trained in and promoted personal hygiene, including hand washing and sanitary food preparation, and cleaning/treating vegetables appropriately during their community level activities. Also as part of the strategy, promoters promoted exclusive breastfeeding for new mothers. Toward the end of the project CBD post managers were trained who stocked and sold ORS packets. The CBD network consisted of 44 village-level boutique owner/managers who were trained and stocked (through an agreement with Population Services International- PSI) ORS sachets to increase access and use of ORS at the community level. This component was added during the project’s last year because of the common stock-outs that the promoters experienced at the MOH health facilities. Results As shown by the summary table above, the project achieved significant increases in the following: • Proportion of mothers who keep Sel Lavi (the PSI marketed brand name for ORS) at home • Proportion of mothers who know how to correctly prepare ORS • Proportion of mothers who can identify at least two signs of dehydration A significant increase is noted in the % of mothers who know how to prepare ORS correctly (from 26.3 to 89.3% and in the % of mothers who used ORT during their child’s last diarrheal episode (from 56 to 83.7%). Correct ORS Preparation 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 12 2002 2005 % increase of mothers who know how to prepare ORS correctly Findings In discussions with some of the trainers having prepared curricula for the training of promoters on the project’s intervention areas, the evaluation team found that messages on diarrhea and especially on hand-washing and the benefit of exclusive breastfeeding to prevent diarrhea were not standardized and not communicated clearly during the training and other community level activities. Interviews with the promoters confirmed that their own knowledge/understanding regarding these key practices was insufficient; 17 of 24 promoters interviewed gave incomplete responses to the questions: “name four ways to prevent diarrhea” and “when should you wash your hands to prevent diarrhea?” In addition, the KPC results showed poor progress towards achievement of the target regarding “increased feeding and liquids during diarrheal episodes”. 10 The evaluation team learned that the training for CBDs began very late in the project cycle (September 2005) due to lengthy negotiations (six months) with PSI, with whom the project had planned to conduct the training. An unfortunate result was that the CBDs initiated activities only during the last quarter of the project (January 2006). CBD posts are not easily identified in the community; no current identifying sign or mark distinguishes them from other boutiques that do not provide these health products; they also have not done any marketing of their health related products. Mothers confirmed that they did not know these “boutiques” were CBD posts. Also, despite initial training, CBD post managers do not know how or when to renew their stock; they could benefit from refresher or on-the-job training and support for managing their stock. Strengths • The promoters used multiple channels to promote and distribute ORS (through the RPs, home visits, and by referring mothers to the CBD posts); this helped to raise initial demand and provided mothers with reinforced messages on when to use and how to prepare the ORS at home. • The training of the CBDs and supplying of ORS helps ensure greater access to ORS by community members (many of whom cannot easily access the health center). • By placing emphasis on treating diarrhea early on at the household level, the project will likely continue to have an impact on reducing mortality due to the fact that once mothers recognize signs of dehydration it is often too late for the child to recover. Challenges • Lack of message standardization and clarity regarding preventing diarrheal disease through health and hygiene practices, such as exclusive breastfeeding and hand washing at appropriate times, and in terms of increased feeding and liquids during disease episodes. • Difficulty in accurately assessing the full impact of the CBD component due to late initiation of training and implementation activities. • Low visibility, low recognition of CBD posts in the communities. • Limited understanding of stock management could mean unnecessary stock-outs of health products at CBD posts. Future Implications / Recommendations / Lessons Learned • During training and follow-up supervision activities for promoters, more emphasis needs to be placed on clarity and completeness of messages; supervisors should also emphasize the importance of prevention vs. treatment of diarrheal disease in their interactions with promoters and mothers. • CBDs should be encouraged to form associations of community distributors; they should improve their marketing and signage for better recognition in the community; they also need refresher training and continued support in stock management. • The approach of promoting the use of and distributing an initial ORS sachet to mothers at the RPs can be an effective method for promoters to raise initial community demand for ORS sachets. • Faced with frequent stock-outs, using CBD posts provides mothers with an alternative source for supplies such as ORS sachets, thus reducing their dependency on the sporadic supplies of the MOH and setting up an alternative structure to ensure access to ORS at the community level. 11 Maternal and Newborn Care Indicators Baseline % Targets % Final KPC % % of mothers with at least 2 TT vaccinations 11 70 62 % of mothers who exclusively breastfed their babies during the first 6 months 44.35 20 67 % percent of mothers cite 3 or more danger signs during pregnancy 6 60 % percent of mothers cite 2 or more signs during delivery 60 % of mothers who receive 3 or more prenatal exams from health professional NA 60 48 % of home deliveries are assisted by TBAs (use of clean delivery kit) 41 85 73.7 % of mothers who receive Vitamin A within 7 days post-natal NA 60 92.2 % of births who receive home visits by promoter within 7 days 62.6 75 87.5 Maternal and newborn care was addressed primarily by strengthening the partnership between promoters and TBAs at the community level and by providing material support to the TBAs (Child Birth Box). The project obtained 50 Child Birth Boxes from UNICEF and distributed them to the health centers who in turn distributed them to the TBAs. These Child Birth Boxes are carrying cases with some instruments and supplies that the TBA brings to the mother’s house when she assists with the birth and their contents need to be renewed regularly. In addition to the Child Birth Box which the project initially supplied, TBAs were expected to obtain a sterile birthing kit for each delivery. These kits were purchased by the TBAs and the mothers at the health facilities. Health facility staff encouraged mothers to purchase the kits to have on hand for their baby’s delivery. Promoters and supervisors were also trained in maternal health and promoted the importance of prenatal visits, use of a clean birthing kit for each delivery, exclusive and immediate breastfeeding, and postnatal visits within seven days. The project anticipated training traditional birth attendants (TBA), but since the Ministry of Health assumed this role directly, the project focused its efforts on creating a strong collaborative relationship between the promoters and the TBAs. Within the project site, promoters and TBAs have developed good rapport and a system for mutual referral when a woman becomes pregnant and after she gives birth. This collaboration contributed to the achievement of many of the project objectives/indicators mentioned above. 5 For 0-5 months olds 6 In the DIP this and the following indicator were combined into one, which caused confusion while doing the KPC analysis, which resulted in no credible data for either indicator. However, as the KPC survey shows mothers are able to name many dangers signs for both pregnancy and delivery. 12 Additional community level activities included the Fixed Points, an intervention that was not a part of the DIP. Fixed Points were established to improve access to prenatal services by conducting consultations at the community level in some of the more remote and isolated locations. While they were supposed to be held once a month, in reality, they are only held sporadically. Discussions with project staff indicated that during these sessions, the promoter records the names of the pregnant women and provides health education while the auxiliary nurse conducts the prenatal exam (weight, BP, uterine height presentation, general exam, checks for danger signs). Iron pills are also supposed to be distributed. For those pregnant women who do not come to the Fixed Point, the promoters make home visits to encourage the women to come to the health center. Unfortunately, due to time limitations, the evaluation team was unable to substantiate the actual implementation of the Fixed Point. However, during discussions with the promoters, nurses, and project staff, the evaluation team learned of several problems associated with the Fixed Points: the need to move equipment, a confusion within the target population regarding the purpose of the Fixed Points (whether they are for general consultations as well or just prenatal care) and limited publicity about them, etc. While VCT counseling had been included as a strategy in the DIP, the project eliminated this activity since the MOH was unable to roll-out VCT sites during the project’s life cycle. Home Deliveries with Clean Delivery Kit 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 12 2002 2005 % of home deliveries using clean birth kit The increase in deliveries with a clean birth kit is significant, from 41% at baseline to 77.2% as reported in the final KPC. Strengths/Accomplishments • While the project did not implement specific activities with the TBAs, the rapport developed between the promoters and TBAs is excellent and proof of a strengthening of linkages between the health facilities and traditional health workers. The majority of TBAs know and collaborate regularly with the promoters; they also refer pregnant women for prenatal consultations. It is unlikely that the significant increases in delivery with clean birth kits and the proportion of new mothers receiving timely post partum visits would have been possible without this collaboration. Challenges • Due to the need to obtain and, in the case of Trou de Nord, purchase sterile birthing kits from the health centers to replenish their supplies, some of the TBAs are unable to keep up their stock; also the Safe Birth Boxes are frequently missing essential 13 elements, such as the sterile birthing kits which leads one to question just how “clean” they really are at the time of delivery. • It is difficult to know to what extent one can attribute the lack of significant change in the number of mothers having completed three prenatal visits; the project needs to study the barriers to completion of the third visit in greater detail. It may be that mothers believe that all prenatal visits require exams done at a health facility, and that the cost is not affordable (in reality mothers are charged a nominal fee, only for the first visit, and subsequent visits are free). • Auxiliary nurses and promoters complain about the limited equipment for the Fixed Points which must be transported impractically between the health facility and the site each time the Fixed Point is conducted (exam table, scales, etc…). • Despite the fact that the approach is new and relatively unknown (since little marketing has been done to raise awareness of their utility), it appears doubtful that the Fixed Point represents a cost-effective approach especially because of the limited numbers of women who are pregnant at any one time in the same zone or adjacent sectors. • While the increase in the number of women receiving post-partum visits is commendable, supervisors have no way of verifying the contents or quality of these visits. Teaching promoters to use the post partum check list that CDS employs in the Eastern part of the department would be a simple way to help ensure quality. Future Implications / Recommendations / Lessons Learned • As mentioned above, research should be conducted to identify the true barriers preventing women from receiving three pre-natal consultations. • To determine the utility of the Fixed Point, it would be helpful to research whether it is providing services to women who would not normally come to the health center for prenatal care or merely shifting those who would make the trip anyways to a closer location for them to access. MOH should research the cost-benefit of the Fixed Points based on the limited numbers of women who are pregnant at one time in the same area; they should study the possibility of combining the Fixed Points with the regular RPs, and offering free multivitamins with iron as an incentive to get mothers to come for prenatal visits. • Even though the project was not involved in training the TBAs, it would have strengthened the approach to maternal and newborn health to have verified the contents of the training to assure consistent messages/skills-building in post partum/ newborn care, birth planning, and perhaps home-based lifesaving skills. Pneumonia Case Management Indicator Baseline % Target % Final % % of mothers who provided the same or more liquids/solids during a disease episode 14.87 60 40 49 liquids 18 solids % of mothers who can cite the signs of illness that require care seeking NA 98 7 for the question those who provided both fluids and solids 14 In addition the following indicators were identified in the DIP, but eliminated when the project reduced its scope: --Increase the percent of mothers who know to complete prescribed treatment for the child following prescription by a provider --Increase the percent of ARIs treated according to protocols --Increase the percent of trained auxiliaries following IMCI management protocols Approach Health Facility Level Due to budget changes and revisions in the scope of project activities, many of the interventions that were planned with the facility level staff were not implemented. For example, while Auxiliary nurses were trained in IMCI (actually the project only helped support a part of the first IMCI training for 20 Auxiliaries since the MOH received funding from other sources to complete the training), little or no follow-up supervision, support or refresher training was conducted by project staff in this area. The evaluation team found that nearly all of the nurses interviewed have a very superficial knowledge of the benefits of using the IMCI algorithms and how to actually apply the algorithms. Household Level Promoters received a 5-day training module on ARI during the 47-day training, although it is not clear if the training was linked to specific competencies. At the household level, promoters taught mothers the danger signs which would require them to take their children to the health center. Due to MOH not approving a new protocol which would have allowed promoters to distribute antibiotics, promoters’ responses are limited to referring mothers to health centers where they can be treated. The promoters and CBD agents were trained in family planning benefits, distribution and correct usage of barrier methods, and referrals to health center staff for first time users of hormonal methods. Promoters distribute barrier methods during home visits and when a client had had an initial counseling session by a health center nurse or auxiliary, promoters also dispensed hormonal methods, including oral contraceptives and Depo-Provera. The evaluation team, however, learned that they failed to promote important FP benefits such as the health of the mother and the children. CBD posts were established by the project and distribute condoms, oral contraceptives, and Depo, but they do not understand how to adequately manage their stock. Child Spacing/ HIV/AIDS/STI Indicator Baseline % Target % Final % % of women who can cite 2 or more FP methods NA 60 96 % of women who do not desire a pregnancy in the next 2 years are using a modern contraceptive method 18.7 35 55.4 % of children 0-23 months born at least 24 months after previous surviving child 61.1 Increased by 20 49 15 Indicator Baseline % Target % Final % % of women that can name at least 2 ways to protect themselves from HIV/AIDS/STIs 52.9 80 948 What percent of men/women used a condom in their last sexual intercourse NA 15 Not collected What percent of women and men can name at least 2 signs indicative of an STI 35.9 30 85 Strengths/Accomplishments • While it is too soon to judge their impact, CBD posts may represent an effective mechanism for increasing access to family planning methods in the community by providing an alternative safeguard against stock-outs. • Training in IMCI has set the stage for this holistic approach to treating childhood illnesses to become a standard practice in health centers in this area; at least some of the nurses, and in particular, those who treat a substantial number of children seem to have a very basic understanding of the appropriate questions to ask a mother and how to apply the algorithms correctly. Challenges • As mentioned in the section on Diarrheal Disease, CBD posts are not clearly identified thus they are losing potential clients. • Also, as earlier mentioned, it is still relatively early to understand the full impact that these CBD posts may have on increasing access and use of contraceptives. • One training is not sufficient to ensure that CBD managers retain appropriate knowledge levels and skills in counseling clients in correct method use, contra-indications, and danger signs; they need follow-up refresher training and supportive supervision. • Promoters are not currently aware of the number of or reason for family planning method discontinuation; there is no current follow-up done on these clients. • Stock-outs of contraceptives were common occurrences for the promoters. • Nurses and auxiliary nurses cannot be expected to become experts in the IMCI protocols after having only participated in a training on the subject; there needs to be supportive supervision and follow-up, on-the-job training, and mentoring to help them become adept and comfortable with the approach. They also need to be assured that frequent stock￾outs of antibiotics will not affect their capacity to treat infants using the IMCI protocols. Future Implications / Recommendations / Lessons Learned • Community promoters and CBD posts can increase community access to services, including family planning; they do require initial support in learning how to manage their stock and market their products; again CBD networks or associations may be an effective method for empowering these individuals to support one another and continually improve their own management and increase opportunities for better revenues. • All training whether family planning methods or IMCI protocols should be followed up with concrete action plans for supportive supervision and mentoring to continue to build 8 Accurate for only one means of protection. 16 skills and increase capacity; checklists to help trainees consistently follow systematic procedures for complex or difficult-to-remember skills should be developed as job aids. • Barrier analysis should be conducted to identify and address the most significant barriers for nurses and auxiliary nurses to applying the IMCI protocols. • While tracking numbers of total family planning methods users is useful for assessing method adoption, it is also necessary to monitor the number of dropouts and their reasons for each method so that strategies can be adopted to transform dissatisfied users into continuing long-term users and to ensure that myths are dispelled before they have deleterious effects on method use in the community. C. Results - Cross-Cutting Approaches Community Mobilization Description of Approach The project’s initial plans included 1) community level work through the promoters; 2) activities targeting the Village Health Committees and mothers/fathers clubs; and 3) CBD approach. Actual community mobilization efforts included three levels: 1) the work of the promoters at the community level; 2) CBD posts, and 3) community-based organizations. Promoters organized and conducted community health education sessions and distributed contraceptives and ORS. The promoters were identified by their communities and trained by the project. They reported to MOH health facility auxiliary nurses and participated in monthly meetings with other promoters in their area and the health center auxiliary nurse. Promoters provided a linkage between the health center and the community by delivering community preventive services and disseminating community awareness-raising messages. They also received a re-supply of contraceptives, vaccines and ORS (when available) and received refresher training. Through home visits, they assisted nurse auxiliaries with follow-up for vaccination dropouts. The promoters collected large amounts of community level data which they recorded on more than10 different forms. The second tenet of the community mobilization approach consisted of training and equipping existing community level boutiques/kiosks as CBD posts to sell ORS, and contraceptives (see discussion of CBD posts in section on Diarrheal Disease). The third element of the approach was the identification and training of representatives from CBOs (such as youth groups, gardening clubs, farmers clubs and road repair groups). The PM determined that this approach should replace the previously planned strategy of working with the Village Health Committees and mothers/fathers clubs because these entities had been inactive in other parts of the country where he had previous experiences. No survey to validate this assumption was conducted with these groups in the project area. Project staff did, however, conduct an inventory of active non-health related community-based organizations. Project staff then trained nursing supervisors who conducted the training for two CBO representatives from each of five CBOs per training. Each training course was 10 hours long (two hours per day over 17 a five-day period) and covered family planning, STI/HIV/AIDS, vaccinations, diarrheal disease, and several other child health topics. Findings Community Mobilization Efforts Via Promoters • The majority of the promoters trained remain active in their communities. The evaluation team found that of a total of 40 community health promoters trained, 36 remained with the project, and of these, approximately three-quarters remain active despite having received a letter from the Project Manager stating that the project was ending. Discussions with MOH auxiliary nurses/supervisors who conduct the monthly meetings, indicated that the majority of the promoters continue to participate in community activities and attend the monthly meetings. Two-thirds of the supervisors we spoke with confirmed the attendance of all or nearly all of the promoters at their last meetings (held in December and February). • However, the PM and project staff felt that community mobilization activities had not resulted in true mobilization of the community to take responsibility for their own health improvement and the evaluation team concurs with this overall assessment. Since the decision had been made to drop the strategy to work with the VHCs and mothers/fathers clubs it is difficult to assess whether these strategies would have been more effective approaches. In reviews of the promoters’ monthly reports, the evaluation team did, however, find evidence of the existence and in some cases current activity of mothers/fathers clubs for exclusive breastfeeding, VHCs, and committees to follow-up on obstetrical emergencies. Interviews with project staff indicated that no one specific staff member was in charge of this component of the project; thus, opportunities may have been lost (such as getting the community to mobilize in helping prepare for the RPs, getting them to help promoters identify and locate vaccination drop-outs, etc.). • The promoters had too many different tasks with too many messages and not enough concrete activities or focus. They were responsible for too many forms. This was not only time consuming, but overwhelming and impacted upon their ability to monitor this data and use it to make decisions (see section on health information system). CBD Efforts • While only in operation a couple of months at the time of the final evaluation, approximately 48 CBD posts are currently operating in two project communes. While the evaluation team did find evidence of the existence of several of these CBD posts (11 were interviewed), the evaluation team found a number of weaknesses with this component. The training, which was to set in motion this approach (and had been planned for April ‘04 according to the DIP), was delayed until late 2005 when the project was in its final phase. This was, at least in part, due to the lengthy negotiations with PSI whom the project had subcontracted to conduct the training. CBD training was deficient in building competencies in stock management and renewal as evidenced by the fact that in response to the question “when should you renew your stock?” nine of the 11 CBD staff interviewed gave incorrect responses and only one person was able to actually describe the process for stock renewal. Another weakness noted by the evaluation team was that CBD posts are not known in the communities. They are not 18 marked and no efforts have been undertaken to increase their visibility. While one out of four of the 24 mothers we interviewed confirmed having purchased from a “boutique”, none of them were able to name the boutique as a CBD post. Activities with Community Based Organizations • A complete inventory of 96 CBOs was developed by the promoters. • A total of 400 men and women from CBOs were trained and 70% of these have replicated information sessions on basic health messages to their family, friends, and colleagues. • An additional 485 individuals received health information from these CBO representatives. While the strategy appears to have been successful initially in mobilizing the masses (885 people in all have participated), and possibly in increasing the level of knowledge on some basic health messages (more than three-quarters of those interviewed could correctly answer basic questions regarding family planning, HIV/STIs, and EBF; but only one person could mention the timeframe within which a child should be completely vaccinated), it was initiated towards the end of the project cycle, making it difficult to draw credible conclusions regarding its effectiveness. More importantly, no direction was provided during the training or afterwards on how to participate in community-level health activities or how to help promote the target behaviors, thus it appears doubtful as to its effectiveness in mobilizing the community to become involved in health activities (none of the 10 individuals the evaluation team interviewed mentioned participation in community health activities as something that was expected of them after the training). Also, no evaluation or assessment of the quality of messages disseminated in the training nor those retained post-training was conducted by the project and since the entire component was started very late, time was too short to provide follow-up refresher training. The evaluation team also learned that financial incentives were given to the trainers and the CBO representatives to encourage them to participate in the one-week training and to replicate the training for other community members. In addition, each of the CBO representatives received a payment for replicating the training contents with at least 10 other community members. These practices put into questions the true motives of the participants. Strengths/Accomplishments --While the work of the promoters has several areas which can be improved upon, it has been successful in increasing community level demand for services, increased access and use of these services, and subsequently contributing to improved health status as evidenced through final KPC results for several indicators. --The CBD and promoters components of the mobilization strategy appear to have been effective in increasing community level availability and access to specific products (ORS and contraceptives). Challenges --Promoters often fall victim to the MOH’s frequent stock-outs making it difficult to assure consistent supplies of community inputs for preventive services (Vitamin A, ORS, and contraceptives). This undermines their effectiveness and credibility, and contributes to significant missed opportunities. 19 --Both CBD and CBO efforts began very late, thus their full impact on mobilizing strong community support for the community-level health activities cannot be accurately assessed. --The CBD strategy did not include a marketing component to ensure CBD-recognition and CBD training was deficient in building competencies in stock management and renewal. Because CBD posts became operational at the very end of the project cycle, additional on-the-job assistance was not provided. --CBO trainees received no direction regarding how they could help promote the target behaviors and participate in health activities. Also, very minimal importance was accorded to the mothers clubs, fathers clubs, village health committees and the committees to follow-up on obstetrical emergencies. The project did not sufficiently research the presence and level of activity of these entities before discounting their importance within the community. --The staff structure at CDS/HOPE was insufficient to provide the necessary support for community mobilization efforts, including follow-up for both CBD and CBO efforts. Recommendations/Lessons Learned /Plans for Sustaining Activities Use of Promoters --Using promoters to organize and implement community level activities maximizes community level impact while taking sustainability and replicability into account. --Promoters provide a range of services at the community level. They are well-known by mothers and make regular visits to follow-up on the families within their zones. --Since promoters are selected by the community, they should involve the community leaders in determining how they will continue to operate without project support (given that they have gained the community’s confidence and trust). --The promoters need continued support from the health centers which can provide a cost effective mechanism for supervision, refresher training and motivation. --The involvement of health center staff in maintaining and supporting community activities is essential (RPs must be done in collaboration with health center staff to maximize their effectiveness). --While promoters have been paid by the project, several of them are continuing to operate even after the project no longer provides their salaries. It may be helpful to consider some type of income generation activity/product linked to the promoter’s regular activities that could help the promoter support his/her livelihood. --The MOH should continue to provide motivation and support to health center staff encouraging them to continue to support the promoters and the community level health activities. They should be encouraged to: 20 i. Organize and facilitate a meeting between the health facility directors to help them think about the ways they can continue to support the promoters; ii. Be involved with selection (replacement for those that are no longer active) of new promoters; iii. Provide in-service training and technical support for the promoters using mini-modules for training during monthly meetings (see training section); iv. Prioritize supervisory support for promoters (in collaboration with other projects such as PLAN and/or PEPFAR-funded activities) (see section on Capacity Building- Supervision below); v. Assure the availability of Vitamin A, ORS, and contraceptives to maximize the effectiveness of the promoters and community distribution points. CBD Approach --By the nature of their business, CBD posts have a vested interest in improving revenue by increasing sales. Fortunately, their entrepreneurial motivation should push them to find new ways to increase their clientele and total sales. Care must be taken; however, to ensure that increased sales translate to knowledgeable, satisfied and long-term clients who are improving their health and that of their families. --As previously mentioned, CBD efforts require training based on specific competencies, such as management of stock. If CBD managers can be provided with follow-up assistance in forming networks, marketing their posts, and stock management, their potential for continuing to improve access to community level preventive health products will be promising. To sustain activities, project staff should negotiate with PSI regional representatives (who currently provide stock renewal) to determine which of these activities to support the CBDs can be taken on by PSI. CBO Strategy --Initial training for CBO representatives should go beyond focusing on increasing knowledge to include discussion and concrete action plans that identify how these representatives can encourage behavior changes and become involved in community health activities (such as preparing the areas for RPs and helping the promoters to follow-up on families with children who have not completed their immunizations). --Prior to embarking on the training of community members, project staff should conduct in￾depth research to determine the feasibility of working with existing community health entities, such as mothers groups, health committees, etc. --Responsible entities, whether pre-existing or newly created should be identified at the beginning of the project cycle to plan, manage, and evaluate community level health activities in collaboration with promoters. This information should be utilized early on to help ensure that mobilization efforts are community-driven. -- Project staff should seriously consider creating the expectation of monetary remuneration and the impact this will have on sustainability before adopting a strategy. 21 --Project staff should provide the list of those trained to PLAN (which will continue to operate in the project sites) and to the MOH auxiliaries and solicit their assistance in following-up with these trained CBO members. Information, Education, Communication / BCC Planned Approach According to the DIP, the project had originally planned the following BCC approaches: Counseling/person-to-person interaction by promoters, group interactions, peer modeling/support networks via health committees and mothers/fathers clubs, use of positive deviants (PD), and key message dissemination (based on actions that parents and health workers are responsible for). For providers (auxiliaries, promoters and TBAs), the BCC strategy entailed training, provision of required minimum equipment and supplies, and monitoring and supportive supervision. Related activities were to include training in adult learning strategies and communication methods for CDS staff/trainers, MOH auxiliaries and promoters; group meetings and home visits; identification and training of peer models as animators and organizers of mothers/fathers clubs and PD/Hearth sessions; creation of a Project Advisory Council. Revisions/Justification Financial support for the BCC activities was reduced from $131,613 to $38,782 when Project HOPE was required to reduce its budget and implementation time upon the realization in December 2004 that it would not be possible to raise nearly $700,000 in cash match (see Northrup memo, 26 January 2005). Specifically, peer support networks, Positive Deviance/Hearth model, training of TBAs, training in adult learning and communication methodologies were all removed from the project. These BCC approaches were specifically selected for omission after much consideration not only to address budget concerns, but because they were deemed less essential to achievement of the project objectives than the BCC strategies that were maintained. Description - Actual Approach Although no new, revised behavior change and communication strategy was developed once these revisions were made, the following six elements contributed to this cross-cutting intervention: 1) Promoters’ community-based activities which increased access to services (Rally Posts and home visits); 2) Promoters’ IEC activities for message dissemination and reinforcement, including community education sessions, education sessions during RPs, and home visits; 3) Training of Promoters, Auxiliaries/Nurses, and CBO representatives; 4) Message dissemination by CBOs; and 5) Provision of required minimum equipment and supplies for facilities. Findings The evaluation team found that this was one of the weakest aspects of this project because there appeared to be no clear strategy, nor many tools to help guide or assure the quality of several of the activities. Nonetheless, the use of the promoters as the primary channel for behavior change and communication was effective in improving knowledge levels and more importantly, in increasing access. This subsequently resulted in significant behavior change as demonstrated by the numerous improvements in key health indicators, especially those that relate to access to 22 services, such as vaccinations, Vitamin A, and post partum consultations. The team felt this was at least partially due to their daily contact (via RPs, community education sessions, and home visits) with the communities and their multiple opportunities for increasing access to services. Of the 23 mothers we spoke with, all of them had attended RPs, half had attended community education sessions and nearly three quarters of them had been visited by a promoter. Twelve of these mothers had had more than four visits by their promoter in the six prior months. Also, all but one mother knew the names of the promoters in their area. While nine of the mothers mentioned that counseling on health care was the purpose of the last home visit made by the promoter, it was not possible for the evaluation team to evaluate the quality of these interventions. No counseling checklist was available, nor evidence that job aides, such as flip charts or counseling cards were being used. It was, however, evident that very little, if any counseling was being done at the RPs (see Challenges under Nutrition and Immunizations). Regarding IEC activities for message dissemination and reinforcement and the “message of month” approach, the evaluators found that even though many promoters had difficulties in providing complete answers for questions relating to ARI signs, prevention of diarrhea, and what to do for a low-weight infant, many of them were knowledgeable about the majority of key messages, showing correct responses for questions pertaining to maternal health, family planning, vaccination, HIV/AIDS, EBF, and others. In judging from the substantial increases in most of the knowledge-based indicators for mothers (see Summary Table of Results), the evaluation team felt that the promoters’ message dissemination and reinforcement efforts had had significant impact on increasing knowledge levels as a preliminary step toward behavior change. Promoters, however, did not appear to have adequate IEC materials to support them in conducting these activities (some promoters carried a few brochures on some of the health topics, such as family planning or HIV/AIDS). Much of the IEC print materials for mothers and key messages that the team leader reviewed were based on national level MOH materials. Most of these print materials appeared to be outdated, limited to a great deal of text and not many illustrations, or not specific enough (for example, specifying the appropriate times when mothers need to wash their hands or the differences between the benefits of exclusive breastfeeding and non-exclusive breastfeeding). These materials were not critically reviewed for their level of technical appropriateness or compliance with international standards, priority, or relevance to local barriers to behavior change. The lack of specificity of many messages led to missed opportunities. While the project staff and program planners did not anticipate having to develop their own materials, little appears to have been done to ensure that each message or intervention area had an appropriate, high quality visual aid. While group education sessions were conducted at the community-level (during large group education sessions and the RPs), the evaluation team was unable to adequately assess the quality of these interactions. This was partially because the team was not able to observe the sessions - the evaluation team made three attempts, but scheduling changes, impassable roads, and longer than anticipated travel time to the RPs prevented team members from observing the group 23 education part of the RP session. Also, no checklists for assuring quality education according to adult learning methodologies were developed by the project. The team was unable to adequately assess the impact of the other BCC components, including: the training of auxiliary nurses and provision of equipment and supplies to the health facilities, and the CBO component. The team’s findings did, however, show: 1) the training of auxiliary nurses included only a one-time, lengthy IMCI training and none of the six individuals interviewed had a very thorough understanding of how the approach is used; 2) some of the auxiliary nurses mentioned that the equipment, office materials, and medical supplies had enabled them to do their job better. As earlier mentioned, the training of CBO representatives and their subsequent message dissemination to other community members, began late, thus it is also difficult to assess its impact on helping the project achieve its objectives. It was clear however that the strategy mobilized large numbers of individuals and transferred health messages to nearly a thousand individuals in the project area. While project staff mentioned that trainees were given a flyer with key health messages for their replication sessions, little or no follow-up support was given to these individuals. Knowledge assessments indicated that some of the important messages had not been retained, such as essential hand-washing times, or the time period for a child’s complete vaccination. As previously discussed, no standardized messages or guidelines for key content were included in the training curriculum. Strengths/Accomplishments --Training for the promoters contributed to a high degree of message retention for the majority of key target behaviors. --Through their community level interactions, the promoters effectively increased knowledge levels and access to services for mothers. --Equipment and supplies provided to the health facilities appears to have assisted some auxiliaries to improve the level of quality (not substantiated by the evaluation team). --The CBO strategy succeeded in disseminating health messages to nearly one thousand community members. Challenges --Project staff had very limited expertise in BCC; therefore, no one really was able to sufficiently provide the time or technical assistance needed to ensure quality in this area. --Project staff, promoters, and auxiliaries (the promoters’ supervisors) had no checklist/job aides to verify quality of counseling and group education sessions. --Project staff, promoters, and auxiliaries had few appropriate IEC materials to help support them in their work with mothers. 24 --The CBO strategy was initiated very late, the curriculum did not provide sufficient guidance in content/key messages, and no follow-up was done to determine the quality of messages disseminated. Lessons Learned/Recommendations/Plans for Sustaining Activities --Due to their daily contact with the communities and their multiple opportunities for reinforcing messages and increasing access to services, promoters can be useful agents of behavior change. --Since many of the promoters will continue to provide services at the community level, it seems likely that many of the activities which have helped influence behavior change (RPs and home visit follow-up for vaccinations, post-natal visits, family planning method distribution during home visits) will be sustained after the project. --The BCC component would have benefited from initial technical assistance from an experienced BCC specialist; this individual could have helped the project develop an appropriate and focused BCC strategy based on Designing for Behavior Change Frameworks for each behavior targeted as high priority (the strategy would have focused on identifying influencing factors - potentially through Barrier or Doer/Non-Doer Analyses) and developing interventions appropriate for accelerating progress towards behavior change. This would have been especially helpful for some of the areas such as pre-natal consultations and diarrheal disease prevention and treatment, and in response to the cancellation and late initiation of several activities due to the December 04 budget/activities reduction. The initial expense would have been justified by the potential impact of complementing the promoters’ efforts with better BCC capacity. --It would have been useful for the Senior Technical Advisor at Project HOPE headquarters to have either provided direct BCC guidance or requested assistance from other headquarters colleagues who have experience in developing BCC strategies. Also, Project HOPE colleagues with experience in developing visual aides, such as the Mothers Reminders Materials for Latin American programs could have offered some tips and lessons learned for IEC materials. Capacity Building Approach A. Project HOPE Planned Approach - Project HOPE’s capacity building goals were identified as follows during their 2001 capacity assessment: • Improved documentation of HOPE methods and experiences; • Improved use of monitoring and evaluation to identify lessons learned; • Improved access to methods and lessons learned through an improved bank of methods and experiences with an improved interface with users; and • Improved active dissemination of lessons learned between HOPE projects and to other organizations through publications. Description of Actual Approach No purposeful changes were made to the actual approach; however, due to the need to focus on other challenges (changes in strategies, reductions in budget and project implementation cycle, 25 amongst others), little attention was given to this area. During the project life cycle, the Senior Technical Advisor attended four CORE meetings where he informally shared lessons learned from the project. Findings/Accomplishments/Lessons Learned According to the Senior Technical Advisor and key headquarters staff person involved with the project, while the learning objectives established at the outset of the project may not have been met, Project HOPE did learn many valuable lessons through the implementation of this project. The four staff members interviewed identified several areas in which they felt their own and the organization’s capacity had been strengthened: • Learning how to improve approaches to work entirely through a local subcontractor; this included learning: 1) how to trust and rely on those on the ground; 2) how to communicate with the MOH via the local subcontractor; 3) the importance of selecting a local partner with political clout and savvy to negotiate the MOH in a country where lines between health and politics are not clear; 4) the required procedures and processes, such as the preparation of formal documents, i.e., MOUs. • Improved regular reporting process, especially financial reporting (which involved additional processes because of working through a local subcontractor); • Improved capacity in LQAS; even though the M & E Specialist at the headquarters had used this sampling method before on an individual project basis, use of the methodology became more institutionalized for other HOPE colleagues, such as the Senior Technical Advisor; • HOPE’s development unit has become wary of taking on 50% cash matches; instead, they will consider GIK for a better part of the match requirement since the agency has this capacity; • Due to the above capacity building lessons learned, (and its experiences in overcoming numerous challenges) the organization will be better equipped to design and implement child survival programs in the future. The evaluation team leader would also recommend that while Project HOPE has taken advantage of the networking and professional development opportunities associated with the child survival program by participating in annual CORE meetings and M & E working groups, and by tapping the expertise available from the Child Survival Technical Support team in LQAS, the organization needs to go beyond this step to seek out opportunities to document and share the unique challenges, potential solutions, and lessons learned of the project with HOPE colleagues in other countries, as well as with the larger CS community through publications and presentations at professional forums. B. CDS Planned Approach- CDS capacity building goals were based on an assessment (the SDMA) conducted by MSH in 2001 prior to the initiation of the project. The plan was to conduct a reassessment in July 2003 and develop a plan of action. Specific capacity building goals as outlined in the DIP were: 1) Improved project design and management 2) Improved provision of technical assistance and advocacy 26 3) Improved transfer of skills and capacity development activities 4) Improved tools development and testing Also, central to the capacity building of the local subcontracting agency were all the changes involved in transfer from its role of a service delivery provider to an organization managing and implementing a project. Actual Approach/Findings Little importance seems to have been placed by CDS senior management on following up on priority areas identified in the SDMA and the subsequent objectives identified in the DIP. In discussions with senior CDS headquarters staff in Port-au-Prince, these CDS staff appeared not to have sufficiently accepted the importance of the deficiencies noted to take specific action towards improving them. In fact, neither the PM nor the CDS headquarters senior managers were aware of which specific areas/objectives had been identified and did not use this document during the project cycle. While the project did not engage in specifically-planned capacity building efforts, CDS’s organizational capacities have increased in the following ways: • The PM became familiar with the DIP as a useful planning and implementation tool; his understanding and appreciation of the document has had an impact on the way he manages the project and on how he plans to manage future projects. • While not mentioned by CDS staff, Project HOPE staff identified capacity building in community mobilization strategies as a result of CDS involvement with the project. Improving capacities in this area was especially important, since CDS had been involved only with direct service delivery and not with community level efforts prior to its experience with the project. • The PM indicated that he will change his management approaches based on the lessons learned from the project (capacity built in program management). Some of the things he will do differently (given the chance) are: holding an orientation meeting between the MOH-DHNE and CDS to carefully go through the DIP together; soliciting greater involvement and ownership of the project by the community early on via a community level orientation to project objectives held with community leaders. • The PM has also changed his personal approaches by identifying and insisting that others explain how a particular strategy will reach the goal/target; by initially establishing a way to measure the effectiveness of the strategy – now he insists at MOH meetings that other colleagues be specific and explain why they believe their proposed strategies will help achieve desired results; he has begun making changes that will effect a new project that CDS is involved in the eastern part of the same department. In fact, during the evaluation process, (based on discussions about weaknesses in the supervision system for promoters) he called a meeting with two of the staff being transferred from the child survival project to the new project to ensure that lessons learned in training, selection, and supervision of the promoters would be applied. • The PM, the CDS Technical Director (based at CDS headquarters in Port-au-Prince), the Project Statistician and other field project staff were introduced to the LQAS methodology, the Verbal Case Review (a method to determine the actual practices of care providers in treating a sick child or patient with STI) and other tools or techniques, such as improved ways to analyze reports which they plan to adopt for future programming. 27 Best Practices/Lessons Learned • If organizational capacity building efforts are not seen as a priority for local subcontractor senior management, this may result in a trickle down effect such that without organizational support for intentional capacity building, it is unlikely that PMs or other project field staff will feel the importance of this intervention. This will likely result in limited attention by the PM and other project staff to own the objectives and to operationalize an action plan. • If a capacity assessment resulting in a concrete action plan with buy-in from all levels of an organization and budget allocations is not conducted specifically within the scope of the project, it is not likely that intentional capacity building will occur. • The mere fact of partnering a local PVO with an international organization, each with their own levels of expertise and organizational capacities will result in bi-directional learning, if not a true increase in capacities, which ultimately strengthens each partner’s ability to design, implement, and evaluate programs effectively and efficiently. • The DIP is a useful planning and monitoring tool, the importance of which project managers need to understand from the beginning of project activities. All project staff involved in monitoring the progress of field level activities need to become familiar and comfortable with the document (especially if they have not participated in its development- see related discussion in Program Management section). • The LQAS sampling methodology has proven to be a helpful way to determine where approaches are not resulting in the desired improvements and has enabled staff to equip themselves with a tool they will continue to use. • While working through a local partner ultimately increases the capacity of that partner, it may require an appropriate interim period in which both partners agree upon target areas in which capacity building will be prioritized to ensure a smooth transition. C. Health Facilities Strengthening and Strengthening of Health Worker Performance Planned Approach/Historical Context- As previously mentioned in the Executive Summary, the project had originally been designed by HOPE in close collaboration with the DHNE. It was based on the lessons learned from two cycles of HOPE CS projects in the adjacent department and project staff was to have been transferred from the CS project in the North to the new project in the North-East Department. During the final evaluation that took place in August 2001, however, the evaluation team concluded that the proposed staffing plan was not appropriate for the new project. After a long recruitment period, a PM was recruited in January ‘02. She began to work with the DHNE and other institution heads, apparently promising support not permissible under USAID standard provisions (salary supplements for MOH staff, direct funds to cover activities given to the MOH, several vehicles, budget for fuel and maintenance, etc.). After having contracted out the baseline KPC survey and attempting to develop a DIP, this first PM was let go in June ‘02. During the following 4 months (July–October ‘02), the DHNE left his position and a new DHNE was hired who had not been involved in the planning of the project. He did, however, hear about the financial and material promises made by the first PM. During the same period, HOPE had been in discussions with CSHGP and the USAID Mission about the future of the project. The USAID Mission in Port-au-Prince strongly encouraged HOPE to develop an alternative 28 management approach and recommended that they consider subcontracting to CDS, a reputable, competent local health and development NGO. HOPE and CDS began negotiations to determine the implementation mechanisms of the new project and a short-term contract was signed between HOPE and CDS in November 2002. The MOH was consulted and requested that a description of the project and its plans be agreed to by the DHNE, HOPE, and CDS prior to his approval of the project. According to the DHNE, he was not informed in a timely manner of the decision made by HOPE to subcontract the project to CDS and only found out after HOPE had subcontracted CDS. The fact that the project had been (re) planned without DHNE involvement and that a number of elements, such as financial and material provisions, the building of an office at Trou de Nord’s MOH UCS (see discussion below) were no longer in the updated planning document (the DIP), and that he had not been informed of the partnership with CDS until after the fact became and continued to be a source of resentment, and resulted in limited willingness to collaborate in project activities. In 2004, when budget reductions required CDS to reduce the intervention area from six communes to four, relations between CDS and the DHNE became even further destabilized. This was felt not only at the level of the DHNE, but all the way to the national level MOH where the Minister, herself, felt that HOPE had not fulfilled its promises. Thus the relationship between the MOH at the North-East Health Department level and CDS was strained from the beginning. As mentioned above, this was due primarily to misunderstandings about the project expectations by the MOH who had understood that budget resources would be provided directly to them and that aspects of the first draft of the DIP would be included into the final DIP. As a result of these strained relations and the desire and need to focus on local level activities, the project abandoned its plans for direct collaboration with the DHNE. These strained relations also impacted the project at the Unité Communales de Santé (UCS, the Central Health Unit) level where overall field supervision was to have happened. The MOH created the UCS to provide the DHNE with a decentralized branch for supervising health facility staff in each commune; however, the UCS- approach has never really functioned in the North￾East Department for lack of resources and an operational plan. To address this deficiency, the project agreed to support two nurses at the USC and to provide a means of transportation (motorcycles) and a computer. These nurses were supposed to supervise field level activities, but they had limited motivation to do so because of the strained relations at upper levels and their limited involvement with the project at the outset. Most of the capacity building objectives as detailed in the DIP (relating to supervision, use of data, management strengthening, and cost recovery at both DHNE and health facility levels, as well as strengthening of provider competencies in IMCI, FP, and STI protocols at the facilities), became irrelevant early on when the project realized that collaboration with the DHNE was not possible, thus preventing any real progress in these areas. 29 Actual Approach/Findings Capacity Assessment/Building at the Departmental Level- No capacity assessment was done with the MOH at the departmental level. According to the Senior Technical Advisor and the PM, this was because CDS staff had met with no success after multiple attempts to schedule the assessment. (The DHNE, however, indicated that while he had asked CDS to conduct a capacity assessment at the department level and he felt they had been in agreement, it never happened). Capacity Assessment/Building at the Facility Level- An initial capacity assessment was conducted at the health facility level using the Service Delivery and Management Assessment (SDMA), a tool developed by MSH and adopted by the MOH at the national level. This same assessment was not conducted as part of the final evaluation however, because the project did not undertake many facility-level capacity building activities, apart from providing materials, equipment and some training. As a result, it was jointly decided by CDS, Project HOPE and the evaluation consultant, to only conduct an inventory of equipment and materials supplied by the project. Based on key informant interviews at the health facilities and discussions with Project HOPE and CDS staff, the following can be reported as ways in which the health facilities and performance of health workers were strengthened: • Material support – The project provided more than $1.3 million in equipment and supplies for the health facilities, the department MOH, and the UCS, including office supplies; medicine and equipment for maternal and child health services; a vehicle to facilitate supervision; motorcycles, fuel, and regular maintenance to help auxiliary nurses/supervisors supervise community activities. • Training – The project provided training of auxiliary nurses/supervisors in IMCI (20) and in Reproductive Health (14); 40 promoters were also trained in topics relating to intervention areas (see Training below). • Community-based information system – The project helped promoters and supervisors increase their capacity to collect important health data at the community level (see below) which was then included in the official MOH data collection forms. • Linkages between health facilities and the community – The project supported the work of the promoters who effectively served to bridge the community with the health facilities through their regular interaction with the communities at the RPs, during community meetings, and home visits; through their interaction with the TBAs; and via referrals made to the health center. • LQAS and KPC – Certain health facility staff increased their knowledge levels and basic skills due to their involvement in using the LQAS sampling method and in collecting KPC data. Lessons Learned/Plans for Sustaining Activities • It is essential to ensure direct and timely communication between the PVO and the MOH during all major project revisions to avoid any misunderstandings; communication is often complicated if it goes through a second party en route to a third, and visa versa. • When turn-over of key project staff and/or key MOH personnel occurs, it is essential that program planning and expectations be clearly understood by each party. Policies and procedures for financial aspects of CS projects (allowable use of USAID resources) should be transparent and explained to MOH partners from the beginning to avoid unnecessary frustrations. 30 • Even if budget limitations translated into a reduction in capacity building activities at the health facility level, the project could have provided opportunities for low-cost ways to make improvements in the quality of services by facilitating cross visits between health facilities in which checklists could be used to identify key areas for improvement which do not require budgetary inputs; • In addition to addressing the transport barrier by providing motorcycles, fuel and maintenance, the project should have also provided training on supervision, supervision tools and support. • Since the auxiliary nurses who were trained in IMCI and reproductive health will remain with the facilities (but under financial support from the MOH) and they will continue to treat children using the IMCI algorithms and provide services in reproductive health, the improved knowledge and very basic level of understanding in these areas (especially regarding IMCI) will continue to help them strengthen their performance; however, they could greatly benefit from on-the-job training and supportive supervision in this area. • Supplies, equipment, and vehicles provided to CDS and the health facilities during the project will remain with these two entities; the Departmental MOH and the CDS PM are currently negotiating which items will be dedicated to use for future activities in the area. This list will be approved by the MOH and the USAID mission in Haiti. Health Management Information System Actual Approach/Findings As previously mentioned, the promoters collected a vast amount of data on health in the community. This information was recorded on approximately ten different forms, including monthly reporting forms, vaccination forms, family planning client forms, RP data forms, etc. Promoters compiled the data from all of these forms into their monthly reports by hand and made three copies of these reports (many did this also by hand, but some did not make copies). Promoters kept one copy of the report and submitted the others to their auxiliary nurses/supervisor at the month’s end. Specific problems were discussed/addressed during monthly meetings with promoters and the data was added to the supervisor’s monthly report which included both facility and community level data. The supervisor then sent her report, along with a copy of the promoters’ individual reports to the Project Statistician who input the data from the supervisor’s monthly report (cross￾checking every promoter’s report to ensure accuracy) into excel spread sheets, which contain summaries of all the project-related data by commune and month. The Project’s Statistician spent a great deal of time following up on this data (at times as much as 1.5 hours reviewing and correcting errors on a single supervisor’s report) and regularly attended the monthly health facility meetings where he helped resolve problems with the data. Once the information was compiled, the Statistician printed the report and discussed it with the PM, comparing month-to-month progress, analyzing for unusual increases or decreases. The evaluation team’s key findings were that while promoters collected impressive amounts of data, and supervisors seemed to understand most of the mechanics of compiling the data into monthly reports, they lacked basic skills in data-based decision making. Neither promoters nor their supervisors were able to tell by looking at their reports if they had experienced an unusually 31 low number of total children who came to the RP, or if the number of malnourished children had been reduced. Challenges --As previously mentioned, promoters collected vast amounts of data, but this data held very little meaning for them. Promoters did not analyze the data. --Monthly reports do not show target values, therefore, it is difficult for the promoters or supervisors to assess their own progress and identify activities that need to be intensified. --Because little middle management supervision was provided by the UCS level to support the work of the auxiliary nurses, project staff (the PM and the Statistician) had to spend much more time than they could afford assisting these nurses and the promoters. Lessons Learned/Plans for Sustaining Activities • Project staff and supervisors should have worked together at the outset of project to determine how they could develop a manageable health information system (consolidating the 10 forms into the minimum number of forms possible). • The promoters and health center supervisors need to be oriented towards measuring impact through monitoring change in indicators rather than using the information system as a way to count activities. It would have been helpful to ensure that both health centers and the project office had indicators and graphs on the wall monitoring progress. • Promoters can excel in collecting large amounts of data, but they need specific guidance in how to use this data to make decisions. Promoters feel a sense of pride in their work; the majority of promoters are diligent in completing their multiple reporting forms. • The project staff need to transfer their knowledge of the project’s health information system to individuals at the UCS (if it receives support in the future for supervision activities of another project) or other departmental MOH staff to ensure that these skills/tools are shared and capacity is built amongst MOH staff who have regular access to computers. Training Planned Approach According to the DIP, the training plan included the training (and supervision) of: --81 promoters (in adult education methods, facilitation of mothers/fathers clubs, CS interventions, weighing, counseling, IEC); --30 animatrices in EBF and breastfeeding support; --30 auxiliaries in technical interventions - IMCI, FP, and STI management protocols, community outreach, IEC, adult education, and supervision; --30 CBDs in family planning, and ORS; --85% of inventoried TBAs in safe pre-natal and delivery techniques, and problems requiring referral. Training was to be conducted in two phases, the first to include training and follow-up for promoters in two communes in Year 2 with training for the remaining four communes to follow 3-6 months later to enable the training team to apply lessons learned. Auxiliary training was to take place for all six communes together to upgrade facility services at the onset of activities. 32 The IMCI training would follow the standard 14-day curriculum; all other training would use the “sandwich approach” with a module followed by assignments and practice for the trainee prior to the introduction of the next module. Methodologies were to emphasize adult learning approaches with practicum experience. Auxiliaries were to provide intensive supervision and refresher training to reinforce new skills. More experienced, competent promoters would be identified and would serve as mentors for their less experienced colleagues. MOH and CDS staff seconded to the MOH would be trained as trainers and mentored by CDS staff; lessons learned and curricula from CDS similar activities in nearby communes would be applied. Actual Approach/Findings A total of 40 promoters, 8 health agents, 20 auxiliary nurses, 13 pharmacists, 48 CBD managers, and 400 representatives from community organizations were trained in various topics (see chart below). Because of timing difficulties and reductions in the budget, intervention zone, and implementation cycle, the following did not take place: phased training (the project had to reduce its target area to two communes); the training of the 41 promoters (although 12 of them participated in the training, the project was unable to continue working with them); the training in IEC and counseling, facilitation of mothers/fathers clubs, and adult education methods for promoters; training for auxiliaries in community outreach, IEC, adult education, and supervision; training of animatrices in EBF and breastfeeding support. The training of TBAs was also eliminated since the MOH includes them in their own training. 33 Training Table Dates of the Training (dd/mm/yy) Total Days/Hours of Training Types of People Trained Numbers Trained Topics Covered Affiliation of Trainers April 28 – Sept. 30, 03 47 days = 282 hours (most 5 day sessions) Promoters & MSPP Health Agents (Terrier Rouge & Trou de Nord) 40 promoters 8 MSPP Health Agents Vaccination, Growth Monitoring, Family Planning, Breast￾feeding, Maternal Health, STD/ HIV/AIDS Community SIG , diarrhea, TB, Community mobilization, ARI 4-5 Trainers each topic; 2 MSPP, 2 CDS May 12 -24, 03 14 days = 84 hours MSPP Auxiliaries 20 IMCI 5 trainers MSPP, CDS June 5-6, 03 2 days = 12 hours MSPP Auxiliaries 14 Reproductive Health Norms 3 trainers MSPP/CDS June 12 -13, 03 2 days = 12 hours MSP pharmacists 13 2004 4 hours Community Based Distributors (CBD) 48 Topics related to the commodities provided to them: condoms, ORS etc. MOH nurses 2004 10 hours Community Organization Representatives 400 men and women 11 MCH topics MOH nurses Also contributing to the reasons for reducing the training topics for the auxiliary nurses to only IMCI were the difficulties/limited collaboration with the DHNE and the CDS nurses posted at the UCS. Project staff felt they had to eliminate substantial project objectives (most of those related to strengthening the health facilities) whose success would depend on collaboration with the DHNE. Thus, no supervision training was provided nor were tools developed to help support the auxiliaries in their supervision of the promoters. The training of animatrices did not take place because, as previously mentioned, the PM (in consultation with the Senior Technical Advisor) felt that many of the mothers/fathers clubs (and VHCs) were inactive. He therefore proposed and implemented what he felt would be a more sustainable strategy--training CBO members. Training of the 40 promoters took six months (April – Sept 30, 2003), and promoters did not start working until all of the training was complete. Promoters received 47 days (295 hours) of training. The nurses received 14 days (84 hours) of training. The evaluation team learned that the PM had tried to negotiate with the nurses who had agreed to be trainers for a shorter training session for the promoters, but they insisted on keeping the training to a maximum number of days presumably to ensure more total per diem. Instead of using the sandwich approach as 34 planned, several lengthy modules were put together resulting in a training sessions which lasted from 2-3 weeks (instead of a couple of days at a time) and at times with only a week in between. The evaluation team found no evidence of: follow-up conducted after the training, tools for on￾the-job refresher training, or mentoring support between promoters. Challenges --The length of training, especially for the promoters, represented an unfortunate financial and time expense. The fact that promoters did not start working until the training was completed meant several months of missed opportunities. --Project staff found it difficult to negotiate with the trainers (nurses) for a more manageable timeframe; they were motivated by the financial incentives of increasing their own per diem. --The evaluation team found no way to verify the quality of the contents or use of adult education methodologies. The training guide consisted only of a list of training topics for each health intervention area and no detailed lesson plans that contained the key messages and detailed content. Messages were developed by each facilitator and put directly on flip charts that were not kept or available to the evaluation team. Curricula design was not competency based. Rather, they included information not necessary to the effective functioning of the promoter, such as the history of vaccinations, different qualities of sources of milk, and different types of diarrhea. Some objectives mentioned in the training guide were not covered by the content (such as ARI). Only suggested training methods (brainstorming, role plays, etc.) were included on the training guide, no specifics were provided regarding the steps to correctly facilitate these methods, or how to process important questions. --Discussions with some of the trainers indicated that some of the project’s key messages when disseminated during the training had not been consistent, nor complete (hand-washing, exclusive breastfeeding to reduce diarrheal disease, etc.) Lessons Learned/Plans for Sustaining Activities • The training strategy for promoters and auxiliary nurses was not as efficient and effective as it could have been. If the training had been conducted in a more efficient way, more people could have been trained, additional in-service training could have been provided and the cost to the project would have been lower. • Training curricula should be designed focusing on specific competencies that project staff expect the participants to gain. Learning needs assessments should be conducted prior to each training to determine specific areas which are particularly weak for training participants. Mentoring between participants should also be explored as a low-cost way of ensuring reinforcement of key messages and skills. • Monthly promoters/supervisors meetings offer an opportunity for systematic and sustainable technical reinforcement and refresher training; MOH should support supervisors in encouraging them to develop mini training modules on topics for which promoters show special challenges. • While promoters are knowledgeable in key messages related to HIV/AIDS/STI prevention, family planning, maternal and newborn care, immunizations, and ORS 35 preparation; they could use a mini-module to reinforce messages on IRA, diarrhea prevention, and how to respond when a child’s weight gain is not satisfactory. • While the project currently has no plans to sustain training activities beyond the life of the project, most of the above suggestions can be incorporated into future MOH training and other CDS projects with little or no additional monetary inputs. • See section on Community Mobilization for challenges and lessons learned in training CBD managers and representatives from community-based organizations. Sustainability Strategy Planned Approach According to the DIP, the following were selected as sustainability objectives: 1) Percent of promoters who graduate at least two mothers/fathers clubs per year 2) VHCs formed and running regularly 3) Volunteer animatrices supporting new mothers in EBF 4) Mothers demonstrate appropriate knowledge and practices at target levels 5) Percent of communities that have active VHCs 6) Percent of promoters who use correct basic health education messages To achieve these objectives, the DIP refers to a project-long transition process. This was to provide a mechanism for retention of the promoters, some of their outreach activities and on￾going activities at the community level via the VHCs and other CBOs, thus shifting significant responsibility for health services to the community. Actual Approach/Findings Of the six sustainability objectives identified in the DIP, only #4 and #6 above: Do mothers demonstrate appropriate knowledge and practices at target levels? and What percent of promoters use correct basic health education messages, remained pertinent due to the changes in the project activities (i.e. elimination of the work with Village Health Committees, mothers and fathers clubs, etc.). See related discussions above. While these two objectives were met in part (targets were met for approximately 80% of the indicators relating to mothers’ knowledge and practices and approximately 75% of the promoters could state nearly 80% of the messages correctly), no attempt was made to update the sustainability plan during the life of the project. Even though the PM had attended a session at the Mini-University and was truly interested in the Child Survival Sustainability Assessment framework9 , this area was not prioritized. In his words, “When I returned from Baltimore, I had so many things to share with the staff; I didn’t put it as a high priority…” Technical assistance was not provided by Project HOPE headquarters in the area (other than including sustainability related indicators in the DIP), and thus very little attention was dedicated to sustainability plans. 9 Sarriot, Eric: “Child Survival Sustainability Assessment” CORE/CSTS document, September, 2002. 36 Despite this being one of the weaker strategies of the project, it is probable that the majority of promoters will continue to provide services after the project due to strong commitment by the promoters, increased demand for their services by the community, an appreciation of the services by the mothers, and commitment by health staff to continue to support and supply the promoters. As one promoter stated regarding the potential of continuing with the majority of project level activities, “I was involved in health activities before the project and will continue to work even without the support of the project.” Discussions with supervisors also mirrored this attitude, “Of course the project helped them [the promoters] do a better job in the communities, but the promoters are dedicated and want to contribute to the health of the communities.” Challenges --Due to competing priorities (and perhaps a limited sense of the importance of addressing the sustainability framework early in the project), the PM was not able to give the time necessary to share the framework with his colleagues; --No point-person was identified to champion the framework or to follow-up on revising the sustainability plan; --HOPE headquarters was not able to provide additional support to encourage the PM to follow￾up on the framework or objectives. Future Plans/Lessons Learned/Recommendations While no technical or management assistance is planned after the project ends and no specific financial sustainability plan was put into place by project staff, they have been involved in negotiating several possibilities and the following phase-over and future financial sustainability options appear promising: 1. Some of the promoters have already been transferred over to the MOH and are being paid through a Plan/Haiti-sponsored project. The auxiliary nurse at the Grand Bassin health center submitted a request to Plan/Haiti, who agreed to pay for the promoters and the supervisor, as well as to purchase supplies for the community activities. 2. As mentioned earlier, the items purchased by the project, such as vehicles and motorcycles, computers, and medical equipment will remain at the MOH facilities. CDS staff is negotiating with the DHNE the best use of these items for sustaining future health related activities in the area. 3. Some of the CDS field project staff is currently working on another project under the umbrella health project HS 2007, financed through the AID mission and implemented through MSH, PLAN, and CDS in the eastern part of the North-East Department. It is probable that numerous “lessons learned” from the CS project will be transferred to this project. 4. Another project, funded through PEPFAR, is on the verge of being signed and once approved, it is likely that more than half the promoters will be supported under this project. 5. In addition, a new project that the Inter-American Development Bank is financing is paying promoters for each Rally Post they conduct. While the evaluation team was unable to obtain more information about this project, it raises a real concern regarding the potential for attention to quantity over quality. Without some guidelines regarding minimal numbers of individuals who must receive services during each RP and some checks on quality, promoters 37 who receive financial incentives for the quantity of RPs held may lose sight of the main purpose of these activities. 6. The PM and DHNE should hold a meeting with promoters and their MOH supervisors to consider various ways that the promoters can sustain their activities with MOH and community support. For example, one promoter mentioned that the three things that would help them continue their activities with the community were: support from MOH authorities, transportation reimbursement/assistance, and creating a health workers association. 38 CHAPTER THREE Program Management Planning As previously mentioned, the original project was designed by HOPE in close collaboration with the DHNE; in fact, based on HOPE’s previous achievements in the North Department, the MOH was quite favorable towards extending HOPE’s work in the country. Through a series of collaborative meetings between HOPE and the MOH, the North-East Department was selected as the project site with HOPE providing institutional capacity building support to the MOH as the main implementing partner. A Project Manager was recruited in January ‘02. According to the current DHNE (who did not work directly with her, but rather learned about her through the heads of the health facilities who worked directly with her), she was very open and inclusive during the planning process. Between January and May 2002, the PM worked closely with the MOH institution heads and other local counterparts. According to the DHNE the result of these discussions was the development of a draft program planning document, in which “all the MOH institution heads had been involved”- this was the first draft of the DIP. By all accounts, it appears that the planning process through this point was inclusive and that the MOH at both regional and local levels had fully participated. Unfortunately the PM was not aware of the standard provisions which govern CS cooperative agreements, or with USAID/Haiti policies, and the planning document contained promises to MOH officials that Project HOPE could not keep. In May ‘02, the Project Manager’s contract was terminated. The period between June–November 2002 was spent in discussions between Project HOPE, CSHGP and the USAID Mission exploring options for project implementation and at one point the USAID Mission recommended that HOPE subcontract project implementation to CDS, a well known Haitian NGO that was already carrying out service provision in the same department with USAID funds. CDS became the subcontractor in charge of project implementation in November 2002 and a new Project Manager was recruited and began in December ‘02. In the preceding months, multiple drafts of planning documents had been exchanged between HOPE and CDS which became the basis for the DIP draft. In February ’03, a five-hour meeting with the DHNE, his senior staff, and CDS was conducted as part of the planning process. According to the Project Manager, HOPE headquarters asked the DHNE to review a draft of the DIP and prepare a three￾month work plan, which he did. The DHNE, however, felt that he had not been provided enough time to respond to the overall document and was disgruntled about the changes in the plan which had much less focus on MOH strengthening and direct support. While the CDS PM had been included in the program planning discussions, during the final evaluation he expressed the notion that he could have been more directly involved in the preparation of the DIP. Perhaps due to the need to expedite the submission of the document, the PM felt that he had to rush through the planning process. Since it was his first time with a planning tool of this type, he felt ill prepared to give the type of suggestions that were required. 39 He also felt that the document was pre-conceived and that his suggestions were perhaps not taken as seriously as they should have been. It was not until he returned from the Mini-University in June 2003, after he had time to reflect, that the PM realized that several approaches in the DIP were not practical, nor realistic and he subsequently eliminated these from the work plan - PD/Hearth component, school children peer education approach, etc. He also assessed that the Management Assessment part of the SDMA tool for health facilities was not realistic and thus, it was put at a lower priority. Regarding the DIP document, the DHNE was only asked to provide his opinion on which targets were most appropriate for the indicators. It is important to reiterate here that many of the targets for the indicators were extremely ambitious. In fact, the evaluation team learned that several of these targets were automatically set in accordance with the MOH’s standards for Haiti which were not realistic for the North-East region. Other project staff were not involved in the DIP, nor did they seem familiar with the document. One obvious weakness of the DIP was that it contains 18 different objectives with 80 indicators. While the majority of these objectives are actually outcomes, they were mistakenly placed at the results levels, revealing a lack of understanding of the logical framework. This overly complex planning matrix made it difficult for the PM to manage the project, which would have been challenging in the best of circumstances. Furthermore, the Work Plan and the Performance, Monitoring and Evaluation Plan were combined into one document which resulted in limitations on the level of detail provided for the implementation of activities. It is likely that a greater degree of involvement of more experienced individuals in Haiti and a more participatory approach (such as a DIP workshop) would have prevented some of these pitfalls. Conclusions/Lessons Learned • The DHNE’s limited involvement in the planning process for the redesign of the project had serious implications on how the DHNE saw its responsibilities towards the project and continued to contribute to the deterioration of collaboration between the Departmental MOH and the project throughout its life cycle. While the project’s initial planning process appears to have been quite inclusive from the beginning, the project should have ensured that the first PM had a clear understanding of the limitations of CSHGP projects. A stronger management structure in country, such as a Project HOPE presence at the country level could have helped to avoid many of the misunderstandings, confusion, and unfulfilled expectations of partners. • Having over 80 indicators (and several with unrealistic targets) for the program made the work plan quite impractical, especially in light of the fact that the project was already a year behind. • The planning process could have been strengthened by holding a DIP workshop to jointly build team consensus and a greater sense of ownership among stakeholders. • If time was too short for a planning workshop, HOPE and CDS could/should have at the very minimum conducted an orientation meeting to familiarize MOH and CDS colleagues with the DIP document as a planning and implementation tool. • Both the PM and the DHNE could have benefited (as well as the project) from more time to review the document and actual direct involvement in its preparation (not only in reviewing drafts). 40 • A series of community level planning meetings should have also been held to ensure community level buy-in and commitment from the start. Staff Development Findings Please refer to the section on Capacity Building for changes in knowledge, skills, and competencies of the project staff. The project provided CDS staff at the headquarters and field project level with an important staff development opportunity during the first year of the redesigned project. Three members of the organization, the PM, the Executive Director, and the President of the organization were supported by the project to attend the Mini-University. As earlier mentioned, the PM did not share the key learning from the Mini-University, such as information regarding the sustainability framework and other lessons learned from child survival projects, with his project colleagues or other stakeholders. Conclusions/Lessons Learned Though he was appreciative of the opportunity he and his two colleagues were given to attend the Mini-University, the PM generally felt he could have benefited from other opportunities for professional growth. Considering the challenges of managing this project and the very short amount of time for the implementation of project activities, it would have been difficult to justify having the PM leave the project for lengthy periods of time to attend formal training in the US; however, as he suggested and the evaluation team leader concurs, he could have benefited from cross-visits to other Project HOPE or other PVO projects in nearby Latin American countries. Language would not have been a problem due to his fluency in Spanish and other CS projects in the region would have provided him with ideas for how to better manage his own project. The Project Statistician could have also benefited from internal, on-the-job training or a cross-visit to another NGO working in maternal and child health, such as CARE, to study their statistical reporting/information system in efforts to make it more user-friendly and efficient at the community, health center, and project level. While substantial resources were invested in sending three CDS staff to the Mini-University which provides an excellent opportunity for staff development, there was no evidence that the lessons learned, new skills, best practices/approaches were applied or transferred to other colleagues or partners in the field. As earlier mentioned (see Capacity Building), the PM should have had an action plan of how he intended to use/share the lessons learned once he returned to the project area; also, a HOPE headquarters mentor could have helped encourage him to complete the action plan. The evaluation team leader feels that a more strategic use of the funds that were used to support the two other CDS headquarters staff in attending the Mini-University would have been very deliberate, well planned cross-visits with specific objectives for the PM and/or his other key field-based project staff, such as the Technical Assistant or the Statistician to nearby child survival projects. Another option that might have actually made a significant impact on improving the relationship between CDS and the DHNE would have been for HOPE to encourage CDS to support the participation of a DHNE representative to attend the Mini￾University along with the PM. 41 42 Regional Dir. Africa/Haiti Tech Advisor -- Haiti Asst Reg Dir Africa/Haiti H 0 P E U S Sen. Vice Pres Intl Operations Technical Director CDS President – Project Leader CDS Executive Director Financial Director C D S P a P Project Field Office Director Statistician Project Administrator Project Technical Director Secretary-Accountant Support Personnel C D S F i e l d Broken line box yellow fill represents HOPE employees Dotted line [ ] represents alternative reporting channel for HOPE field employees PROJECT ACCOUNTABILITY STRUCTURE HOPE-CDS-MSPP Project -- Haiti C O M M U N I T Y --- Village Health Committees DEPARTMENT OF HEALTH OF THE NORTH-EAST (MSPP) Supervision The different levels of support and supervision consisted of the following: 1) HOPE/HQ- Technical assistance and support in programmatic issues were provided primarily by the Senior Technical Advisor and the Monitoring and Evaluation Officer. These individuals were supervised by the Senior Vice President for International Operations. Additional assistance as needed was provided by the Regional Director for Africa (primarily at the beginning of the project), the Assistant Regional Director for Africa (the primary administrative back-stop for the project), the HQ Financial Officer for the Africa Region and a GIK Officer (regarding provision of materials/supplies). Because HOPE had subcontracted to CDS, there were no supervisory lines/expectations between HOPE and CDS. 2) CDS- The CDS President, who is called the “Project Leader” in the DIP, works from the CDS headquarters in Port-au-Prince. Reporting directly to him was the CDS Executive Director, and reporting directly to him were three individuals with varying degrees of association with the project: the Technical Director, who, while involved in other projects, was to make regular field visits to provide on-going local TA; the Financial Director, who oversaw issues of finance primarily at the headquarters level, but who was also responsible for the project’s overall financial oversight–in coordination with the Project Accountant; and the Project Manager (PM) - referred to in the DIP as the Project Field Office Director. At the Field Office level, which is a day’s drive or a short flight from the CDS HQ, the project staff included the PM. Reporting to the PM were the Technical Assistant, who served as a field supervisor; the Statistician, who served in a Monitoring and Evaluation role; a Project Administrator, a Secretary/Accountant (these last two positions were collapsed into one towards the final year of the project- the accountant served in this new role), and other support personnel (including two drivers, and housekeeping staff). 3) MOH- As previously mentioned, the project staff supported two nurses at the MOH-UCS, one in Terrier Rouge and the other in Trou de Nord. According to the original project design, these two nurses were supported by the project, but supervised by the DHNE; they were also supposed to provide supervision to the auxiliary nurses at the health facilities who were the direct supervisors of the promoters. Findings/ Conclusions/ Lessons Learned CDS Staffing Structure and Supervision While no one seemed to complain to the evaluation team, this staffing structure at multiple levels was confusing and illogical. Neither the staffing structure, nor the supervision system appears to have been very effective. This was likely due to the complicated structure of the system, and the fact that while many positions were part of the structure, only a few of the individuals in these positions provided the level of support they had been designated. For example, the PM was directly supervised by the Director of CDS at the national level in Port-au-Prince, and supported by the CDS national level Technical Director. The Technical Director visited the project site to assist with the SDMA and then again, for very short periods on only a couple of occasions over the life of the project. Project staff could have benefited from lengthier and more frequent visits by this CDS internal technical resource. 43 On the other hand, the PM and the CDS Executive Director did seem to have an effective communications and supportive feedback system in place. While the Executive Director made relatively few trips to the project area to discuss the CS project, he maintained near-daily telephone and email contact with the PM and provided on-going guidance (this was no easy feat as the project area is quite isolated and had very limited communication channels.) MOH- Supervision between MOH/UCS and Auxiliary Nurses Regarding the supervision between the project-paid nurses seconded to the MOH- UCS and the MOH auxiliary nurses (who supervised the promoters), there was quite a bit of confusion as to who had supervisory responsibility between CDS and MOH. While the project was paying for the nurses based at the UCS specifically to supervise the auxiliaries at the health facilities and the promoters in the field, they performed this task infrequently and grudgingly (and expected the project to provide some additional monetary incentives beyond their salaries). Their poor attitude and lack of motivation was, in large part, due to the on-going discord between CDS and the DHNE (see above discussion in Planning regarding the historical antecedents of this discord). According to the PM, the primary reasons for the non-functioning of the system were that while the MOH had wanted the project to strengthen the UCS (which it did through the provision of staff, vehicles and computers), there was not enough discussion at the beginning with respect to the MOH’s expectations, nor an operational plan as to how to set-up and strengthen the supervisory system, and no position description for the two UCS staff. In his view, neither CDS nor the nurses at the UCS wanted to take the initiative as to how to improve the system. The PM had contacted the DHNE at the beginning, but he was not receptive to discussing supervisory roles with the PM. Supervision between Auxiliaries and Promoters While the rapport between the promoters and auxiliaries appeared quite good, and promoters reported being visited fairly regularly by auxiliaries, formal structured and planned supervision of promoters by auxiliaries was clearly deficient. The auxiliaries did not know about supervision and the project did not train them; consequently, these individuals had varying degrees of understanding of supervision and the process; standard procedures and tools were lacking rendering an analysis of results of supervisory visits impossible. Fortunately, discussions with the PM and other project staff indicate that this is a key lesson learned from the project and that a greater priority will be placed on effective supervision systems in future programs. One element of the project’s approach to strengthening the supervision system that did seem to be effective in at least increasing the quantity of supervision visits was the provision of motorcycles, gas and maintenance so that supervision of the promoters would be easier. During interviews, the auxiliaries mentioned the provision of motorcycles, fuel, and supplies as having had an impact on their ability to provide increased supervision. Four out of five motorcycles provided to these auxiliaries remain in good working condition and will most likely continue to help the supervisors increase their support to the promoters in the communities. Project Level Supervision While the evaluation team leader found few examples of effective formal supervision at the CDS project level (no performance reviews, no regularly scheduled feedback mechanisms, no tools for 44 supervision, etc.), the PM provided informal supervision to his project staff in the form of occasional staff meetings (called when required as judged by the PM); informal impromptu one￾on-one discussions regarding various program aspects; and primarily meetings that he felt were required due to some error or performance issue. The PM could have benefited greatly from supervision training, tools and role models to help him see the positive aspects of supportive supervision and mentoring. The best supervision which seems to have been provided was from the level of the project staff, the Technical Assistant/Field Supervisor and the Statistician to the auxiliary nurses and promoters who acted as mentors to these individuals regularly attending their monthly meetings and helping them to work through issues at the field level. Supervision was one of the least supported of all the program management areas. While good relations were maintained between all of the levels, none appear to have provided solid role models in supervision. Project HOPE staff felt they were not mandated to provide direct supervision since they were implementing the project through a sub-contract with a local NGO, CDS. In general, the PM felt supported by the feedback provided upon his request from the Project HOPE headquarters Senior Technical Advisor and the Assistant Regional Director for Africa, as well as from his own supervisor in Port-au-Prince. He appreciated the quality of comments he received from the Project HOPE Senior Technical Advisor when he submitted quarterly reports and the rapid response he received from other HOPE colleagues, such as the Assistant Regional Director for Africa. This being said, it is the evaluation team leader’s opinion that informal, supportive supervision from the Senior Technical Advisor to the PM could have been very helpful and potentially have resulted in greater transfer of skills. It should be mentioned here, however, that he made very short trips (3-4 days) to the project site, and prioritized the type of input he provided to issues primarily within his direct expertise, technical quality. Human Resource Management Please refer to Supervision above for a description of the staffing pattern. Few policies and procedures for appropriate HR management were established at the project level. As mentioned under Supervision above, staff were not given performance reviews, nor did they develop individual performance objectives based on the project objectives. The community development strategy could have been developed earlier and more extensively if the project had recruited someone to work in community development (or perhaps a more cost￾effective option would have been for the MOH to second someone to fill this role). The core staff were stretched too thin because they were essentially performing the supervision function of the MOH in having to work very closely with the health centers. Also, the lack of consistent technical health (clinical) staff at the middle management level compromised the project’s ability to assure consistent quality in the development of messages and training content and to effectively track progress and provide necessary follow-up on new approaches to the interventions. Key project staff working directly with the PM were self motivated which affected program implementation perhaps more than the morale or relationships between colleagues which was 45 also positive, especially between the Technical Assistant/Field Supervisor and the Statistician who often made joint rounds to support the promoters during their monthly meetings. Some staff would have liked to have been more involved in decision making and activities planning. They also felt that the sharing of information could have been improved (especially regarding the CSSA and other technical aspects which the PM had learned at the Mini-University). They would have preferred a more on-the-job, training related type of supportive supervision and more opportunities for joint planning. For example, one staff member related “one of the reasons for the lack of a more harmonious relationship between the MOH and CDS was that there was no participatory planning; activities implementation became more of an imposition – a planning calendar and regular regional level meetings with the DHNE would have helped.” The level of staff turnover was relatively high: the first PM lasted only five months; the Project’s Administrator lasted only until late 2004 (due to insufficient skills in budgeting and less than ideal performance levels); the first Technical Assistant/Field Supervisor left the project prior to its final semester to take a position with CARE (for which he was offered twice the project salary); his successor lasted only one and a half months; a month later, in October 05, when the project was nearing the end, another successor was recruited. This turnover coupled with the highly volatile political environment in Haiti meant that months sometimes passed when project activities were stalled or at least seriously hindered. As previously mentioned, the PM, who is currently negotiating his own follow-on contract with CDS, plans to retain all direct project staff and is recommending that at least half the project promoters be hired to work on the PEPFAR project when approved. Financial Management Although CDS financial management at the field level was efficient and project accountability for finances and budgeting were appropriate, the financial structure for this project was less than ideal. Long delays between appropriation of funds from the Project HOPE headquarters and the actual availability of these monies for project activity expenses in the field meant that staff often had to wait a month and a half before receiving the money. This was due to the way the system was set up with funds being transferred through a US bank to a Haitian bank, then a hold being placed while checks would clear in the US. Another delay was caused by the fact that each time funds were disbursed from HOPE headquarters to the field they had to pass through the CDS headquarters, causing an additional delay in getting funds out to the field. The delay was admittedly in both directions. Often project staff would encounter challenges in obtaining the receipts from small businesses or individuals in the field to justify the expenses with HOPE headquarters; this caused a delay in reporting actual expenses and further delayed the release of the next allocation of funding. The project budget was adjusted in late 2004 when HOPE realized they would not be able to raise the required cash match of $1.3 million. Potential financial resources do appear to be in place (through PEPFAR and HS 2007) to ensure that many of the activities initiated during the project will be carried on beyond the cooperative agreement. Also, as alluded to in the section on Capacity Building, more than $670,000 worth of 46 vehicles, office and medical equipment and supplies have been provided to the MOH and will remain with them for use in supporting continued activities. In terms of budgeting skills to help with sustaining sound financial management systems, both the PM and the Accountant (who has been with CDS for 12 years) have developed effective budgeting skills to accurately estimate costs and elaborate on budgets for future programming. Logistics The project purchased several items, such as a washing machine, air conditioners, LCD projector, etc., which at the end of the project have not been used; a better projection of needs would have substituted these purchases with items that the project desperately needed to improve its communication system (such as a satellite phone and/or radio communication system). Information Management As explained in the section on Capacity Building, the project developed and utilized a comprehensive system to collect, record, and map data on monthly spread sheets. Despite some misunderstandings about data collection this appeared to be an effective system for collecting and managing information on the project activities. Once the monthly data was compiled (please see Health Management Information System above) from supervisors/promoter’s monthly reports, the Project Statistician and the PM discussed progress towards process objectives. The KPC used with LQAS sampling also proved to be a very effective approach to measuring progress on results indicators and making programmatic/management decisions. While the PM and the Statistician regularly analyzed the data at the central project level, additional skills building will be required for program staff at all levels to become comfortable with making decisions based on an analysis of the data collected. The program indirectly strengthened the MOH’s data collection system by using MOH staff to conduct field work (baseline and final KPC surveys, and qualitative evaluation) and by continuously working with the promoters to improve the quality of their reporting on community health aspects. While HOPE and CDS field project staff may have had an idea regarding the project’s achievements, it is doubtful that prior to the evaluation, the majority of the project partners had a clear understanding of what the project has achieved. Technical and Administrative Support As previously mentioned, the Senior Technical Advisor was the primary provider of programmatic technical support during the project cycle. He made approximately two short trips to the project site each year. In addition, the administrative backstop, the Assistant Director for the Africa Region and a Finance Officer visited the project and assisted in setting up financial systems at the outset of the project. While CDS staff felt satisfied with the support provided, they felt it could have been more helpful in other areas, such as supervision and monitoring and evaluation. Finally, the project hired an external evaluator for the final evaluation. Approximately 114 person days per year of Project HOPE staff time has been devoted to supporting this project. 47 Collaboration with USAID Mission Project HOPE and CDS have been in continuous contact with the Mission throughout the duration of the project. The Mission was instrumental in encouraging HOPE to retain the project when, initially, the challenge of replacing the first PM and regaining lost time seemed overwhelming. USAID directed Project HOPE to work with CDS as an implementing partner in the North-East Department. Because the Mission’s health objectives were fairly broad, many of the project’s objectives contributed to the objectives related to women and children. The Mission was also contacted for assistance with the challenge of substantial budget cuts. The project results and lessons learned have not yet been used by the Mission, but discussions with Chris Barratt, the head of Health, Population and Nutrition at USAID/Haiti indicate keen interest in developing a small research project to study cost recovery to demonstrate the impact community level activities have had on increasing health center revenue. The project used tools from the bilateral project, HS 2004, such as the SDMA; however, no follow-up was conducted. The project staff also participated in the annual strategic planning sessions in December 2004 and 2005 organized by the DSNE and MSH for the whole department. MSH and the Mission staff were part of the team that conducted interviews at the beginning of the project in order to select the field staff of the project Also, the CDS and MSH representative in the North-East department have established good relations. On many occasions, he has served as an intermediary between the project and the DHNE. Program Management - Lessons Learned • A stronger management structure in country, such as a Project HOPE country Office would have been helpful in avoiding some of the difficulties between the Departmental MOH and CDS (see discussion on Planning); • Focused technical assistance in program management, financial planning, human resource development, supervision, information management, and developing training strategies can ensure that relevant project staff have the appropriate skills to perform their jobs in a high quality fashion and that project staff provide good role models to their partners (see discussion on Supervision, Human Resources, and Technical Support); • Improved budget appropriation process can help project ensure that activities continue to move forward without delays (see Financial Management); • An improved staffing structure could have enabled the project to make earlier and more remarkable progress with community mobilization activities (see Human Resources). 48 CHAPTER FOUR CONCLUSIONS & RECOMMENDATIONS This was a complicated project that faced many challenges; yet despite these, the KPC results and qualitative data collection show remarkable results. Accomplishments The most significant accomplishments include: 1. Significant change in the majority of project indicators, reaching project objectives in most cases, for two communes with a total target population of nearly 56,778. The greatest achievements in improved behavior change and health status were in: o Improved coverage for Vitamin A o Immunizations for mothers and children o Increased access to safe maternity care o Increased immediate and exclusive breastfeeding 2. Development of a replicable and sustainable model for community level interventions on a large scale through training and support of promoters 3. Improved access to and demand for health services at the community level Constraints The most important constraints affecting project results were: 1. Monitoring System/Data for Decision-Making: While much information was gathered regularly and many efforts were made to strengthen the data, it was not used to make strategic decisions. There were also misunderstandings about calculating targets that were problematic. Promoters were expected to complete too many data collection forms. Better forethought was needed in determining what data was essential when developing the monitoring system. 2. Training: The training strategy for promoters was not as efficient and effective as it could have been. If the training had been conducted in a more efficient way, more people could have been trained and additional in-service training could have been provided. There is no way to verify the quality of the contents of the training because there were no detailed lessons plans that contain the key messages and detailed content. Curricula design was not fully competency based and some objectives mentioned in the training curriculum are not covered by the content. 3. Supervision: Supervision of project staff by the PM and of the MOH health facility staff by the project and DHNE was deficient. The latter was due in large part to unclear expectations and an ineffective UCS. While the PM feels that supervision of promoters was lacking – in terms of time spent in the field – the most obvious deficiency documented during the evaluation was with 49 the quality of the supervision. Auxiliaries lack training, tools, and standard procedures for carrying out supervision. 4. The Partnership between MOH and CDS/HOPE was not as productive as anticipated. There was a lack of full cooperation and mutual support between these two partners which resulted in less than acceptable support to the project. Plans to strengthen the Department MOH were abandoned when their involvement in the project appeared less strategic. 5. Community Mobilization efforts were not strong enough and were initiated too late in the project to ensure strong community support for the community-level health activities. Work with the CBOs focused on increasing knowledge but neglected ways for CBO members to help promote the targeted behaviors. The CBD strategy did not include a marketing component to ensure CBD-recognition and CBD training did not include enough information about stock management and renewal. The staff structure at CDS/HOPE was too thin to provide sufficient support for community mobilization efforts. Lessons Learned Key lessons learned include: 1. An improved staffing structure could have enabled the project to make earlier and more remarkable progressing with community mobilization activities. 2. Training curricula should be designed focusing on specific competencies that project staff hope to develop amongst training participants; learning needs assessments should be conducted prior to each training to determine specific areas which are particularly weak for training participants; mentoring between participants should also be explored as a low-cost way of ensuring reinforcement of key messages and skills. 3. From the beginning of the project, managers need to establish a comprehensive supervision system that includes training, procedures, tools and support/mentoring. 4. Project staff need to develop a practical, yet comprehensive system for monitoring project progress and helping promoters and their supervisors to make decisions based on essential data collected at the community level. A stronger management structure in country, such as a Project HOPE Country office presence would have been helpful in avoiding some of the difficulties between the Departmental MOH and CDS (see discussion on Planning). 5. Focused technical assistance in program management, financial planning, human resource development, supervision, information management, and developing training strategies can ensure that relevant project have the appropriate skills to perform their jobs in a high quality fashion and that project staff provide good role models to their partners. 6. Improved budget appropriation process can help project ensure that activities continue to move forward without delays. 7. Project plans need to be realistic and feasible and to this end the choice of interventions needs to be more restrained and strategic and the number of objectives and indicators should be limited. 50 8. Projects need to receive a broad range of backstop support including assistance in such topics as health information system development; M&E tool development, supervision, sustainability, and management. 9. When working in such an isolated location where communication is very difficult, a high priority should be placed on making sure that it’s possible to communicate with the field and between field sites. Specific recommendations for ways the MOH can continue to reinforce community level activities, as well as ways the new projects can incorporate some of the strategies and lessons learned from this project have been mentioned throughout the report. 51 Annex 1 List of Training Team Members • Dr. Joanel Modestin – Project Manager • Nelta Joseph – Hygiene Nurse – CDS/PLAN International - malaria • Prenante Angrand (Nunush) – Nurse, Terrier Rouge • Emmanuel Jean Simon – Statistician (CDS) • Macdaline Dursier – Trou de Nord (CDS now MOH – assistant to Dr. Pierre Noele • Dr. Hervé Longchamp (CDS/project) – Technical Assistant/Field Supervisor • Marise Hyppolite – Head of the TB Program (CDS/PLAN) 52 Annex 2 Total Number & Positions of Persons Interviewed 24 Promotors and Health Agents 8 TBAs 10 Representatives of Community-Based Organizations 11 CBD's Managers 23 Mothers 5 Auxiliaries and Nurses 4 Heads of Health Facilities Project Manager, Dr. Joanel Mondestin Statistician/ M & E Officer, Jean Simon Emmanuel Technical Assistant/Field Supervisor, Dr. Herve Longchamps CDS Executive Director, Dr. Pierre Despagne CDS Technical Director, Dr. Barthelemy Project HOPE Senior Technical Advisor, Dr. Robert Northrup Project HOPE Assistant Regional Director for Africa and Haiti, Sue Adams Project HOPE GIK Humanitarian Aid, Manager of Special Projects, Colin Credle Project HOPE International Operations Accountant, Kelli Morris 53 Annex 3 Evaluation Process 1.- Conduct final KPC 2.- Analyze KPC results 2.- Document review (Annual Reports–Years 1-4, DIP, Project Proposal, Description of Budget Changes–Memo Northrup, January 26 2005, Baseline KPC Report, Others) 3.- Interviews with HOPE staff in Millwood, Virginia 5.- Interviews with USAID 6.- Establish and orient the evaluation team (Mini Team-building, Consensus on Objective of the Evaluation, Project History/Timeline, Discussions on Cross-Cutting Approaches, Interventions) 7.- Identify types of people to interview, locations, and numbers 8.- Design/Pretest questionnaires 9.- Collect data (during 2 days) 10.- Analyze the data 11.- Debriefing/Dissemination of Preliminary Results with MOH Stakeholders and Promoters 12.- Prepare evaluation report 54 Annex 4 Goal – The goal of this project is to reduce infant and child mortality and morbidity and to improve reproductive health in six communes in the North-East Department of Haiti. Objectives and Interventions Child Health – Household Level Objective # 1: Improved preventive actions to maintain child health Objective # 2: Improved home management of common childhood illnesses Objective # 3: Improved care-seeking practices Child Health – Community Level Objective # 4: Increased community participation in child health and disease prevention activities Child Health – Health Facility Level Objective # 5: Improved management of child health and community outreach Child Spacing Objective # 6: Improved knowledge and practices Objective # 7: Increased community participation Objective # 8: Improved access and quality of services HIV/AIDS/STIs Objective # 9: Improved knowledge and practices Objective # 10: Increased community participation Objective # 11: Improved access and quality of services Pregnancy and delivery management Objective # 12: Improved knowledge and practices Objective # 13: Improved access and quality of services 55 Capacity Building – HOPE Objective # 14: HOPE management and technical expertise strengthened. Capacity Building – Community, MSPP, and CDS Objective # 15: Community Level Objective # 16: Health Facility Level Objective # 17: Departmental MSPP Objective # 18: CDS 56 Integrating Child Survival and IMCI Activities into Six Target Communities in the North-East Department of Haiti CS-XVII Cooperative Agreement No. HPF-A-00-01-00031-00 Final KPC Report Project Location: Department of the North-East, Haiti Project Duration: September 28, 2001 – March 31, 2006 Submitted to: U.S. Agency for International Development GH/HIDN/NUT/CSHGP Room 3.7-72B Washington, DC 20523-3700 Attn: Susan Youll Submitted by: Project HOPE – The People-to-People Health Foundation, Inc. Millwood, Virginia 22646 Tel: (540) 837-2100 Fax: (540) 837-1813 March 31, 2006 Prepared by: Dr. Pierre Despagne Bernateau Desmangles Executive Director, CDS M&E Specialist Contact persons: Shaun W. Walsh Bonnie L. Kittle Regional Director, Africa and Haiti Director, Health of Women and Children TABLE OF CONTENTS ACKNOWLEDGEMENTS ....................................................................................1 ACRONYMS.........................................................................................................2 A. BACKGROUND..............................................................................................3 B. OBJECTIVE OF THE KPC SURVEY...............................................................4 C. METHODOLOGY............................................................................................5 Groups targeted by the study ...................................................................................................... 5 Choice and size of the sample ..................................................................................................... 5 Instruments used to collect the data........................................................................................... 5 Profile and training of interviewers and supervisors ................................................................ 5 Data collection ............................................................................................................................... 6 Data processing and analysis...................................................................................................... 6 D. FINAL KPC SURVEY RESULTS....................................................................6 Demographic Characteristics of the Respondents.................................................................... 8 Childhood Immunization............................................................................................................... 8 Exclusive Breastfeeding and Complementary Feeding ............................................................ 9 Nutritional Status of the Children .............................................................................................. 10 Prenatal Consultations ............................................................................................................... 12 Assisted Deliveries...................................................................................................................... 13 Postnatal Care.............................................................................................................................. 14 Child Spacing and Contraception Use ...................................................................................... 15 Childhood Illness........................................................................................................................ 15 STI-HIV/AIDS ................................................................................................................................ 18 Hand Washing.............................................................................................................................. 19 i E. DISCUSSION OF RESULTS..........................................................................19 ANNEX 1 Table of KPC Results.............................................................................................. 21 ANNEX 2 KPC Questionnaire ................................................................................................. 22 ii ACKNOWLEDGEMENTS This report has been written with the cooperation of Mr. Webert José, economist statistician; Mr. Guy Fred Célestin, statistician; and Mr. Beldor Jean-Louis, specialist in information technology. A special thanks to Dr. Pierre Despagne, executive director of the Centres pour le Développement et la Santé (CDS); Dr. Joanel Mondestin, coordinator of CDS/HOPE Health Project in the North-East health department; and Dr. Lionel Barthelemy, CDS technical director, who have trusted the team with the processing and analysis of the data, and drafting the evaluation report of CDS/HOPE Health Project. The team would like to take advantage of this occasion to pay a well deserved tribute to the field personnel for their participation in collecting data for the survey despite their busy schedule. The good collaboration of mothers who voluntarily accepted to answer the interviewers’ questions is also much appreciated. The team’s most sincere thanks also go to those who, in one way or another, have contributed to the implementation and success of this evaluation. May the lessons learned from this endeavor contribute to improving the results and impact of future projects to the benefit of the whole population. Bernateau Desmangles M&E Specialist 1 ACRONYMS BRGNNE CDS Regional Management Office CDS Centres pour le Développement et la Santé CORE Coalition of non-governmental organizations CSHGP USAID Child Survival and Health Grants Program DIP Detailed Implementation Plan FP Family Planning HIV/AIDS Human Immunodeficiency Virus /Acquired Immunodeficiency Syndrome HOPE Project HOPE IMCI Integrated Management of Childhood Illness KPC Knowledge, Practice, Coverage survey LAM Lactational Amenorrhea MOH Ministry of Health MSPP Ministry of Public Health and Population NGO Non-Governmental Organization ORS Oral Rehydration Salts ORT Oral Rehydration Therapy RtH Road to Health card STI Sexually Transmitted Infection TBA Traditional Birth Attendant UCS Community Health Units USAID United States Agency for International Development WFP World Food Program WHO World Health Organization 2 A. BACKGROUND Project HOPE has been working in Haiti since the early 1990s, with the first child survival project beginning in 1994. That project was based in the North Department. The current child survival project was an extension of that initiative and expanded child survival activities into the six districts of the neighboring North-East Department beginning in October 2001. Due to difficulty finding and keeping personnel to manage and implement the project in this remote under-developed part of Haiti, at the suggestion of USAID/Port-au-Prince and with concurrence of CSHGP, Project HOPE (HOPE) sub￾contracted to a local NGO, Centres pour le Développement et la Santé (CDS), and in November 2002, CDS became the prime implementer of the project. Project HOPE provided technical guidance, support and oversight. In late 2004, when HOPE experienced difficulty raising the 50% cash match10, the target area of the project was reduced from six districts to three (Trou du Nord, Terrier Rouge and Caracol). This project has been financed by USAID and Project HOPE. Through this project and through the collaboration with the departmental MOH, a package of services dealing with child survival and reproductive health was made available to children under age 5 and women of reproductive age. This package includes the following services: • Immunization • Nutrition and micronutrients • Promotion of immediate and exclusive breastfeeding • Control of diarrheal disease • Prevention and treatment of acute respiratory infections (ARI) • Pre and postnatal consultations • Safe assisted deliveries • Family Planning • Prevention of STI/HIV/AIDS During the first two quarters of the project, a baseline survey was carried out in the selected communes to determine the level of knowledge, attitudes and coverage of the targeted mothers with respect to the common health problems. The results of this survey were used to elaborate a detailed implementation plan (DIP) and to define the targets for the behavior change strategies. The planned interventions were implemented in coordination with the directors in charge of the Community Health Units (UCS) in Trou du Nord and Terrier Rouge. The interventions targeted individuals, families, and sought to improve the practices of the health workers serving these communities. In December 2005, a final KPC survey was conducted to determine to what extent the efforts employed have led to the anticipated results, in particular the levels of coverage of the basic services and individuals’ care giving and care seeking behaviors. 10 An over- match was subsequently provided in medical supplies and equipment. 3 The KPC survey was carried out by a multi-disciplinary team made up of members of the CDS Regional Management Office (BRGNNE), directors of the UCS in Trou du Nord and Terrier Rouge, the health unit of the North-East Department, and the Health Technical Direction of CDS. An independent consultant was engaged to direct and coordinate the survey. The officials of BRGNNE were in charge of the recruitment of the interviewers, supervisors and guides. The Department MOH officials and the directors of the UCS supervised the interviewers during the data collection process. The CDS technical director was in charge of the development of the study’s terms of reference and recruitment of a consultant for the collection and analysis of the data. The recruitment process, training of interviewers and supervisors lasted eight days. B. OBJECTIVE OF THE KPC SURVEY The objective of the study was to measure the extent to which the project achieved the stated objectives as defined by the following indicators. 1. Percentage of children 0-23 months with a low weight for their age, i.e., equivalent to (-2 SD) compared to the normal; 2. Percentage of children 0-23 months born at least 24 months after previous living child; 3. Percentage of children 0-23 months born with the assistance of a trained health person 4. Percentage of mothers with children 0-23 months who have received at least 2 doses of tetanus toxoid before the birth of their youngest child; 5. Percentage of children 0-5 months exclusively breastfed in the past 24 hours; 6. Percentage of children 6-9 months who were breastfed and received complementary meals during the past 24 hours; 7. Percentage of children 12-23 months fully immunized before their first birthday; 8. Percentage of children 12-23 months vaccinated against measles; 9. Percentage of mothers of children 0-23 months aware of at least two ways to reduce the risk of HIV infection; 10. Percentage of mothers of children 0-23 months who wash their hands with soap/detergent before preparing meals, before feeding their children, after defecating and after attending a child who has defecated; 11. Percentage of mothers of children 0-23 months aware of at least two signs of childhood diseases that identify a need for treatment; 12. Percentage of sick children 0-23 months who have received more liquids and have continued to be fed during an illness in the past two weeks. 4 C. METHODOLOGY Groups targeted by the study Based on the 2003 census, the geographical area to be surveyed was divided into operational units made up of several localities. These operational units were defined by the working areas of individual health agents and health promoters. The survey was carried out in these areas to measure the coverage of the targeted services and behavior change. The universe of this survey was the group of mothers who lived in these operational units and who had at least one child less than 24 months old at the time of the interviewers’ visit. Choice and size of the sample This survey followed the 30-cluster methodology recommended by CORE and WHO. The clusters to be included were chosen systematically from the list of localities issued during the 2003 census. According to the standard cluster survey, each cluster should be made up of seven households with a mother/child care giver with at least one child less that 24 months. To minimize the effect of the cluster sampling method, this number has been raised from 7 to 10 households per cluster. Because the district of Caracol has too few households, its population was added to Terrier Rouge. The survey area thus consisted of two areas: Trou du Nord and Terrier Rouge/Caracol. On the basis of this methodology, a total of 583 mothers were interviewed with a total of 600 children under age two. Instruments used to collect the data The standard KPC 2000+ questionnaire to evaluate USAID-funded Child Survival projects was adapted, taking into account the specific objectives of the project. Broad consensus was obtained on this adaptation before it was utilized for the collection of data in the field. The questionnaire (see Annex 1) contains information related to the demographic characteristics of the surveyed population and their knowledge, comportment and practices regarding their health problems and those of their children. The questions related to malaria were removed because this illness was not considered a project priority. Profile and training of interviewers and supervisors Most of the health agents and promoters selected to be interviewers were men less than 45 years old, having completed grade 4. They could correctly read and write French and Creole. They were recruited and chosen from among the best performing promoters/ health agents of the project and worked in Trou du Nord and Terrier Rouge. The supervisors, on the other hand, were selected from among the nursing personnel of the project. The epidemiologist of the North-East Department was also part of the supervision team. 5 A three-day training was carried out for the survey team. This training covered their role and responsibilities, the questionnaire, interview techniques, technique for the choice of the first household and the subsequent households to visit, the completion of the questionnaire, and tests of coherence and validation of data. The questionnaire was tested at the end of the training and no major problems were encountered. A typical interview lasted approximately twenty (20) minutes. Data collection The data was collected between December 12-19, 2005. These operations took place without major incidents except for a day of rain that caused an increase in the time set for collecting the data. The presence of guides knowing the local geography during the data collection phase greatly facilitated the work, allowing rapid identification of houses and households. Thirty percent (30%) of the completed questionnaires were reviewed by the supervisors to avoid repetition of errors during the following days. Data processing and analysis The data collected and validated by the supervisors and team leaders was tabulated manually, then processed and analyzed using SPSS. The data entry and processing was done by the project statistician and the analysis was done by the consultant. Processing errors detected during the data cleaning were returned to the statistician for necessary corrections. This took several weeks. For the section on nutritional status of infants, the weights recorded and ages of the child were processed using EPI INFO to calculate a Z score for each child. These were then entered into SPSS to calculate the proportions of the sample which were moderately or severely malnourished. D. FINAL KPC SURVEY RESULTS The following table summarizes the results of the KPC survey for the project’s objectives. Table 1 Status of project indicators Indicator Baseline KPC Final % Target % % children 12-23 months completely vaccinated 6.8%11 80.4 80 % children who received a dose of Vitamin A in the last 6 months 19.3 85.2 30 % of mothers with at least 2 TT vaccinations 11 62 70 % of mothers who keep Sel Lavi at home to NA 51.9 30 11 This reflects verbal reports only since according to the the vaccination cards the level of coverage was 0% 6 Indicator Baseline KPC Final % Target % prevent dehydration % of mothers who exclusively breastfed their babies during the first 6 months NA 67 20 Increase in the % of children U2 weighed in previous four months NA 70 50 % increase in mothers who used ORT in last diarrheal episode 56 74 60 % know signs of dehydration 38.8 85 80 % increase of mothers who know how to prepare ORS correctly 26.3 89.3 80 % of mothers who provided the same or more liquids/solids during a disease episode 14.812 44 liquids 39 solids 60/40 % increase of mothers know how to complete prescribed treatment to the child following prescription by a provider NA Not asked % of women who do not desire a pregnancy in the next 2 years are using a modern contraceptive method or LAM increased from 18.7% 55.4 35 What percent of women can cite 2 or more FP methods NA 96 60 % of children 0-23 months born at least 24 months after previous surviving child 61.1 49 Increases by 20 % of women that can name at least 2 ways to protect themselves from HIV/AIDS/STIs 52.9 9413 80 What percent of men/women used a condom in their last sexual intercourse NA 15 What percent of women and men can name at least 2 signs indicative of an STI 35.9 85 30 What percent of mothers cite 3 or more danger signs during pregnancy & 2 or more signs during delivery 6.4 60 % of mothers who receive 3 or more prenatal exams from health professional 45.6* 41 60 % of mothers who can cite the signs of illness that require care seeking 98 % of home deliveries use clean delivery kit 41 73.7 85 % of mothers who receive Vitamin A within 7 days post-natal 5.9 92.2 60 % of births who receive home visits by promoter within 7 days 7.7 87.5 75 % of mothers with children 0-23 months who wash their hands before eating, before feeding a child, after defecating and after attending a child who has defecated14. 6.4 56 12 for the question those who provided both fluids and solids 13 Can name one preventive method 14 Actual question was related to knowledge of four time when to wash hands. 7 Demographic Characteristics of the Respondents The average age of the respondents is 28 years old and 74% have only one child. Moreover, 49 % of those who had more than one child have observed an interval of at least twenty-four (24) months between the two births. Table 2 Age of the survey respondents Indicator N Minimum Age Maximum Age Mean Age Age of the Mother 583 14 45 28 Table 3 shows the breakdown of the 600 children by sex and age. They were distributed relatively evenly across the age range studied with the exception of those born prior to most project activities (ages 18-23 months), and there was a slight preponderance of females. Table 3 Analysis of the children for sex and age Total 0-5 months 6-11 months 12-17 months 18-23 months n % n % n % n % n % Masculine 286 48% 68 45% 92 51% 70 45% 56 48% Feminine 314 52% 82 55% 88 49% 84 55% 60 52% TOTAL 600 100% 150 100% 180 100% 154 100% 116 100% Childhood Immunization In Haiti, the Ministry of Health considers a child “completely vaccinated” when they have received the following antigens by their first birthday: • DTP -- three doses at an interval of at least one month and no more than 2 months • Polio -- three doses at an interval of at least one month and no more than two months • BCG -- a single dose at the time of birth or within the first two months • Measles -- one dose after the age of nine months Vaccination coverage rates are calculated by referring to the child’s Road to Health Card which includes the immunization record. Because the project has done so much work in the area organizing growth monitoring posts, fully 98% of the children ages 0–23 months possessed Road to Health cards at the time of the survey. The final KPC survey showed a dramatic increase in the percentage of children who had been completely immunized. Compared to the baseline study of 2002 which showed coverage of 0%, proven by card 8 and only 7% according to the mother’s declaration15, the coverage in 2005 was 80.4%; a remarkable increase observed in all locations. Table 4 Completion of immunization among children 12-23 months Complete Childhood Vaccination KPC 2002 KPC 2005 According to the Card only 0%* 80.4% Card and mother’s declaration 6.8% * Road to Health cards were not in use in the department at the outset of the project, therefore, while vaccination coverage rates were in fact quite low, due to the absence of RtH cards the rate could not be ascertained. Details of the coverage by individual antigen in the same age group are provided in Table 5. Table 5 Coverage by individual antigens for children ages 12-23 months Antigen Received antigen Total in age group Result (%) all required vaccinations (BCG, polio3, DTP3, measles) 226 281 80.4 measles vaccine 240 281 85.4 BCG vaccine 254 281 90.4 3 doses of DTP 252 281 89.7 3 doses of polio 249 281 88.6 As expected, the coverage levels achieved for individual antigens were even higher than the levels for complete coverage. Even measles immunization, typically the most difficult to implement with high levels of drop-outs during the 9-11 month period, was higher, at 85% coverage. Exclusive Breastfeeding and Complementary Feeding Breastfeeding is the norm in all three communes. More than 99% of mothers breastfed the child in question at some time, and fully 84% of were breastfeeding the child at the time of the survey. Table 6 shows the results of the desired breastfeeding behaviors. • Exclusive breastfeeding (no food or liquids other than breast milk in the past 24 hours)? • Complementary feeding of children in addition to breast milk? 15 This very low coverage is due in large part to the fact that this was a newly formed MOH department and health services had just begun to be provided. Further more, the health information system was not well established at that time. 9 Exclusive breastfeeding practice was evaluated for children from 0 to 6 months and the survey showed that 67% of mothers from this group of children had exclusively breastfed their children, as demonstrated by the proxy indicator of 24 hour recall. Figure 1 shows an increase in this behavior from the time of the baseline survey when only 44% of mothers reported exclusive breastfeeding. Initiation of complementary feeding was assessed in the next older age group, children 7 to 11 months of age. A high level of mothers of children of this age – 13% -- had failed to initiate complementary feeding, and were still giving only breast milk. Table 6 Breastfeeding and complementary feeding practices Breastfeeding Practices Results (%) Percentage of mothers with children from 0-5 months old who had exclusively breastfed during the 24 hours before the survey (N=150) 67 % Percentage of mothers of children from 6-11 months old who gave food in addition to breastfeeding in the previous 24 hours (N=112) 87 % Figure 1 Exclusive breastfeeding in previous 24 hours. % o f c hi l dr e n 0 - 5 mo nt hs e x c l us i v e l y br e a s t f e d during the previous 24 hours 44.3 67.8 0 10 20 30 40 50 60 70 80 Basel i ne sur vey Fi nal eval uati on sur vey Sur vey Nutritional Status of the Children The nutritional status of the targeted children was evaluated on the basis of weight for age. Unfortunately it was not possible to provide a scale to each interviewer. Accordingly the children’s weights and ages were determined on the basis of the Road to Health cards, using the most recent weight from the past four months. These weights were converted into Z scores based on standard deviations of the data by the EPI INFO 10 nutrition module, which were in turn analyzed for frequency to determine the prevalence of moderate and severe malnutrition. As shown in Table 7, 98% of the children had Road to Health cards. Table 7 Percentage of children ages 0–23 months having a Road to Health card n % Children ages 0-23 months with RtH card 592 98% Children age 0-23 months w/out RtH card 12 2% Table 8 shows the frequency of weighing by age group over the four months prior to the survey, as determined by examination of the Road to Health card. Children without Road to Health cards were considered to not have been weighed at all. Table 8 Coverage and frequency of weighing children 0-23 months of age in previous four months Number of times weighed in past 4 months Age 0-5 months Age 6-11 months Age 12-17 months Age 18-23 months Not weighed 49 33% 55 31% 37 24% 39 34% Once 23 15% 23 13% 17 11% 18 16% Twice 25 17% 23 13% 20 13% 18 16% Three times 29 19% 27 15% 27 18% 16 14% Four times 24 16% 52 29% 53 34% 25 22% Children weighed at least once 101 67% 125 69% 117 76% 77 66% Approximately two-thirds of children in the 0–11 months and 18–23 month age ranged had been weighed at least once during the prior quarter; while three-fourths of the children ages 12–17 months had been weighed at least once. Table 9 indicates the degree of malnutrition identified among the children weighed in the prior quarter. These figures may reflect greater malnutrition than normal because the World Food Program (WFP) is operational in the area and gives food rations to families with malnourished children. During the final evaluation it was revealed that some mothers underfed their children before the growth monitoring session to increase their chances of qualifying for food rations through WFP. Furthermore, once the food rations are distributed there is no control over which family members are fed and how much. Table 9 Percentage of children 6-23 months old with moderate and severe malnutrition Degree of Malnutrition 0-23 months N=136 6-23 months N=118 12-23 months N=69 n % n % n % % children moderately malnourished (<-2 SD 29 21.3 29 24.6 18 26.1 11 et >= -3 SD) % children severely malnourished(<-3 SD) 18 13.2 17 14.4 9 13.0 Prenatal Consultations Promoting behaviors to ensure health pregnancies and deliveries was a priority for the project, with indicators in the project plan aimed to improve knowledge of the symptoms of danger signs during pregnancy, adequate prenatal care, attendance at the delivery by a trained birth attendant, and a timely post-natal consultation. Regarding improvements in maternal knowledge about danger signs during pregnancy, the final KPC survey examined which complications were best known by the mothers. Table 10 Complications during pregnancy TYPES OF COMPLICATION n % No complication known 3 1% Fever 167 28% Bleeding 318 53% Swelling of the feet 95 16% Respiratory difficulties 17 3% Total 600 As shown in Table 10, all but three women were able to mention a danger sign and more than half identified bleeding. The project sought to increase the percentage of pregnant women who had at least two TT vaccinations. Since these are provided during pre-natal consultations, the rate of care seeking among pregnant woman was also emphasized. In the project area, prenatal consultations were available in health centers and in doctors/nurses offices. The final KPC survey showed that 92% of all women had had at least one prenatal consultation, a marked improvement over the 2002 results which showed that only 73% had received any prenatal care. The use of prenatal consultations offered by a qualified personnel has been relatively high in the commune of Caracol (82%) and very high in the communes of Trou du Nord (91%) and Terrier Rouge (96 %). As Table 11 shows, 45% of all women had received two or more prenatal consultations. Table 11 Number of prenatal visits during most recent pregnancy Card as evidence Card and mother's statement Number of prenatal consultations n % n % 0 118 31% 149 25% 12 1 89 23% 118 20% 2 130 34% 185 31% 3 or more 43 11% 146 24% Totals 380 100% 598 100% The proportion showing the recommended three or more visits, however, remained low, only 11% based on inspection of pregnancy cards, or 24% from card and mother’s report. In comparison to the 2002 baseline survey, this was a serious decrease, going from more than 50% during the baseline survey to 11% or 24% according to the 2005 survey. Given that health services had not yet been fully developed in the department in 2002 and the health information system at the time was quite weak, the explanation for these results is not clear. According to Table 12, about 48% of pregnant women did not receive the requisite two doses of TT. Table 12 Doses of tetanus toxoid vaccine received by pregnant women during their previous pregnancy according to their pregnancy health card or their verbal reports Doses of TT vaccine received card reported only n % n % 1 dose 82 22% 113 19% 2 doses 234 62% 333 56% 3 doses 0 0% 94 16% 4 doses 0 0% 21 4% No doses 64 17% 39 7% Total 380 600 Assisted Deliveries The goal of the project was to increase the number of deliveries which were attended by a “trained person”. For the purposes of this survey, the definition of “trained person” included professionally trained medical people, such as doctors and nurses, and community-level birth attendants who were trained by a MOH-approved program (72 hours of formal training over a six-month period). Although the project design called for training traditional birth attendants (TBAs), because the MOH had already planned to train TBAs and had received support for this from UNICEF and PLAN/Haiti, it was ultimately deemed unnecessary. Because these courses hadn’t yet been initiated in 2002, the baseline survey found that only 13% of deliveries had been attended by a trained person. In contrast, the final KPC survey shows that 95% of the deliveries had been attended by a trained person; with fully 74% of all deliveries being assisted by trained TBAs. 13 Table 13 Percentage of deliveries assisted by trained personnel Birth Attendant n % Doctor 76 12.7 Nurse 43 7.2 Trained birth attendant 442 73.7 Health agent 7 1.2 Family member 26 4.3 Not attended 6 1 Total 600 100% Postnatal Care At the time of the baseline data survey, only 53% of respondents indicated that they had received a postnatal consultation; whereas during the final KPC survey, 87.5% of these new mothers reported having had a postnatal visit at home (Table 14). These visits were done by health agents and the project’s promoters, and in 96% of the cases they took place during the first week following the delivery (Table 16). Compared to the baseline survey of 2002, a great improvement was observed in the coverage of postnatal services monitoring provided to new mothers. Table 14 Percentage of new mothers who received a postnatal visit (n = 600) Indicator Baseline Survey Results (2002) Final Survey Results (2005) % of respondents who received a postnatal consultation 53%* 87.5% * primarily by TBAs, not health agents or promoters Table 15 Timeliness of postnatal home visit (n = 525) Time period of postnatal visit to new Result mother by health agent or promoter n % 1 to 7 days following delivery 504 96% 8 to 14 days following delivery 14 2.7% 15 to 21 days following delivery 4 0.8% More than one month after delivery 3 0.6% It is not only important that new mothers receive a postnatal consultation, but the timeliness of the visit is also crucial to life saving and health promotion. Among other things, deaths due to post partum hemorrhage and sepsis can be averted through timely postnatal consultations. Table 15 shows a remarkable response time with regard to 14 postnatal visits. Child Spacing and Contraception Use Thirty-seven (37) of the 600 survey respondents, about 6%, said they were pregnant at the time of the survey (Table 16). This result indicates a decrease in the fertility rate from the time of the baseline survey, which was assessed at 8%. The project’s active promotion of family planning and its efforts to improve access to family planning counseling and contraceptives in all the targeted districts strongly influenced these results. The contraceptive prevalence, evaluated at 39% during the 2002 survey, increased to 55% in December 2005. Table 16 Pregnancy prevalence and contraceptive Type of prevalence (pregnancy, CYP) Final KPC Results (%) Fertility Rate (N=600) 6.5 Contraceptive use (N=561) 55.4 As shown in Table 17 the preferred contraceptive method is the injectable form. Table 17 Use of family planning methods (n = 306) Type Of Family Planning method Use rates in % Injectable contraceptive 54.6 % Pills 11.6 % Condom 10.9 % Mama 8.6 % Other natural methods 7.3% Abstinence 5.0 % Surgical sterilization 1.3% Norplant 0.7% Childhood Illness The results of the evaluation of mothers’ knowledge regarding the signs of childhood illness are presented in Tables 18 and 19. Sixty-four percent (64%) of the mothers identified four or more danger signs, and 36% could identify nine or more signs. Table 18 Ability of mothers to identify multiple signs of illness which indicate the need to seek medical attention rapidly (n = 600) Knowledge of Signs of Illness related to Care Seeking N % Mother cannot cite any danger signs of childhood illness 10 1.7% 15 Mother cites only 1 sign 0 0% Mother can cite 2 signs 0 0% Mother can cite 3 signs or more 590 98.3% Table 19 indicates that weight loss was the sign most frequently stated by the mothers, followed by vomiting, apathy and fever. Table 19 Danger signs of childhood diseases most frequently stated by survey respondents SIGN OF ILLNESS % identifying that sign Weight loss 26.3 % Vomiting 17.5 % Apathy 12.7 % Fever 9.2 % Does not drink 7.7 % Dyspnea 5.7 % Sixty-four percent (64%) of the respondents reported having seen one or more signs of illness in their child under age two during the two weeks prior to the survey. This is a high rate of morbidity, which would suggest an incidence of as many as 16 episodes of illness (or a fewer number of more chronic conditions) during a year. While such morbidity indicators will vary dramatically by season, this is still a very high incidence of illness relative to data from other developing countries. It is similar to the morbidity prevalence figure from the baseline survey of 66% of children 0-23 months old, suggesting little change in the factors which lead to illness. The specific illnesses reported are show in Table 20. Diarrhea continues to be the most common illness experienced among young children. Table 20 Illnesses experienced in the preceding two weeks by children of respondents (n = 383) Illness experienced by children in the surveyed families n % Diarrhea 243 63% Fever 61 16% Cough 34 9% Skin disease 21 5% The project’s behavior change strategy emphasized actions which families could take at home to manage diarrhea. Table 21 indicates the responses made by the respondent mothers. Table 21 Maternal response to home based care of diarrhea occurring in the previous two weeks (n = 243) 16 Mother's actions in response to episode of diarrhea in her child n % Commercial oral rehydration solution (Sel Lavi) 181 74% Homemade ORS 26 11% Other household remedies 13 5% Medicines prescribed by doctor 25 10% Other remedies 3 1% Gave nothing 1 0% No data recorded 2 1% More than three-fourths of the mothers, 85%, gave oral rehydration, either ORS prepared from a sachet of salts (Sel Lavi) or a homemade solution of similar composition. Clearly the project’s efforts to promote ORS use have been effective. The project also promoted the general administration of additional fluids and continuation of feeding during illness, particularly diarrhea. This question was characterized by a large number of no responses, for no apparent reason. Of those replying, 44% increased the amount of liquids given to a sick child, and 39% gave more food than usual. Assessing the combination of food and liquid, 56% of those responding indicated they had given the same or more than usual amounts of both food and fluids. Table 22 Home-based care of a sick child Mother’s behavior n % of responses % of all mothers Gave less or nothing to drink 153 40% 26% Gave same as usual to drink 62 16% 10% Gave more to drink 167 44% 28% No response recorded 218 -- 36% Total 600 100%* 100% Gave less or nothing to eat 180 47% 30% Gave same as usual to eat 53 14% 9% Gave more to eat 150 39% 25% No response recorded 217 -- 36% Total 600 100%** 100% Gave same or more fluids and food 181 56% 30% Gave less fluids and/or liquids 143 44% 24% No response to both questions 276 -- 46% Total 600 100%*** 100% * n = 382 ** n = 383 *** n = 324 17 STI-HIV/AIDS The knowledge of the mothers regarding methods available to prevent HIV infection was assessed by the KPC survey. The results overall are presented in Table 23. They indicate that 95% of the mothers were acquainted with at least one method of preventing HIV infection. Table 23 Knowledge of methods to prevent HIV/AIDS among mothers ( n = 600) Knowledge of HIV preventive methods n % Knows at least one preventive method 568 94.7% Does not know a preventive method 32 5.3% Table 24, below, details the preventive methods most familiar to the mothers. Table 24 Methods for preventing HIV/AIDS mentioned by mothers Methods for prevention of HIV n % Use of condom 281 46.8% Abstinence 144 24.0% Fidelity 94 15.7% Avoid contaminated blood transfusions 22 3.7% Avoid unprotected sexual contact with prostitutes 12 2.0% Avoid the use of used syringes 9 1.5% Other methods 4 1.1% Nothing can be done to prevent HIV 4 1.1% Do not know any ways to prevent HIV 30 5.1% Total 600 100% Regarding STIs (other than HIV/AIDS), 95% of the mothers had heard of other sexually transmitted infections (STIs), as shown in Table 25. Most of them said these infections are manifested primarily by pelvic pains and vaginal discharge. Very few of the mothers mentioned the redness of the vulvar mucosa, burning during urination and genital ulcer as signs of sexually transmitted infections. It should be noted that 7% of the mothers questioned were not able to mention any signs of STI manifestation. Based on their limited knowledge of symptoms of STIs, about 33% of the respondents confirmed that they had had at least one episode of a STI in the 12 months preceding the survey. Table 25 Knowledge of the signs of a sexually transmissible infection as they occur in women with such infection (n = 600) Signs of STI mentioned by mothers Frequency in % Does not know signs of STI 7.4 % 18 Pelvic pain 36.8 % Vaginal discharge 38.4 % Fetid vaginal discharge 10.6 % Pain during urination 4.8 % Redness of the vulvar mucosa 0.2 % Chancroid 0.7 % Hand Washing Washing of hands, an important measure of hygiene for the prevention of transmission of diarrhea, was evaluated during the survey. The objective of the intervention was to promote hand washing such that mothers washed their hands, with soap, on four occasions: before preparing food, before eating or feeding a child, after using the toilet, after cleaning a child who has defecated or urinated. The data obtained is presented in Table 26. Table 26 presents the frequency of women responding that they had washed their hands on one, two, three, four, or all five of the occasions, differentiating whether they used soap or not. Table 26 Percentage of mothers who washed their hands on 1, 2, 3, 4, or 5 occasions when it is appropriate # of occasions utilized to wash hands wash hands without specifying use of soap or not wash hands with soap n % n % 1 372 62% 33 6% 2 138 23% 305 51% 3 81 14% 145 24% 4 4 1% 100 17% 5 5 1% 13 2% 6 4 1% Total 600 100% 600 100% E. DISCUSSION OF RESULTS The comparison between the results of the 2002 and 2005 KPC surveys shows that among the 23 indicators, 16 meet or surpass the target level of achievement set at the outset of the project. Noteworthy among the accomplishments are: childhood vaccination coverage, which climbed from less than 5% to over 80%; contraception use, which increased from 18.7% to 55.45 (exceeding the target by 15 percentage points); assisted deliveries (by a trained person) which increased from 41% to 74%; the provision of postnatal consultations within 7 days of the delivery, which increased from 7.7% to 19 87.5%. These achievements are in part due to significantly improved access to health care services and the training and work of the community-based promoters (community health workers). While most of the project objectives were met, the survey also reveals the interventions that felt somewhat short of the stated levels of achievement. For example, the project’s efforts to make sure that each pregnant woman had at least three prenatal consultations did not succeed. Furthermore, though access to condoms has improved, it is not clear if condom use has likewise increased. And finally, hand washing practices did not reach acceptable levels. The results of this KPC compare very favorably with the some national statistics. For example the Demographic Health Survey (DHS) for Haiti completed in 2000 showed that vaccination coverage nationally is only 33.5%, while the KPC survey showed 80% coverage. Used of ORS during the last diarrheal episode in the project area was reported at 74% whereas the DHS shows only 40.7% ORS use. In the project area 55.4% of women report using a contraceptive whereas nationally contraceptive use in 2000 was only at 22.8%. Project HOPE is very proud of the achievements of this project and looks forward to sharing the results of this KPC survey and the final evaluation with the larger child survival community. As has been done in the past, Project HOPE plans to present the results during an internet vClass session to which all interested parties will be invited to attend. 20 ANNEX 1 Table of KPC Results INDICATORS Numerator Denominator Results (%) % of children 12-23 months who have received all required vaccinations (polio3, DTP3, measles) 225 281 80% % of children 12-23 months who have received measles vaccine 239 281 85% % of children 12-23 months who have received BCG vaccine 262 281 93% % of children 12-23 months who have received 3 doses of DTP 255 281 91% % of children 12-23 months who have received 3 doses of polio 252 281 90% % of mothers with children 0-23 months who received 3 or more prenatal care exams, card confirmed 43 371 12% % of mothers with children 0-23 months who received at least two doses of TT during pregnancy, card confirmed 230 371 62% % of mothers with children 0-23 months who received 3 or more prenatal care exams during their last pregnancy (before birth of the last youngest child), mothers' own answers 261 544 48% % of mothers with children 0-23 months who received at least 2 doses of TT during their last pregnancy (before birth of the last youngest child), mothers' own answers 428 528 81% % of mothers with children 0-23 months who report giving their child increased fluids during child's last illness episode in the past two weeks 165 340 49% % of mothers with children 0-23 months who report giving their child continued feeding during child's last illness episode in the past two weeks 53 298 18% % of children 0-5 months fed only breast milk in past 24 hours 101 150 67% % of mothers with children 0-23 months who wash their hands before eating, before feeding child, after defecating, after attending a child who has defecated 579 584 99% % of children 0-23 months born at least 24 months after previous surviving child (we consider only mothers with two children or more) 61 125 49% 21 ANNEX 2 KPC Questionnaire PROJET MSPP / HOPE / CDS Keksionè ankèt KPC Manman timoun ki gen plis pase 24 mwa Kesyonè No: Verifye pa : …………………………… Date de l’ entrevue ____/ _____/ _____ Departement ________________________ Commune ____________________________ Zone ______ Secteur ______ Localité ________________________ No maison : _____________ Nom de l’enquêteur ______________________________________ Nom du Superviseur ____________________________________ Bonjou. Mwen rele ……………………………………………… . Mwen nan zòn nan pou n poze n kèk keksyon sou kijan zafè sante a ap mache nan zòn nan depi Pwojè Ministè ansanm ak HOPE ak CDS te tabli bò isit. Ti pale sa-a m pral fè avè w la ap rete sekrè. Pèsonn lòt moun pap konnen sa nou te pale a. Sepoutèt sa mwen ta renmen ou reponn keksyon m pral poze w yo san pwoblèm epi tou fòk ou di m sa ki verite.. (Ki jan w rele ?) Nom de la mère ____________________________________ ( Ki laj ou ?) Age de la mère _________ (Ki jan w rele ?) Nom de l’enquêté ___________________________________ Sexe ___________ (Ki laj ou ?) Age _______ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@ 1.- Survie de l’enfant (Peze / Vaksinasyon / Vitamin A) SE 1.1. 1.11.- Konbyen ti moun k ap viv nan kay la ki gen pipiti pase 5 an ? 0 _____ 1 _____ 2 _____ 3 _____ 4 _____ 5 _____ 1.1.2.- Konbyen nan timoun sa yo ki se pitit ki soti nan zantray ou ? 0 _____ 1 ______ 2 _____ 3 ______ 4 _____ 5 ____ 22 1.1.3.-(Mande manman an) : Eske ti moun yo gen Kat ? (Mande l pou l ka montre w « Kat Chemen la sante » timoun nan. Pran li, epi enskri non timoun sa yo ; seks yo ; laj yo dat yo te fèt epi konbyen yo peze (Fòk ou mache ak yon balans) Si timoun nan pa gen kat, mande manman « Ki jan timoun yo rele, ki laj yo e ki dat ak ki lane yo fèt? (Enskri non timoun say o nan tablo ki anba a. Kòmanse avek sa ki pipiti a, dènye timoun nan) Gason (G) ou Ti Fi (F) Laj (mwa) Dat li fèt Mwa li fèt Lane li fèt Konbyen li pezé (an kilo) Apre sa, tout lòt keksyon yo konsènan sèlman timoun ki gen pipiti pase 2 zan 23 SE 1.2 (Enskri sou tablo anba a 1.2.1..- non timoun yo ki pitit madan m nan e ki gen pipiti pase 2 zan , 1.2.2.- dat yo te pran vaksen : • Pou BCG : Eksplike l ke se vaksen ki kite mak nan bra timoun nan • Pou Polio : Ve ti dlo a yo lage nan bouch Timoun nan • Pou DTP : di l se piki a yo bay Timoun nan nan fès la e ke yo bay menm lè avek vaksen nan bouch lan • Pou Woujòl: dil se vaksen timoun nan pran lè li gen 9 mwa a 1.2.3.- dat ti moun nan te ale peze (depi nan mwa Daou ki sot pase la a), 1.2.4.- dat yo te pran Vitamin A (depi nan mwa Jen ki sot pase la a) Timoun # 1 Timoun # 2 Timoun # 3 DTP 1 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane DTP 2 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane DTP 3 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Polio 0 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Poilio 1 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Polio 2 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Polio 3 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane BCG Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Roujòl Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Peze 1 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Peze 2 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Peze 3 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Peze 4 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Vit A 1 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Vit A 2 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane Vit A 2 Dat Mwa Lane Dat Mwa Lane Dat Mwa Lane 24 2.- Grossesse / Accouchement GR 2.1 2.1.1.- Madanm, eske w gen Kat oubyen Kanè pou fanm ansent ? (Si repons lan se WI, mande l pou l montre w li. Epi enskri : • dat li te konsilte • dat li te pran Vaksen) WI _______ NON ____ Dat Konsilte VAT Dat Konsilte VAT 2.2.1.- Konnyè la, m pra l poze w kèk keksyon sou kòman bagay yo te ye pandan w te ansent dènye timoun nan ……….. (Di non timoun nan) epi tou sou kijan akouchman an te pase. Eske w te swiv (konsilte) dènye fwa ou te ansent lan (pandan dènye gwosès la) ? WI _______ NON _______ 2.2.2.- Ki kote ou te swiv lan ? (Ki kote ou t al konsilte a) ? Nan Sant sante _____ /Kay Doktè ou kay Miss _____ Nan Pos (ou Pwen) _______ /Kay Ajan de sante _____ GR 2.2 (Si manman an pa gen kat, mande l) : 2.2.3.- Konbyen fwa ou te konsilte kote sa a ? 1 fwa _____ /2 fwa _____ /3 fwa ____ /4 fwa ___ (Si manman an pa gen Kat mande l) 2.3.1.- Avan ou te akouche, eske w te resevwa yon vaksen nan bra ? GR WI ________ NON ______ 2.3 (Si l pa gen kat, mande l) 2.3.2.- Konbyen fwa ? 1 fwa _____ /2 fwa _____ /3 fwa _____ /4 fwa ____ GR 2.4 2.4.1.- Dapre ou, ki pwoblèm ki ka fè yon fanm ansent kouri ale nan Sant sante osnon kay doktè. Di n ki siy ki pandan gwosès lan ki ka fè w kwè gen danje ? Fyèv ____ /Bay san ____ /Pye anfle ____ /Souf anlè __ GR 2.5 2.5.1.- Ki kote ou te akouche …………… (Di non timoun nan) ? Lakay mwen ________ /Kay matwòn _______ Kay yon Miss ________ /Nan Sant sante _____ GR 2.6 2.6.1.- Lè w te gen tranche, kilès moun ki te akouche w ? Doktè _______ Mis ______ Matwòn ak bwat ________ Ajan de sante _______ Yon moun nan fanmy lan ______ Pèsonn ______ GR 2.7 2.7.1.- Eske w te gen yon pake tou pare pou akouchman an ? (Mande l ki sa ki te nan pake sa) WI ___________ NON ______ GR 2.8 2.8.1.- Lè w te ansent lan, eske w te gen yon moun ki te di w pou w al fè tès pou maladi SIDA ? WI ___________ NON __________ 25 2.9.1.- Lè w te fin akouche……..(Di non Timoun nan ), eske te gen yon ajan de sante ki te vin wè w lakay ou ? WI ____________ NON _________ GR 2.9 2.9.2.- Konbyen jou apre akouchman ? 1 a 7 jou apre _________ /8 a 14 jou apre ________ 15 a 1 mwa apre _______ /Plis pase yon mwa ______ 2.10.1.- Eske Ajan an (oubyen Mis nan Sant lan) te ba w Vitamin A? (Montre madanm nan yon kapsil Vitamin A) GR WI ____________ NON ____________ 2.10 2.10.2.- E kilè li te ba w Vitamin A a ? Konbyen mwa (oubyen konbyen fwa) apre w te fin akouche li te ba w li ? Dat Dat Dat Dat Dat 3.- Planification familiale 3.1.1.- Eske w ansent konnyè la ? WI ____________ NON ____________ (Si repons lan se NON, mande l) : 3.1.2.- Eske w vle gen yon timoun ? WI ____________ NON ____________ (Si repons lan se WI, mande l ) : 3.1.3.- Se pou kilè ou ta renmen genyen yon lot timoun ? Nan 2 zan ? ____________ Plis pase 2 zan ? _________ (Si repons keksyon 3.1.2 an se NON, oubyen si repons li pou keksyon 3.1.3 se : « plis pase 2 zan » , mande l): 3.1.4.- Eske ou menm oubyen mari w ap swiv yon metòd pou pa fe pitit nan moman n ap pale la a ? WI ____________ NON ____________ PF 3.1 (Si repons lan se WI, mande l) : 3.1.5.- Kilès metòd n ap itilize ? Abstinans(pa fè bagay) ____ Kapòt _____ Grenn ____ Piki twa mwa_____ Metòd natirèl ______ Krèm ______ Nòplan _____ Esterilè ____ Ligati _____ Ovil ______ Vasektomi ______ Alètman (Mama) ______ Gason an retire pijon l avan l voye andedan fi a _____ Lòt metòd ___________________________________ 3.2.1.- Eske w ka di n konbyen metòd pou yon fanm pa gen plis pitit ou konnen ? WI _____________ NON ____________ 26 PF 3.2 (Si repons lan se WI, mande l) : 3.2.2.- Kilès metòd ou konnen ? Abstinans(pa fè bagay) ____ Kapòt _____ Grenn ____ Piki twa mwa_____ Metòd natirèl ______ Krèm ______ Nòplan _____ Esterilè ____ Ligati _____ Ovil ______ Vasektomi _____ Alètman (Mama) _____ Gason an retire pijon l avan l voye andedan fi a _____ Lòt metòd __________________________________ 4.- Nutrition / Allaitement maternel NU 4.1 4.1.1.- Eske w te bay…….. (Di Non timoun nan) tete? WI _____________ NON ____________ NU 4.2 4.1.2.- Kilè, apre w te fin akouche ……… (Di Non timoun nan) ou te kòmanse ba l tete ? Menm lè apre _____ Pita nan nenm jounen an ______ Yon (1) jou apre ________ De (2) jou apre ________ Plis pase 2 jou apre ______ 4.1.3.- Eske ………… (Di Non timoun nan) nan tete toujou ? WI _____________ NON ____________ NU 4.3 4.1.3.- M ta renmen mande w ki sa ou te ba ………. (Di non timoun nan) manje ak bwè ayè . Ni pandan lajounen ni pandan lannwit ? (Poze l keksyon pou chak sa ki nan tablo a) MANJE AK LIKID WI NON Konbyen fwa ? Tete (let manman l) Dlo ? Lot likid ? (kola, kafe …) Pire, Sòs, Bouyon, Pannad, Lasoup, labouyi? Ji Manje lakay _____________ 5.- Maladie de l’enfance ME 5.1 (Kòmanse keksyon sa a konsa) : 5.1.1.- Gen kèk fwa timoun yo konn malad e lè sa a, li trèzenpotan pou l al pran la swenyay. Eske w ka di n ki siy, ki maladi ki ka parèt sou timoun nan e ki montre w li bezwen trètman ? (Si manman bay yonn ou 2 sèlman, kontinye mande l pou l ka site plis siy. Mande l konsa :) Kilès ankò? M pa konnen ___________ /Li pa manje __________ Li pa bwè ___________ /Li tris, li pa jwe _____ Li respire fò oubyen li gen difikilte pou l respire ______ Li gen feblès e li gen difikilte pou l leve ________ Li gen bouton, l ap grate _____/ Li gen Kriz ________ L ap vomi __________ / Li gen vant fè mal ________ Li gen la fyèv _________ / L ap touse ___________ Maladi cho fwet _________ Dyare______ San nan poupou ______________ 27 (Si repons li bay yo montre manman an konnen siy yo, di l) : 5.2.1.- Mwen byen kontan ou ban m siy (oubyen maladi) sa yo, men, eske …. (di Non timoun nan) te genyen yonu ou plizè nan siy sa yo (oubyen maladi sa yo) nan 2 semenn ki sot pase la yo ? WI _____________ NON _____________ ME 5.2 5.2.3.- E kilès nan yo ? Li pa manje __________ Li pa bwè ___________ /Li tris, li pa jwe _____ Dyare______ Li respire fò oubyen li gen difikilte pou l respire ______ Li gen feblès e li gen difikilte pou l leve ________ Li gen bouton, l ap grate _____ / Li gen Kriz ________ L ap vomi __________ / Li gen vant fè mal ________ Li gen la fyèv _________ / L ap touse ___________ Maladi cho fwèt _________ San nan poupou ______________ 5.3.1.- Lè ……….. (Di non timoun nan) te malad, eske w te ba li bwè ? WI _____________ NON ____________ ME 5.3 5.3.2.- Ki kantite ? Konbyen fwa : Mwens ke abitid ?, Menm jan ke abitid ? Oubyen plis ke abitid ? Mwens ke abitid ______ Menm jan ke abitid _____ Plis ke abitid _____ 5.4.1.- Lè…………. (Di non timoun nan) te malad, eske w te ba li manje ? WI _____________ NON ____________ ME 5.4 5.4.2.- Ki kantite? Konbyen fwa ? : Mwens ke abitid ?, Menm jan ke abitid ? Oubyen plis ke abitid ? Mwens ke abitid ______ Menm jan ke abitid _____ Plis ke abitid _____ 5.5.1.- Lè ………….. (Di non timoun nan) te malad, eske w te ba li tete? WI _____________ NON ____________ ME 5.5 5.5.2.- Ki kantite ? Konbyen fwa ? : Mwens ke abitid ?, Menm jan ke abitid ? Oubyen plis ke abitid ? Mwens fwa ke abitid _____Menm jan ke abitid _____ Plis fwa ke abitid _____ 5.6.1.- Lè ………….. (Di non timoun nan) te malad, e li te kòmanse refè, eske w te ba li manje gwo manje? WI _____________ NON ____________ ME 5.6 5.6.2.- Lè ………… (Di non timoun nan) te malad e li te kòmanse refè, konbyen fwa pa jou ou te ba li manje ? Mwens fwa ke abitid? Menm jan ke abitid? Plis fwa ke abitid? Mwens fwa ke abitid ______ Menm jan ke abitid _____ Plis fwa ke abitid _____ ME 5.7.1.- Tou ta lè a, ou te di m ke ……(Di non timoun nan) te gen dyare pandan li te malad semenn pase yo. Eske sa ou te di a, se li menm ou toujou kontinye kenbe ? WI _____________ NON ____________ 28 5.7 (Si repons lan se WI, mande manman an) : 5.7.2.- Lè …………… (di non timoun nan) te gen dyare a ki sa ou te ba li ? Sachè sewòm Sel Lavi ____ / Sewòm oral oral lakay __ Remèd lakay ___________ Remèd doktè ___ Lòt bagay (di ki sa) ____________________________ ME 5.8 5.8.1.-Eske w gen yon sachè sewòm oral lakay ou konnyè la ? (Mande l pou l ka montre w Sachè Sewòm Oral la) WI _____________ NON ____________ 5.8.2.- Eske w konnen ki jan pou w prepare sewòm oral? (Mande l pou l di w oubyen pou l montre w kòman) WI ___________ NON ____________ ME 5.9 5.9.1.- Madanm, kèk fwa yon ti moun ki gen dyare rive pèdi anpil dlo nan fè tata plize fwa nan jounen an. Li ka menm rive dezidrate. Ki jan ou ka rekonèt lè yon timoun dezidrate ? Li gen feblès e li gen difikilte pou l leve ________ Dlo pa sòti nan je l menm si l ap kriye _________ Po l fennen (fè pli) _________ Li pa pipi _______ Kou l kase (Li pèd konesans) ________ 6.- IST/VIH/SIDA IST 6.1 6.1.1.- Kisa pou yon moun dwe fè pou evite SIDA oubyen viris ki lakòz yon moun fè maladi SIDA ? M pa konnen _____ Li pa bezwen fè anryen _________ Abstinans (Pa fè bagay) _______ Sèvi ak Kapòt ______ Fè bagay ak yon sèl patnè ______ Evite pran san _____ Evite fè bagay ak bouzen (jenès) ____ Evite pran piki __ Pa sèvi ak sèreng ki deja sèvi _____ Pa pran lang / pa anbrase ______ Evite sèvi ak razwa ou ak jilèt ki sèvi deja ______ Lòt bagay ___________________________________ 6.1.2.- Eske w konnen kijan yon moun ka konnen ke li menm oubyen mari l gen jèm SIDA a nan san l ? M pa konnen _______ Li pa konnen _______ (Dapre sa li di) Lè li fè tès nan laboratwa ______ Nan sant sante ______ Nan yon sant kote yo bay konsey ________ IST 6.2 6.2.1.- Eske w tande pale de lòt maladi yon moun ka trape nan fè bagay ? WI _____________ NON ____________ 6.2.2.- Kilès ? Ekoulman (Gonore/ Kanal brile) ________ Sifilis ____________ Chank __________ IST 6.3 6.3.1.- Lakay yon fanm ki siy ki ka fè w kwè li pran yon maladi nan fè bagay ?: M pa konnen ________ Doulè anba vant _______ Ekoulman (dlo nan bouboun) ____________ Ekoulman ak odè _____ Kanal brile lè l ap fè pipi ____ Bouboun li wouj ______ Chank (ilsè,/ bouton) _______ L ap megri ___________ San nan pipi (Pipi a wouj) ___ Li pa ka fè pitit _______ Gratèl (demanjezon) _____ Lòt bagay _______________________________ IST 6.4.1.- Eske w te gen yonn nan siy ak sentòm sa WI _____________ NON ____________ 29 6.4 yonan lane ki sot pase la yo ? 6.4.2.- Kilès nan yo ? Doulè anba vant _______ Ekoulman ak odè ______ Ekoulman (dlo nan bouboun) __________ Kanal brile lè l ap fè pipi _____ Bouboun li wouj _____ Chank (ilsè,/ bouton) __________ L ap megri _____ San nan pipi (Pipi a wouj) ______Li pa ka fè pitit _____ Gratèl (demanjezon) ________ Oken nan sa yo ______ Lòt bagay ____________________________________ IST 6.5 6.5.1.- Dènye fwa ou te fè lanmou avek mari w, eske l te sevi ak kapot ? WI _____________ NON ____________ 7.- Hygiene HG 7.1 7.1.1.- Kilè ou lave men w ? (Si manman an di sèlman yonn ou 2, mande l) : Kilè ankò? Avan nou manje ____ Avan nou bay timoun manje ___ Apre nou fin ale nan twalèt _______ Apre nou fin netwaye yon ti moun ki fè tata ______ Lè men nou sal ________ HG 7.2 7.1.2.- Kilè ou lave men w ak savon ? (Si manman an di sèlman yonn ou 2, mande l) : Kilè ankò? Nou pa sèvi ak savon ditou _____ Avan nou manje ____ Avan nou bay timoun manje _______ Apre nou fin ale nan twalèt ___________ Apre nou fin netwaye yon ti moun ki fè tata ______ Lè men nou sal ________ 30