Final Evaluation of a Child Survival Program Submitted by: Gretchen Glode Berggren, M.D., M.Sc.Hyg. Judith Gillens, MSW, PhD Philip Moses, MPH Tom Davis, MPH Micheline Baguidy, M.D. Dilia David Keneth Vernet, M.D. Foundation of Compassionate American Samaritans (FOCAS) Haiti, District of Petion-Ville Cooperative Agreement Number FAO-A-00-97-00028-00 October 1, 1997 – September 30, 2003 October, 2003 2 Table of Contents Page A. Summary Description of Program and Objectives 4 B. Assessment of Results and Impact of the Program 6 C. Results -- Technical Approach 27 D. New Tools and Approaches; Operations Research and Special Studies 44 E. Training 57 F. Program Management 57 G. Technical and Administrative Support 62 H. Other Issues Identified by the Team 63 I. Conclusions and Recommendations 63 J. Results Highlights 67 K. Evaluation Team and Survey Methodology 69 Annexes 1. Mentoring Report 74 2. Mortality Report 109 3. Malnutrition Report 115 4. Indicator Tracking Table 120 5. Water Quality Study 129 6. Training Schedules 137 3 List of Acronyms AIDS Acquired Immune Deficiency Syndrome ARI Acute Respiratory Infection BF Breast-feeding CHA Community Health Agent CSP Child Survival Program DHS Demographic and Health Survey EMMUS Enquete Mortalite, Morbidite, Utilisation des Services EOP End-of-project (usually referring to EOP survey carried out by Phil Moses) FOCAS Foundation of Compassionate American Samaritans HIV Human Immunodeficiency Virus IEC Information, education, and communication MEI Mission Evangelique Internationale MMR Maternal Mortality Ratio MNC Maternal and Neonatal Care MSPP Ministere de la Sante Publique et de la Population NMR Neonatal Mortality Ratio NGO Nongovernmental Organization OBDC Oeuvres de Bienfaissance et Developpement Communautaire OPS/OMS Organisation Panamericaine de la Sante/Organisation Mondiale de la Sante (PAHO/WHO) PAHO/WHO Pan American Health Organization/World Health Organization PNC Prenatal Care PVO Private Voluntary Organization QIC Quality improvement Checklist RP Rally Post STD Sexually Transmitted Diseases TBA Traditional Birth Attendant TT Tetanus Toxoid U5MR Under-five Mortality Rate USAID United States Agency for International Development 4 A. Summary Description of Program and Objectives The FOCAS Child Survival Program reaches poor rural and peri-urban communities above Petion-Ville, Haiti with health and development activities. The program grew out of a partnership of two indigenous Haitian NGOs, OBDC (Oeuvre de Bienfaisance et Developpement Communautaire) and MEI (Mission Evangelique Internationale), working with their American partner, FOCAS (the Foundation of Compassionate American Samaritans). MEI and OBDC had established health centers and community development activities in the adjacent “counties” (Communes) of Etang du Jonc and Bellevue de la Montagne. They formed a partnership with FOCAS/USA, a faith-based organization, to assist in child survival and maternal health activities in 1997. The program was mentored by Curamericas, previously Andean Rural Health Care (see Annex 1). The result is a census-based, community-oriented child survival program utilizing home visitation through resident Community Health Agents (CHAs) who now serve over 100,000 poor Haitians. CHAs register family members in their communities, followed by door￾to-door efforts to provide a “personal prompt” for mothers and children to attend nearby assemblies or to avail themselves of curative services at health centers. With community volunteers, they organize regular monthly “rally posts” for activities such as growth monitoring/counseling (GMC) and immunization, as well as for health education on, for example, oral rehydration therapy and other specific interventions. The program seeks enrichment through related partners. For example, FOCAS added activities with the CARE-MoRR (Management of Reproductive Risks) project. With expertise from the University of Miami (Cincinnati), it carried out a study of water quality from program area water sources. Currently, the program has a lending program with World Relief. 1. Objectives: The program has quantifiable objectives described later in this document. These include, in nutrition, the objective of decreasing the percentage of children under five years of age who are malnourished and increasing the percentage in regular growth monitoring/counseling (GMC). Related objectives are to increase the percentage of mothers who breast-feed appropriately and decrease the percentage of babies ever bottle-fed. Micronutrient objectives include increasing the percentage getting Vitamin A regularly. The program’s diarrhea case management objectives are directed at increasing the percentage of mothers who offer increased fluid and food intake, continued breast-feeding, and ORT with diarrhea episodes. Pneumonia case management objectives are meant to increase the percentage of mothers who seek CHA treatment and referral (N.B.: Qualified CHAs detect and treat for early signs of pneumonia). Vaccination objectives are aimed at increasing the proportion of 12 – 23 month olds who are fully immunized and women fully immunized against tetanus. In child spacing objectives the program seeks to increase the proportion of mothers who use modern contraceptives and the percentage of children ages 0-23 months who were born at least 24 months after the previous surviving child. Maternal health objectives include increasing the percentage of women who have a “qualified” (trained) attendant when giving birth, increasing the percentage of mothers who had at least two prenatal consultations during pregnancy, increasing the percentage of women whose birth attendant used clean delivery techniques, increasing the proportion of mothers who can mention 5 at least two danger signs that warrant special care for pregnant woman or newborns, and increasing the proportion of newborns seen by health workers in first seven days of life. 2. Main accomplishments: FOCAS has developed the administrative capacity to implement, monitor, and maintain accountability for a successful child survival program in partnership with two indigenous NGOs that have also been strengthened. This census-based program makes sure that, to the greatest extent possible, no child (or mother) is left out; this is a major accomplishment given the mobility of the marginal urban populations served. Trained and supervised CHAs organize local rally posts with volunteers to offer preventive and curative services. From their door-to-door registry arranged by address, each can detect absentees and, as a result, conduct follow-up home visits. Program area children and their mothers are immunized, children undergo monthly growth monitoring/counseling (GMC), and their mothers are offered family planning methods and pregnancy referral services as well as ongoing health education that includes training in oral rehydration therapy for diarrhea, and in treatment for respiratory illness, and AIDS prevention. Over 100 local traditional birth attendants (“matronnes”) have been identified, recruited, trained and supervised; local women have organized mothers’ clubs and nutrition clubs that meet monthly, and the program has recently added a microenterprise project for mothers in partnership with World Relief/Haiti. The ongoing monitoring system includes death reports and verbal autopsies, permitting death-by-cause reporting and documentation of the U5MR that may be compared to national data. 3. Highlights from the comparison of baseline and final evaluation: · Decline in under-five death rates (U5MR) (see Annex 2, Curamericas study). OBDC: Decline of 31% in 4 years (from 68/1000 to 47/1000); MEI: Decline of 65% in 4 years (from 188/1000 to 66/1000). · Steady decline in malnutrition rates revealed in ongoing growth monitoring data (see Annex 3) · Proportion of children breast-fed within one hour after birth has increased -- OBDC baseline (1998) was 52%; EOP was 70%; MEI baseline (98) 57%; EOP (03) was 73%. · Propensity to bottle-feed: A major finding is that literate mothers were 5.7 times more likely to use a bottle than were non-literate mothers. · Proportion of children weighed six times in last 12 months has increased dramatically -- OBDC: Baseline was 11%; 2003: 81%; MEI Baseline was 32%; 2003: 92%. · The program is not simply weighing children and counseling their mothers; it developed interventions. · Oral Rehydration Therapy: 85% (MEI) – 91% (OBDC) of mothers fed more liquids during diarrhea and about 3/4 included ORT. · Over 2/3 of children are fully immunized before their fist birthday: MEI: 67%; OBDC: 70%; vaccination “drop out rate” has declined. · Proportion of women using a modern contraceptive method has increased dramatically: OBDC baseline: 12.5% up to 57% in 2003; MEI baseline was 15%, up to 50% by 2003. · More than ¾ of mothers use a trained traditional birth attendant; this is equal to or higher than baseline levels. 6 · Mothers tended to receive at least two doses of tetanus toxoid vaccine at a significantly higher rate than at baseline, OBDC from 9% of all mothers interviewed in 1998 to 26% of all mothers interviewed in 2003, MEI from 11% of all mothers in 1998 to 31% in 2003. Due to stock-outs of vaccine and an international supply shortage these proportions fell short of the goal of 55% and 42% for OBDC and MEI respectively. 4. Priority conclusions: · This has been a successful program with an appropriate mix of child survival and maternal health strategies for Haiti, carried out by two indigenous NGOs in partnership with FOCAS, an organization that had little previous experience with child survival strategies. In the opinion of the evaluators, USAID/Child Survival is to be complimented for funding the successful mentoring of the program by Curamericas. The extensive training offered by Curamericas has permitted the program to have excellent supervision, monitoring, and evaluation parameters. · The field program, run entirely by Haitians, appropriately maintains health centers where early referral can be accomplished. Thus, integrated child care is possible. The program can therefore address the issue that Haitian children are more malnourished than any other children in the western hemisphere, and that frequent infections contribute to this fact. Many infections are prevented (as in immunization against measles). Under supervision, CHAs have implemented early detection and treatment of diarrhea and respiratory infection in children, and children can be referred to clinics for further care. · The program has had to overcome two major barriers: 1) high mobility of marginal urban populations and 2) frequent “stock-outs” of vaccine, Vitamin A, and growth monitoring cards that had been promised to the program by the government district health office. · Haiti’s highly mobile marginal populations: CHAs register new residents as soon as it is apparent they have moved in with intent to stay and invite them to rally posts. The turnover or migration is high in the marginal zones; for example, up to 28% of children selected for interviews in some of the marginal zones had either moved away or were outside of the zone for another reason at the time of the 2003 KPC survey. This complicates the ability to compare “baseline” with “final evaluation” data; the program has nevertheless been able to meet most objectives. B. Assessment of Results and Impact of the Program (summary charts) The results are presented for each of the two program partners (MEI and OBDC) separately. For a summary chart with both in same chart, see Annex 4. 7 FINAL KPC RESULTS FOR MEI August 20031 MEI PROGRAM INDICATORS INDICATOR Baselines2 1997 Targets3 FINAL RESULTS4 % of infants/children ( < 24 m) who were breast-fed within the first hour after birth. 57% (CI: 49-65%) 75% 73% (78/107) CI: 63% -- 81% 0-3m 3% CI: 0-6.9% None 69%* (9/13) CI: 38 -- 91% % of infants less than 6 months, who are being given only breastmilk. 0-5m 2% CI: 0-6% 69% 0-5M: 50%* (11/22) CI: 28 -- 72% % of children 0-11m who are being fed with a bottle 58% 42% 30% (13/43) CI: 17% -- 46% % of all children 17-23m who have received at least two doses of Vitamin A in the past 12 months. 45% CI: 26 -- 63% 80% 83% (24/29) * CI: 64 -- 94% 1 Given the wide 95% confidence intervals for this type of survey, reporting data to one decimal point might give a misleading impression of greater accuracy than can be obtained. Thus, we chose to present most proportions rounded to the nearest integer. Also, in light of the large number of percentages given in this report and the occasional use of decimals, we have parted from convention: Percentages that begin a sentence are written as numerals rather than as words (e.g., “47%” rather than “Forty-seven percent”). 2 With Confidence Intervals. Source: “Report on Baseline KPC Survey: October 28 to November 21, 1997, FOCAS / MEI / OBDC Child Survival Project.” November 1997. Arsene Ferrus, MD, MPH, FOCAS Project Manager, Thomas P. Davis Jr., MPH and Julie Mobley, MPH, Project Consultants ARHC. 3 Certain targets were modified in 2001 due to the sizable growth and high mobility of the service population, in conjunction with the midterm evaluation results (see Section C.6). 4 For each proportion the numerator and denominator are listed in parentheses followed by the confidence interval. The confidence intervals were calculated using Epi-Info 2002’s FREQUENCY function in the Data Analysis module. 8 MEI PROGRAM INDICATORS INDICATOR Baselines2 1997 Targets3 FINAL RESULTS4 % of children 2-23m of age who had their growth monitored for the first time at <1m of age (of children with a growth card). 37% CI: 28-46% 62% 82%* (88/107) CI: 74% --89% % of children 12-23m weighed at least six times in the past 12 months. 32% CI: 20 -- 44% 83% 92%* (49/53) CI: 81.8% - 97.9% Percentage of children age 6-9 months who received breastmilk and complementary foods during the last 24 hours. (Non-program Rapid CATCH indicator) 96% (46/48) CI: 91-100% (Note. Baseline questionnaire asked only if mother had initiated complementary foods. The final uses a more liberal indicator than that used for the baseline KPC) Non-program Rapid CATCH indicator 100% (22/22) CI: 85 – 100% % of children 0-23m who were given the same amount/more liquids (other than breastmilk) during a recent diarrheal episode. Same/More: (including breastmilk) 69% (CI: 59-79%) Same/More liquids (other than breastmilk): 46% (CI: 34-59%) 67% Same/More liquids (including breastmilk) : 100% * (27/27) CI: 87 – 100% SAME/MORE liquids (other than breastmilk) 85%* (23/27) CI: 66 – 96%) % of children 0-23m who were given the same amount/more food during a diarrheal episode. 54% CI: 44 -- 64% 50% 82% (22/27) CI: 62 – 94% % of mothers who say they can easily find ORS in their community. 46% 70% 90% (96/107) CI: 82 -- 95% 9 MEI PROGRAM INDICATORS INDICATOR Baselines2 1997 Targets3 FINAL RESULTS4 % of mothers who seek medical assessment or treatment for their child < 24m with cough and rapid, difficult breathing in the past 2 weeks. 44% CI: 33 -- 54% 50 % 36% (4/11) CI: 11 –- 69% Percentage of all children 12-23 m who received BCG, DPT3, Polio3, and Measles before their first birthday. (Card confirmed.) 51% 92% 67% (29/43) CI: 52 -- 81% Vaccination Dropout Rate. Percentage of children immunized with DTP3 less children immunized with DTP1. 16% (CI: 8-92%) 6% 18% (18/98) CI: 11 -- 28% The percentage of non-pregnant mothers of children 0-23m of age who desire no more children in the next two years, or are not sure, who are using a modern method of child spacing 15% CI: 9 -- 21% 40% 50%* (50/101) CI: 39 -- 60% % of mothers of children 0-23 months who had a trained health professional attend to the birth of her child. 74%5 82% 76% (81/107) CI: 66 -- 84% Percentage of mothers who received at least two tetanus toxoid injections (card confirmed) before the birth of the youngest child less than 24 months of age. 61% of those w/cards CI: 43 --79% 11% of all mothers CI: 6 -- 16% 42% 31% *OF ALL MOTHERS (33/107) CI: 22 -- 40% 5 This proportion came from a survey conducted in 2000. 10 OTHER INDICATORS FOR MEI INDICATOR FINAL RESULTS 6 Percentage of children aged 0-23 months with diarrhea in the last two weeks who received ORS and/or recommended home fluids (RHF) 85% (23/27) CI: 66 -- 96% Percentage of mothers age 0-23 months who report that they wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated. (Non-Program Indicator: Rapid CATCH) 6% (7/107) CI: 3 -- 13% Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child (Non-program indicator from Rapid CATCH) 62% (24/39) CI: 45 – 77% Percentage of mothers who received at least two prenatal visits during pregnancy (based on mother’s recall) 74% (79/107) CI: 64 -- 82% Percent of children aged 0-23 months whose delivery involved use of clean instruments or whose cord was cut with a new razor 89% (95/107) CI: 81 -- 94% Percent of mothers of children 0-23m of age who can mention at least two danger signs that warrant emergency care for the mother (during or just after deliver) or the newborn (within the first week of life) (Proposed indicator from 02 proposal. Include as baseline for future programs. Maternal Danger signs 64% (68/107) CI: 54 -- 73% Newborn danger signs 84% (90/107) CI: 76 -- 90% 6 “*” indicates that data show a statistically significant difference between baseline and final means. Confidence intervals are 95% unless otherwise noted. 11 OTHER INDICATORS INDICATOR FINAL RESULTS 7 Percentage of newborns seen by health worker (at the health center or via home visit) within first 7 days of life. 85% (91/107) CI: 77 -- 91% Percentage of children age 0–23 months who are underweight (-2 Standard Deviations (SD) from the median weight-for-age, according to the WHO/NCHS reference population). This is the RapidCATCH indicator. 4.9% (5/103) CI: 2 -- 12% Percentage of children age 0–23 months who are severely underweight (-3 SD from the median weight-for-age). 1% (1/103) CI: 0 -- 6% Median WAZ for 0-23 months -0.33 Mean WAZ for 0-23 months -0.28 CI: -0.51 to -0.6 Percentage of children age 6–23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population). (This age range coincides with FOCAS program monitoring data) 6.3% (5/79) CI: 2.4 -- 14.8% Percentage of children age 6–23 months who are severely underweight (-3 SD from the medi an weight-for-age). 1.3% (1/79) CI: 0.1 -- 7.8% Median WAZ for 6-23 months -0.31 Mean WAZ for 6-23 months -0.34 CI: - 0.61 to 0.07 Percentage of children age 12–23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population). (This age range coincides with DHS nutritional indicator). 4.1% (2/49) CI: 0.7-15.1% Percentage of children age 12–23 months who are severely underweight (-3 SD from the median weight-for-age). 0% (0/46) Median WAZ for 12-23 months -0.40 Mean WAZ for 12-23 months -0.27 CI: -0.60 to 0.05 7 “*” indicates that data show a statistically significant difference between baseline and final means. Confidence intervals are 95% unless otherwise noted. 12 Percentage of mothers who have taken an HIV test 9% (10/107) CI: 5 -- 16% Percentage of mothers responding “yes” to the following question: “If a family member were ill with HIV/AIDS would you be willing to care for him/her in your home?” (This is an indicator of stigma around HIV and coincides with the DHS indicator on stigma) 72% (77/107) CI: 62 -- 80% Discussion of Programmatic Implications, Key Findings from the KPC in Relation to the Baseline Data, and Program Implications 1. Nutrition, growth monitoring and promotion, and anthropometry: Breast-feeding The children in the MEI program area are starting life with better nourishment than they were in 1998. 98% of all mothers reported that their youngest child was currently being breast-fed (up from 77% at baseline). Of the mothers who had weaned their child at the time of the interview, 25% had weaned by nine months, 50% weaned by 12 months and 75% weaned by 17 months. 73% of mothers breast-fed their baby within the first hour after birth. This is an improvement over the 57% measured at baseline. Another 22% put the child to the breast within eight hours of their birth leaving about 97% receiving breastmilk within the first eight hours of birth. LQAS8 analysis shows that mothers in Supervision Areas One, Two, and Three breast-fed their baby within the first hour at a higher rate than of the target (75%) while mothers in Supervision Area Four did so at a rate substantially lower than 75%. It would be beneficial for MEI to compare their program activities in Supervision Area Four with those of the other Supervision Areas in order to find ways to close this gap in performance. Data from the calculation of the Odds Ratio9 suggest that non-literate mothers are about twice as likely to have initiated breast￾feeding within the first hour than were literate mothers. (Although this association is not statistically significant). It might be useful for MEI to investigate this association further to see if there are important practices that non-literate mothers could teach to literate mothers. 90% of the mothers report giving their baby colostrum while only 6% reported giving their child Lòk (a traditional purgative that is often the source of diarrheal disease in newborns). This is down from 47% of mothers giving Lòk at baseline. A notable achievement. 69% of mothers exclusively breast-feed their children until four months. Up from only 3% at baseline. 50% exclusively breast-feed until six months. Up from 2% at baseline.10 Now only 30% of mothers feed their child under 12 months with a bottle. This is down from 58% at baseline and surpassing the original target of 42%. Data from the calculation of the Odds Ratio show that literate mothers were five to six times more likely to use a bottle than were non-literate mothers. It is not clear whether using a bottle is something that most mothers would prefer to do if they could afford it 8 For detailed LQAS results see Annex E, “Results of LQAS and Odds Ratios for Selected Project Indicators” in Report on the Final Knowledge, Coverage, and Practice Survey, August 2003. 9 For detailed Odds Ratio results see Annex E, “Results of LQAS and Odds Ratios for Selected Project Indicators” in Report on the Final Knowledge, Coverage, and Practice Survey, August 2003. 10 MOH Haiti uses 0-3 months for their exclusive breast-feeding (EBF) indicator. International standards define EBF as 0-5 months. This report reflects both indicators for clarity. 13 or if there is another factor influencing bottle use that is associated with maternal literacy. This would be a good issue to explore with a focus group or with other qualitative methods. Complementary Foods All mothers report giving their child aged 6-9 months breastmilk and complementary foods within the previous 24 hours. 83% of all children 17-23 months old received at least two doses of Vitamin A within the past year. This is significantly higher than the 45% reported at baseline. Most mothers offered their child a wide variety of different kinds of food over the previous 24 hours. For a detailed listing of types of foods that mothers report giving their child within the previous 24 hours see “Questionnaire Filled in with Results & Computer Tables for Each Question” in FOCAS 2003 KPC Report. The survey did not, however, measure the number of times per day that a child receives food. This would be a good question to pursue in future studies. Growth Monitoring and Anthropometry In terms of growth monitoring, 97% of mothers could show the interviewer a card for their child (up from 45% at baseline). 82% (88/107) of children two to 23 months old had been weighed within the first month of life (up from 37% at baseline) and 92% of children 12 –23 months had been weighed at least six times within the previous year. This is three times the baseline rate of 32%. The KPC Core team chose to conduct anthropometry as a means of triangulating with very detailed program data. The baseline KPC did not include anthropometry so there is no point of comparison over time. Only 4% (2/49) of children 12-23 months were below two standard deviations from the median weight for age Z-score and 0% were below three standard deviations. A substantially smaller proportion of children are malnourished in the MEI area than at the national level where 21.6 % of 12 –23 month olds were below two standard deviations and 5 % were below three according to the 2000 DHS survey report. 11 2. Sick child questions, diarrhea prevention and case management, and ORS use and availability: 70% of mothers (75/107) could state at least two correct signs12 of childhood illness that indicated the need for immediate treatment by a health worker: · 6% mentioned “looks unwell or not playing normally,” · 32% mentioned “not eating or drinking,” · 26% mentioned “lethargic or difficult to wake,” · 75% mentioned “high fever,” · 41% mentioned “fast or difficult breathing,” · 27% mentioned “vomits everything,” and · 11% mentioned “convulsions.” Most of these danger signs are included in FOCAS educational messages. Since program mortality data indicates that pneumonia, though reduced, is still a major cause of death in 11 Cayemittes, M. Placide, M.F., Barrrère, B., Mariko, S., Sévère, B. (2001) Enquête Mortalité, Morbidité et Utilisation des Services: EMMUS-III Haïti 2000. 12 Signs included as correct were adapted from international IMCI protocols. 14 children, FOCAS should consider emphasizing “fast or difficult breathing” and “convulsions” as well as “chest indrawing” in their education activities with mothers. Diarrhea 25% (27/107) of children had diarrhea in the two weeks prior to the survey. This is 40% lower than the 41% reported at baseline. None of the mothers (0/107) reported washing their hands before preparing food, before eating, after defecating and after cleaning a child that has defecated. FOCAS should consider future efforts in promoting consistent hand washing as an effective means of preventing diarrhea. The program has made much progress in the area of diarrhea case management. 90% (96/107) of mothers said that they can easily find ORS in their community. This is nearly double the baseline rate and overreaches the program target of 70%. Mothers report strong rehydration and feeding practices for their children with diarrhea. 85% (23/27) of children with diarrhea received ORS and/or recommended home fluids. This is rate is 85% higher than the national rate of 46% reported in the DHS 2000. All mothers (27/27) offered the same amount or more of breastmilk and other liquids to a child with diarrhea. While 85% (22/27) of mothers offered that same amount or more of liquids other than breastmilk. These proportions are significantly higher than the baselines of 69% for more liquids including breastmilk and 46% for liquids other than breastmilk. 82% of mothers gave the same amount or more of food during a diarrheal episode. This is up from 54% at baseline. 3. Acute respiratory infections and care seeking: Only 10% (11/107) of mothers reported that their children had an illness with a cough and faster breathing than usual during the two weeks prior to the survey. This is much lower than the proportion of mothers at baseline 44% (131/300). Staff mentioned that August (the time of the interview) is month when few children have acute respiratory infections. Only 36% (4/11) mothers reported seeking care from at least one person qualified to manage pneumonia. While this rate is lower than the 44% at baseline and lower than the 50% target, the small sample size makes it hard to determine any real decrease or increase in the rate. An examination of program data would give a much more accurate determination of how likely mothers are to seek qualified help for pneumonia. 4. Child immunization (cards, full coverage, dropout rate): 67% (28/49) of all children 12-23 months old were fully immunized (received BCG, DPT3, Polio3 and Measles before their first birthday). The breakdown of immunizations received before the first birthday is as follows: · 97% received BCG · 76% received DPT3 · 73% received Measles · 91% received Polio3 While this rate is higher than the baseline of 50.9% it falls short of the target of 92%.13 The vaccination dropout rate showed a slight increase (though not statistically significant) from 16% at baseline 13 For a detailed listing of vaccines received by children 12 – 23 months before their first birthday, see Annex B1, “MEI Child Questionnaire Filled in with Results & Computer Tables” in Report on the Final Knowledge, Coverage, and Practice Survey, August 2003. 15 to 18% (18/98) at final. It would be good for MEI to investigate the reasons behind this lower￾than-expected performance given the strong emphasis this program places on immunization coverage. 5. Maternal and neonatal care: Child Spacing Data show strong advances over baseline in the rate of mothers who desire no more children in the next two years, or are not sure, who are using a modern method of contraception. The final rate of 50% is more than double the baseline and overreaches the final target of 40%. Lot quality assurance sampling (LQAS) analysis on this indicator show that three of the four supervision areas overreached the target for modern contraceptive use while one supervision area (#3) was below target. MEI should consider examining what is happening in both the environment and the services in the high performing areas that is not happening in SA #3 in order to find ways to improve performance there. 62% (24/39) of the youngest children of the mothers interviewed were born at least 24 months after the previous surviving child. This implies that over half of the mothers in the program area have been using some form of contraception over the three years prior to survey. Care During Pregnancy (Prenatal Care and Tetanus Toxoid Coverage) 74% of mothers received at least two prenatal visits during their last pregnancy as compared with 46% at baseline. While mothers tended to receive at least two doses of tetanus toxoid vaccine at a significantly higher rate than at baseline (11% of all mothers at baseline vs. 31% of all mothers at final), this rate falls short of the final target of 42%. LQAS analysis shows that Supervision Areas One, Three and Four were above target while Supervision Area Two was below target. Calculation of the Odds Ratio shows that non-literate mothers were significantly more likely to have received 2 doses of TT than were literate mothers. This is somewhat counter-intuitive since often those with a higher socio-economic status access health care more than those of a lower socio-economic status. MEI might find it useful to explore the association between mothers’ literacy and tetanus toxoid coverage in future qualitative studies. Care of Mothers and Neonates During Labor and Delivery and Postpartum Slightly more mothers of children 0-23 months than at baseline had a trained health professional (including a trained TBA) attend the birth of their youngest child (76% at final versus 74% at baseline) although this percentage falls short of the final target of 82%. LQAS analysis showed that mothers in Supervision Areas One, Two, and Three had a trained health professional attend their birth at a higher rate than that of the final target while mothers in Supervision Area Four received such attention at a lower rate than targeted. MEI should consider exploring the differences between SA Four and the others to find ways to improve performance there. Data from calculating the Odds Ratio suggest that literate mothers were more likely to have a skilled health personnel attend their birth than were non-literate mothers (Although this is not statistically significant). This may reflect the fact that non-literate mothers tend to be less well off than are literate mothers and thus may be less able to pay for trained TBAs who tend to charge more than do non-trained TBAs. It might be useful to explore the differences between literate and non-literate mothers in terms of their preferences to see if the basis for this gap is one of economic access or some other issue. Mothers displayed a significantly stronger knowledge 16 of danger signs14 for their newborns than for themselves during labor and delivery. 84% (90/107) of mothers could mention at least two danger signs that warrant emergency care for her newborn within the first week of life. 64% (68/107) of mothers could mention at least two danger signs that warrant emergency care for themselves during or just after delivery. While mothers’ knowledge of danger signs is strong in general, MEI might find it useful to explore ways to improve knowledge of maternal danger signs during labor and delivery. 89% of mothers (95/107) reported that their youngest child’s umbilical cord was cut with a clean instrument or a new razor. 85% (91/107) of all newborns received a home visit by the community health agent within the first seven days of life. This shows a high degree of care and attention given to mothers and newborns in the program area. 6. HIV/AIDS: 9% (10/107) of mothers report having taken an HIV test. This is higher than the national rate of 4% for all women15 (not just mothers of children 0-23 months old). 72% (77/107) of mothers said that they would be willing to care for a family member in her home if the family member were ill with HIV/AIDS. This is more than double the national rate of 42% for all women. This indicator is used as a way to estimate the level of social stigma that respondents attach to people living with HIV/AIDS. Such a result suggests that stigma may be less of an issue in the MEI program area than in other parts of the country. Data from calculating the Odds Ratio show that literate mothers are three times more likely to be willing to care for a family member with HIV/AIDS than are non-literate mothers. It is not clear whether this relative hesitancy on the part of non-literate mothers is a function of stigma towards people living with HIV/AIDS or one of non-literate mothers having fewer economic means to care for an ill relative in her home. This would be an important issue to explore with qualitative methods before implementing any HIV/AIDS interventions. When asked where they would prefer to take an HIV test if they wanted one only 8% said they did not know where to get an HIV test. This is almost ten times lower than the national average in which 78% of women said that they did not know where they could get an HIV test. Of the mothers who identified a preferred setting for an HIV test, 55% (56/101) mentioned a hospital, 27% (26/97) mentioned a health center and 19% (19/99) mentioned a private clinic or private doctor. The high rate of mothers who have already taken and HIV test, the evidence of lower-than-average stigma attached to HIV infection, and the high rate of mothers who can name a place where they would prefer to get an HIV test display a very promising potential for implementing HIV/AIDS interventions in the MEI program area. The large number of mothers receiving at least two prenatal visits (74%) puts MEI in an especially good position to introduce interventions for the prevention of maternal to child transmission of HIV (PMTCT). 14 Project staff adapted the maternal and neonatal danger signs from those mentioned in “Promoting Quality Maternal and Newborn Care: A Reference Manual for Program Managers”. 1998 Cooperative for Assistance and Relief Everywhere, Inc. (CARE). 15 Cayemittes, M. Placide, M.F., Barrrère, B., Mariko, S., Sévère, B. (2001) Enquête Mortalité, Morbidité et Utilisation des Services: EMMUS-III Haïti 2000. 17 FINAL KPC RESULTS for OBDC August 2003 OBDC PROGRAM INDICATORS INDICATOR Baselines16 1997 Targets17 FINAL RESULTS18 % of infants/children ( < 24 m) who were breast-fed within the first hour after birth. 52% CI: 44 --60% 80% 70% * (100/142) CI: 62- 78% 0-3m: 5% CI: 0-10% None 0-3M: 50%* (10/20) CI: 27 --73% % of infants less than 4 months, who are being given only breastmilk. 0-5m: 4% CI: 0-9% 64% 0-5M: 44% * (16/36) CI: 28 -- 62% % of children 0-11m who are being fed with a bottle 58% 42% 24% (13/54) CI:14 – 38% % of all children 17-23m who have received at least two doses of Vitamin A in the past 12 months. 19% CI: 5 -- 33% 70% 78%* (31/40) CI: 62 – 90% % of children 2-23m of age who had their growth monitored for the first time at <1m of age (of children with a growth card). 54% CI: 44 -- 64% 64% 74%* (104/141) CI: 66 -- 81% % of children 12-23m weighed at least six times in the past 12 months. 11% CI: 3 -- 20% 70% 81% * (58/72) CI: 70 –- 89% 16 With Confidence Intervals. Source: “Report on Baseline KPC Survey: October 28 to November 21, 1997, FOCAS / MEI / OBDC Child Survival Project”. November 1997. Arsene Ferrus, MD, MPH, FOCAS Project Manager, Thomas P. Davis Jr., MPH and Julie Mobley, MPH, Project Consultants ARHC 17 Certain targets were modified in 2001 due to the sizable growth and high mobility of the service population, in conjunction with the midterm evaluation results (see Section C.6). 18 For each proportion the numerator and denominator are listed in parentheses followed by the 95% confidence interval. The confidence intervals were calculated using Epi-Info 2002’s FREQUENCY function in the Data Analysis module. 18 OBDC PROGRAM INDICATORS INDICATOR Baselines16 1997 Targets17 FINAL RESULTS18 Percentage of children age 6-9 months who received breastmilk and complementary foods during the last 24 hours. (Non-program Rapid CATCH indicator) 94% CI: 86 -- 100% NONE 9319% (26/28) CI: 76.5 – 99.1% % of children 0-23m who were given the same amount/more liquids (other than breastmilk) during a recent diarrheal episode. Same/More Liquids (including Breastmilk): 63% CI: 55 --71% More liquids (other than Breastmilk): 43% CI: 31 --54% More liquids other than breastmilk 72% SAME/MORE: 100% * (55/55) CI: 94 – 100% 91%* (50/55) CI: 80 -- 97% % of children 0-23m who were given the same amount/more food during a diarrheal episode. 58% CI: 49 --68% 67% 64% (35/55) CI: 50 – 76% % of mothers who say they can easily find ORS in their community. 46% 70% 91% (133/146) CI: 85 -- 95% % of mothers who seek medical assessment or treatment for their child < 24m with cough and rapid, difficult breathing in the past 2 weeks. 53% CI: 41 -- 65% 60% 53% (19/36) CI: 36 -- 70% % of all children 12-23 m who received BCG, DPT3, Polio3, and Measles before their first birthday. (Card confirmed.) 17% (19/114) CI 9 – 24% 83% 70% * (37/53) CI: 56-82% Vaccination Dropout Rate (% of children immunized with DTP3 less children immunized with DTP1) 39% CI: 25 – 53% 6% 22% (31/138) CI: 16 -- 30% 19 Baseline questionnaire asked only if mother had initiated complementary foods. The baseline KPC uses a more liberal indicator than that used for the final KPC. 19 OBDC PROGRAM INDICATORS INDICATOR Baselines16 1997 Targets17 FINAL RESULTS18 % of non-pregnant mothers of children 0-23m of age who desire no more children in the next two years, or are not sure, who are using a modern method of child spacing 12% CI: 7 -- 17% 40% 57%* (83/145) CI: 49 -- 65% % of mothers of children 0-23 months who had a trained health professional attend to the birth of her child. 77% 89% 80% (118/148) CI: 72 -- 86% % of mothers who received at least two tetanus toxoid injections (card confirmed) before the birth of the youngest child less than 24 months of age. 61% of those w/cards CI: 41 -- 81% 9% of all women CI: 5 --14% 55% 26% * of all women (38/148) CI:19% -- 34% 20 OTHER INDICATORS FOR OBDC INDICATOR FINAL RESULTS 20 % of children aged 0-23 months with diarrhea in the last two weeks who received ORS and/or recommended home fluids (RHF) 76% (42/55) CI: 63 – 87% % of mothers age 0-23 months who report that they wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated. (Non-Program Indicator: Rapid CATCH) 6% (9/147) CI: 3 -- 11% Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child (Non-program indicator from Rapid CATCH) 72% (31/43) CI: 56 -- 85% Percentage of mothers who received at least two prenatal visits during pregnancy (based on mother’s recall) 78% (115/147) CI: 71 -- 85% Percent of children aged 0-23 months whose delivery involved use of clean instruments or whose cord was cut with a new razor 94% (137/147) CI: 88 – 97% Percent of mothers of children 0-23m of age who can mention at least two danger signs that warrant emergency care for the mother (during or just after deliver) or the newborn (within the first week of life) (Proposed indicator from 02 proposal. Include as baseline for future programs. Maternal Danger signs 65% (95/147) CI: 56 -- 72% Newborn danger signs 81% (120/147) CI: 74 -- 88% Percentage of newborns seen by health worker (at the health center or via home visit) within first 7 days of life. 84% (118/141) CI: 76.5 – 89.4% 20 “*” indicates that data show a statistically significant difference between baseline and final means. Confidence intervals are 95% unless otherwise noted. 21 ANTHROPOMETRY INDICATORS FOR OBDC INDICATOR FINAL RESULTS 21 Percentage of children age 0–23 months who are underweight (-2 Standard Deviations (SD) from the median weight-for-age, according to the WHO/NCHS reference population). This is the RapidCATCH indicator. 12.5% (18/144) 95%CI: 7.8-19.3% Percentage of children age 0–23 months who are severely underweight (-3 SD from the median weight-for-age). 0% (0/144) Median WAZ for 0-23 months -0.51 Mean WAZ for 0-23 months -0.42 CI: -0.63 to -0.21 Percentage of children age 6–23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population). (This age range coincides with FOCAS program monitoring data) 15% (16/107) CI: 9.0-23.4% Percentage of children age 6–23 months who are severely underweight (-3 SD from the median weight-for-age). 0% (0/107) Median WAZ for 6-23 months --0.71 Mean WAZ for 6-23 months -0.69 (CI: -0.90 to -0.47 ) Percentage of children age 12–23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population). (This age range coincides with DHS nutritional indicator). 21% (14/67) CI: 12.3 to 32.9% Percentage of children age 12–23 months who are severely underweight (-3 SD from the median weight-for-age). 0% (0/67) Median WAZ for 12-23 months -0.81 Mean WAZ for 12-23 months -0.87 CI: -1.15 to -0.59 21 “*” indicates that data show a statistically significant difference between baseline and final means. Confidence intervals are 95% unless otherwise noted. 22 HIV/AIDS INDICATORS FOR OBDC INDICATOR FINAL RESULTS 22 Percentage of mothers who have taken an HIV test 13.0% (19/146) CI: 8 -- 20% Percentage of mothers responding “yes” to the following question: “If a family member were ill with HIV/AIDS would you be willing to care for him/her in your home?” (This is an indicator of stigma around HIV and coincides with the DHS indicator on stigma) 59% (86/146) CI: 50 -- 67% Discussion of Programmatic Implications, Key Findings from the KPC in Relation to the Baseline Data, and Program Implications 1. Nutrition, growth monitoring and promotion, and anthropometry: Breast-feeding The children in the OBDC program area are starting life with better nourishment than they were in 1997. 80% (115/144) of all mothers reported that their youngest child was currently being breast-fed (up from 73% at baseline). Of the mothers who had weaned their child at the time of the interview, 28% had weaned by eight months, 50% weaned by 12 months and 71% weaned by 15 months. 70% (100/142) of mothers breast-fed their baby within the first hour after birth. This is a statistically significant improvement over the 52% measured at baseline. Another 20% put the child to the breast within eight hours of their birth leaving about 90% receiving breastmilk within the first eight hours of birth. LQAS23 analysis shows that mothers in Supervision Areas Five, Six, Eight, Ten, and Eleven breast-fed their baby within the first hour at a rate equal to or higher than the target (80%) while mothers in Supervision Areas Seven and Nine did so at a rate substantially lower than 80%. It would be beneficial for OBDC to compare their program activities in Supervision Areas Seven and Nine with those of the other Supervision Areas in order to find ways to close this gap in practice. 90% (128/143) of the mothers report giving their baby colostrum while only 16% (24/147) reported giving their child Lòk (a traditional purgative that is often the source of diarrheal disease in newborns). This is down from 39% of mothers giving Lòk at baseline. 50% of mothers exclusively breast-feed their children until four months. Up from only 5% at baseline. 44% exclusively breast-feed until six months. Up from 4% at baseline.24 Data from the calculation of the Odds Ratio25 suggest that non￾literate mothers may be more likely to breast-feed exclusively than literate mothers and that mothers living in marginal zones may be more likely to breast-feed exclusively than rural 22 “*” indicates that data show a statistically significant difference between baseline and final means. Confidence intervals are 95% unless otherwise noted. 23 For detailed LQAS results see Annex E, “Results of LQAS and Odds Ratios for Selected Project Indicators” in Report on Final Knowledge, Coverage, and Practice Survey, August 2003. 24 MOH Haiti uses 0-3 months for their exclusive breast-feeding (EBF) indicator. International standards define EBF as 0-5 months. This report reflects both indicators for clarity. 25 For detailed Odds Ratio results see Annex E, “Results of LQAS and Odds Ratios for Selected Project Indicators” in Report on Final Knowledge, Coverage, and Practice Survey, August 2003. 23 mothers (data do not show statistical significance, however). It might be useful for OBDC to explore these associations further with qualitative research methods in order to promote exclusive breast-feeding more effectively. Now only 24% of mothers feed their child 0 - 12 months with a bottle. This down from 58% at baseline and surpassing the original target of 42%. Data from the calculation of the Odds Ratio show that literate mothers were five to six times more likely to use a bottle than were non-literate mothers. It is not clear whether using a bottle is something that most mothers would prefer to do if they could afford it or if there is another factor influencing bottle use that is associated with maternal literacy. This would be a good issue to explore with a focus group or with other qualitative methods in order to help guide future efforts to promote EBF. Complementary Foods 93% (26/28) of mothers with a child aged 6-9 months report giving them breastmilk and complementary foods within the previous 24 hours. Vitamin A coverage for children 17-23 months increased three fold over baseline. 78% of all children 17-23 months old received at least two doses of Vitamin A within the past year compared with 19% reported at baseline. Most mothers offered their child a wide variety of different kinds of food over the previous 24 hours. For a detailed listing of types of foods that mothers report giving their child within the previous 24 hours see “Questionnaire Filled in with Results & Computer Tables for Each Question” in FOCAS 2003 KPC Report. The survey did not, however, measure the number of times per day that a child receives food. This would be a good question to pursue in future studies to better inform efforts in nutrition. Growth Monitoring and Anthropometry OBDC has shown strong achievements in growth monitoring. 94% (138/146) of mothers could show the interviewer a card for their child (up from 64% at baseline). 74% (104/141) of children two to 23 months old had been weighed within the first month of life (up from 54% at baseline). Children 12 – 23 months old are being weighed six times more frequently than at baseline. 81% (58/72) of children 12 –23 months had been weighed at least six times within the previous year as compared to the baseline rate of 11%. The KPC Core team chose to conduct anthropometry as a means of triangulating with very detailed program data. The baseline KPC did not include anthropometry so there is no point of comparison over time. 21% (14/67) of children 12-23 months were below two standard deviations from the median weight for age Z-score and 0% were below three standard deviations. Fewer children in the OBDC (0%) fall below 3 standard deviations of the mean than do children at the national level (5%). However, in spite of major programmatic advances, the percentage of children below two standard deviations of the mean in OBDC’s program area remain at about the same level as children at the national level, where 21.6 % of 12 –23 month olds were below two standard deviations according to the 2000 DHS survey report. 26 These rates are substantially higher than those gleaned from program data. A recent analysis of program anthropometry data estimates the rate of children 0-23 months two or more standard deviations below the mean at 3.78%. It is not clear whether this discrepancy is due to selection bias on the part of the program or whether it is an artifact of having such a small survey sample size. The program data is based on anthropometry results from several hundred children, each of whom was weighed regularly over many months. The anthropometry data from 26 Cayemittes, M. Placide, M.F., Barrrère, B., Mariko, S., Sévère, B. (2001) Enquête Mortalité, Morbidité et Utilisation des Services: EMMUS-III Haïti 2000. 24 the KPC has a relatively small sample size of 147 and only measures children at one point in time. Given the highly mobile nature of families living in the marginal zones, it is quite probable that a significant proportion of children weighed during the survey were too new to the program area to have benefited from the program interventions. A closer look at OBDC’s program data would help to determine which of these sets of data is a better indicator of program performance. 2. Sick child questions, diarrhea prevention and case management and ORS use and availability: 69% of mothers (102-147) could state at least two correct signs27 of childhood illness that indicated the need for immediate treatment by a health worker: · 12% mentioned “looks unwell or not playing normally,” · 22% mentioned “not eating or drinking,” · 16% mentioned “lethargic or difficult to wake,” · 82% mentioned “high fever,” · 26% mentioned “fast or difficult breathing,” · 34% mentioned “vomits everything,” and · 16% mentioned “convulsions.” Most of these danger signs are included in OBDC’s educational messages. Since recent analysis of the program’s verbal autopsies indicates that pneumonia, though reduced, is still a major cause of death in children, OBDC should consider emphasizing “fast or difficult breathing” and “convulsions” as well as adding “chest indrawing” in their education activities with mothers to improve timely care seeking for children with pneumonia. Diarrhea The program has made much progress in the area of diarrhea case management. 37% (55/147) of children had diarrhea in the two weeks prior to the survey. This is down from the 51% reported at baseline. 91% (133/146) of mothers said that they can easily find ORS in their community. This is nearly double the baseline rate and overreaches the program target of 70%. Mothers report strong rehydration and feeding practices for their children with diarrhea. 76%(42/55) of children with diarrhea received ORS and/or recommended home fluids. This is rate is 85% higher than the national rate of 46% reported in the DHS 2000. All mothers (55/55) offered the same amount or more of breastmilk and other liquids to a child with diarrhea. While 91% (50/55) of mothers offered that same amount or more of liquids other than breastmilk. These proportions are significantly higher than the baselines of 69% for more liquids including breastmilk and 46% for liquids other than breastmilk. 64% (35/55) of mothers gave the same amount or more of food during a diarrheal episode. This is up from 54% at baseline. 6% (9/147) of mothers reported washing their hands before preparing food, before eating, after defecating and after cleaning a child that has defecated. OBDC should consider future efforts in promoting consistent hand washing as an inexpensive and effective means of preventing diarrhea. 27 Signs included as correct were adapted from international IMCI protocols. 25 3. Acute respiratory infections and care seeking: Only 24% (36/147) of mothers reported that their children had an illness with a cough and faster breathing than usual during the two weeks prior to the survey. This is much lower than the proportion of mothers at baseline 63% (190/300). Staff mentioned that August (the time of the survey) is a month when few children have acute respiratory infections. 53% (19/36) of mothers reported seeking care from at least one person qualified to manage pneumonia. This rate has held steady with that of the baseline (also 53%), while still lower than the target of 60%. However, the small sample size makes it hard to determine any real decrease or increase in the rate. An examination of program data would give a much more accurate determination of how likely mothers are to seek qualified help for pneumonia. As mentioned above, only 26% of mothers correctly identified “fast or difficult breathing” as a danger sign for their child. This, along with no measurable improvement in care-seeking, suggest that OBDC might find it useful to emphasize more strongly pneumonia danger signs and the need for timely care as part of its promotional activities with mothers. 4. Child immunization: 70% (37/53) of all children 12-23 months old were fully immunized (received BCG, DPT3, Polio3 and Measles before their first birthday). The breakdown of immunizations received before the first birthday is as follows: · 97% received BCG · 81% received DPT3 · 76% received Measles · 80% received Polio3 While the full immunization rate falls short of the target of 83% 28 , it is noteworthy that OBDC has increased the full immunization rate four times over the baseline of 17%. Data from the calculation of the Odds Ratio suggest that children living in rural zones may be more likely to receive full immunization than are children in marginal zones. The vaccination dropout rate showed a marked increase (though not statistically significant) from 39% at baseline to 22% (31/138) at final. This falls short of the target of 6%. OBDC might find it helpful to investigate the reasons behind this lower-than-expected performance given the strong emphasis this program places on immunization coverage. 5. Maternal and neonatal care: Child Spacing Data show strong advances over baseline in the rate of mothers who desire no more children in the next two years, or are not sure, who are using a modern method of contraception. The final rate of 57% is almost five times the baseline of 12% and overreaches the final target of 40%. 72% (31/43) of the youngest children of the mothers interviewed were born at least 24 months after the previous surviving child. This implies that a large proportion of the mothers in the program area have been using some form of contraception over the three years prior to survey. 28 For a detailed listing of vaccines received by children 12 – 23 months before their first birthday, see AnnexB3, “OBDC Questionnaire Filled in with Results & Computer Tables for Each Question” in the Report on the Final Knowledge, Coverage, and Practice Survey, August 2003. 26 Care During Pregnancy (Prenatal Care and Tetanus Toxoid Coverage) 78% (115/147)of mothers reported receiving at least two prenatal visits during their last pregnancy as compared with 65% at baseline. Calculation of the Odds Ratio shows that rural mothers are much more likely to have received two prenatal visits than mothers in marginal zones. This strong association might be explained by the fact that mothers tend to be better off economically in the rural zones than in the marginal zones. Another factor may be that the highly mobile nature of families in the marginal zones could make it difficult for mothers to gain access to regular prenatal care. OBDC may find it useful to explore this association further with qualitative methods in order to inform future efforts to improve prenatal care coverage. While mothers tended to receive at least two doses of tetanus toxoid vaccine at a significantly higher rate than at baseline (9% of all mothers at baseline vs. 26% of all mothers at final), this rate falls short of the final target of 55%. LQAS analysis shows that Supervision Area Five was the only one to reach the target of 55%. Supervision Areas Six, Seven, Eight, Ten and Eleven all performed at above the mean of 26% but fell short of the 55% target. Supervision Area Nine, however fell far below both the target of 55% and the mean of 26% (no mother surveyed from Supervision Area Nine received two or more tetanus toxoid (TT) vaccines). OBDC should pay special attention to Supervision Area Five to see if there are good program practices that the other Supervision Areas can replicate to improve performance on TT coverage. OBDC should also explore its program activities in Supervision Area Nine to determine why the TT coverage rate is so low there. Calculation of the Odds Ratio shows that rural mothers were about three times as likely to have received 2 doses of TT than were mothers living in marginal zones. OBDC might find it useful to explore the association between mothers’ zone of residence and coverage for tetanus toxoid and prenatal care in future qualitative studies to inform future efforts to improve care during pregnancy. Care of Mothers and Neonates During Labor and Delivery and Postpartum Slightly more mothers of children 0-23 months than at baseline had a trained health professional (including a trained TBA) attend the birth of their youngest child (80% at final versus 77% at baseline) although this percentage falls short of the final target of 89%. LQAS analysis showed that mothers in Supervision Areas Five, Six, Nine and Eleven had a trained health professional attend their birth at or above the final target rate of 89% while mothers in Supervision Areas Eight and Ten received fell short of the target. OBDC should consider exploring the differences between SA Eight and Ten the other SAs to find ways to improve performance there. Data from calculating the Odds Ratio suggest that rural mothers in rural zones were more likely to have a skilled health personnel attend their birth than mothers in the marginal zones (Although this association is not statistically significant). This may reflect the fact that mothers living in the marginal zones tend to be less well off than are rural mothers and thus may be less able to pay for trained TBAs who tend to charge more than do non-trained TBAs. It might be useful to explore the differences between mothers in rural and marginal zones in terms of their preferences to see if the basis for this gap is one of economic access or some other issue. Mothers displayed a significantly stronger knowledge of danger signs29 for their newborns than for themselves during labor and delivery. 81% (120/147) of mothers could mention at least two danger signs 29 Project staff adapted the maternal and neonatal danger signs from those mentioned in “Promoting Quality Maternal and Newborn Care: A Reference Manual for Program Managers”. 1998 Cooperative for Assistance and Relief Everywhere, Inc. (CARE). 27 that warrant emergency care for her newborn within the first week of life. 65% (95/147) of mothers could mention at least two danger signs that warrant emergency care for themselves during or just after delivery. While mothers’ knowledge of danger signs is strong in general, OBDC might find it useful to explore ways to improve knowledge of maternal danger signs during labor and delivery. 94% (137/147) of mothers reported that their youngest child’s umbilical cord was cut with a clean instrument or a new razor. 84% (118/141) of all newborns received a home visit by the community health agent within the first seven days of life. This shows a high degree of care and attention given to mothers and newborns in the program area. 6. HIV/AIDS: 13.0% (19/146) of mothers report having taken an HIV test. This is three times the national rate of 4% for all women30 (not just mothers of children 0-23 months old). Data from calculating the Odds Ratio suggest that literate mothers may be more likely to report having an HIV test than non-literate mothers (This association was not statistically significant). 59% (86/146) of mothers said that they would be willing to care for a family member in her home if the family member were ill with HIV/AIDS. This is higher than the national rate of 42% for all women. This indicator is used as a way to estimate the level of social stigma that respondents attach to people living with HIV/AIDS. Such a result suggests that stigma may be somewhat less of an issue in the OBDC program area than in other parts of the country. Data from calculating the Odds Ratio suggest that literate mothers may be more willing to care for a family member with HIV/AIDS than are non-literate mothers. It is not clear whether this relative hesitancy on the part of non￾literate mothers is a function of stigma towards people living with HIV/AIDS or one of non￾literate mothers having fewer economic means to care for an ill relative in her home. This would be an important issue to explore with qualitative methods before implementing any HIV/AIDS interventions. When asked where they would prefer to take an HIV test if they wanted one only 14% said they did not know where to get an HIV test. This is more than four times lower than the national average in which 78% of women said that they did not know where they could get an HIV test. Of the mothers who identified a preferred setting for an HIV test, 53% (71/133) mentioned a hospital, 22% (28/127) mentioned a health center and 18% (23/124) mentioned a private clinic or private doctor. The high rate of mothers who have already taken and HIV test, the evidence of lower-than-average stigma attached to HIV infection, and the high rate of mothers who can name a place where they would prefer to get an HIV test display a very promising potential for implementing HIV/AIDS interventions in the OBDC program area. The large number of mothers reporting at least two prenatal visits (78%) puts OBDC in an especially good position to introduce interventions for the prevention of maternal to child transmission of HIV (PMTCT). C. Results -- Technical Approach 1. Brief Overview The program is census based and community oriented; community volunteers help salaried community health agents (CHAs) to organize monthly rally posts in their own villages where 30 Cayemittes, M. Placide, M.F., Barrrère, B., Mariko, S., Sévère, B. (2001) Enquête Mortalité, Morbidité et Utilisation des Services: EMMUS-III Haïti 2000. 28 services are offered. CHAs are trained in house numbering and mapping their communities; they go door-to-door to register and educate local families in preventive health activities such as growth monitoring/counseling (GMC), oral rehydration therapy, and specific interventions such as early detection and beginning treatment of respiratory infections in children. After monthly rally posts, CHAs carry out home visits to absentees to provide a “personal prompt” and to offer to weigh children in their own homes if the mother is indisposed. At village rally posts, CHAs are trained to carry out immunization, growth monitoring/counseling, with distribution of Vitamin A, early diagnosis and treatment of respiratory infections in children, demonstration education in ORT, deworming, and referral of ill children to nearby health centers. Education in hygiene, sanitation, breast-feeding and weaning foods, and general nutrition is also carried out at rally posts. Maternal health services include early detection and referral for prenatal care and family planning. The program employs at least three physicians at all times who keep regular hours in the health centers. The information system is well developed and has been used to guide the program. There are monthly reports on rally posts held: number of children seen by age group and immunization/dose received, number of children weighed by age group and nutritional status, number receiving Vit. A, and the like. There are monthly “field report” data (including birth and death reports, reports on respiratory illness cases diagnosed and treated, pregnancies recorded, monthly weights on children under three, immunization administered, Vit. A and iron administered ) coming into headquarters. These reports are generated by salaried resident home visitors known as “Agents de Sante” (CHAs). The CHAs were identified and recruited in their own communities and centrally trained. The program benefited from many training sessions carried out by their mentor, Curamericas (previously Andean Rural Health Care). CHA activities include growth monitoring/counseling with nutrition education, distribution of Vitamin A, deworming, family planning education and contraceptive distribution, immunization, training in hygiene and sanitation, education about antenatal care, use of trained birth attendants, neonatal care (including use of colostrum), exclusive breast-feeding for the first six months of life, and information on AIDS prevention. Four nutrition “monitrices” in the past organized village level nutrition rehabilitation/education units according to the Hearth/positive-deviant method. They now organize “nutrition clubs” for mothers of malnourished children. The program identifies, recruits, trains, equips, and supervises traditional birth attendants (“matronnes”), CHAs, and community volunteers. The latter help CHAs organize monthly “rally posts,” or assembly points, in local neighborhoods, to bring services such as immunization, distribution of contraceptives, and growth monitoring/counseling (GMC). Ill persons, pregnant women, persons in need of more advanced family planning or medical care, such as severely malnourished children, are referred to the program’s own health centers or to nearby hospitals near the impact areas for follow-up on cases needing rehabilitation. Overall Nutrition Objectives: Decrease proportion of children 6 months-59 months of age who are malnourished; increase the proportion who have been weighed and mother counseled at least six times in year prior to survey; increase the proportion of mothers who breast-feed within first 29 hour after birth; increase proportion who breast-feed exclusively in first 4 – 6 months of life; and decrease proportion ever bottle-fed. Diarrhea Treatment Objectives: Increase proportion of mothers offering increased fluid and food intake with diarrheal episodes, including continued breast-feeding; increase proportion who can procure and use ORT salts. Pneumonia Case Management Objectives: Increase proportion of mothers who seek assessment or treatment and referral at CHA level. (N.B.: CHAS have been trained to count respirations and look for early signs of pneumonia.) Vaccination Objectives: Increase the proportion of 12 – 23 month olds who are fully immunized; mothers fully immunized against tetanus; and decrease the vaccine coverage “drop￾out” rate. Child Spacing Objectives: Program sought to increase the percentage of mothers who use modern contraceptives and the percentage of children 0-23 months who were born at least 24 months after previous surviving child. Maternal Health Objectives: Most of the indicators below were added due to the result of the program’s collaboration with CARE in the CARE-MoRR project that enhanced interest and training in this area. The program also trained approximately half the active TBAs in the area. The program sought to: · Increase % of women who had a “qualified” (trained) health attendant at birth of last baby. · Increase % of mothers who had at least two prenatal consultations during pregnancy (N.B.: Added indicator in 2002). · Increase % of women whose birth attendant used a clean delivery kit or who used a new razor to cut the umbilical cord (N.B.: Added indicator in 2002). · Increase proportion of mothers who can mention at least two danger signs that warrant special care for a pregnant woman or newborn (N.B.: Added indicator in 2002). · Increase proportion of newborns seen by health worker in first 7 days of life (N.B.: Added indicator in 2002). The program began in 1997, with a baseline survey in 1998 and the establishment of the information system that tracks key indicators. The local division “Department” of the Ministry of Health and Population of Haiti (MSPP) enthusiastically endorsed the program from its outset. However, as neither of the implementing NGOs had previous experience with child survival activities, USAID established a mentoring system for this program. The staff of the mentor, Curamericas, have made repeated trips to Haiti to help train, plan, implement, and assure that activities are carried out following good protocols, and that good management and evaluation techniques are in place. 30 FOCAS staff created a detailed implementation plan (DIP) in 1998 that was followed by a midterm evaluation (Sept 2001) and this final evaluation, which includes results from an end-of￾project “knowledge, practices, and coverage” (KPC) survey. The final evaluation team added results from ongoing monitoring data, from focus groups, from field visits, and from interviews with staff at all levels, as well as from local groups of mothers who live in the communities served. The program was enhanced by an early project to evaluate and improve water and sanitation sources, by the CARE-MoRR project for maternal health, and by many workshops and trainings through the Ministry of Health and Population Haiti. In addition, in cooperation with CARE international, the program was evaluated as a part of the CARE-MoRR project initiated in Haiti. A Maternal and Neonatal Care Assessment report was prepared as part of this project in March of 2002. Finally, a report on the use of ongoing data, including mortality study results, was prepared by Tom Davis (Curamericas) in 2002. 2. Locale and population covered: FOCAS in Haiti began child survival activities by working in sections of the commune of Petion-Ville: Fourth Rural District – Bellevue la Montagne with Mission Evangelique Internationale (MEI) and in the Third Rural District – Etang du Jonc with Ouevres de Bienfaissance et de Developpement Communautaire (OBDC). The two indigenous nongovernmental organizations (NGOs) began community-based health services in these two districts. Each NGO covered different defined geographic areas, adjacent to each other, above Petion-Ville, and each served a growing marginal population living in “bidonvilles” (slums) at the periphery. The program now covers more than 100,000 censused inhabitants; both cover populations that a few years ago were mainly rural, but that now abut and include the rapidly growing slums around Petion-Ville. 3. Intervention mix: There are six key CSP interventions for this program: ÿ Nutrition improvement, including Vitamin A distribution and promotion of breast-feeding (25% level of effort); ÿ Diarrhea case management (10%); ÿ Pneumonia case management (20%) ÿ Immunization (10%) ÿ Child spacing promotion (25%) ÿ Maternal and newborn care (10%) AIDS awareness and education has been addressed as possible within the present scope of work; additional efforts are being developed through other funding resources and partnerships. The intervention mix includes preventive and limited curative services: 31 Preventive: At neighborhood rally posts, preventive services are available in nutrition (growth monitoring/counseling, nutrition education, and Vit. A distribution), immunization (tetanus toxoid for women; BCG, DPT, Measles, and Polio vaccination for under-fives), and through identification and referral of pregnant women. CHAs carry out follow-up home visits to absentees and to children found to be at risk for malnutrition. There has been spotty availability of dry ration supplements in cooperation with Catholic Relief Services and other providers. Curative: CHAs can do simple exams to detect early pneumonia and begin treatment with cotrimoxazole, then refer; CHAs make oral rehydration therapy salts available and train mothers in its use. The program has identified, recruited, and trained about half the active traditional birth attendants and has additional TBA safe birthing supplies on order. The program maintains four health centers staffed by auxiliaries, with specific hours for physician consultations. 4. General program strategy: Each NGO maintains its own employees, health centers, CHAs, and information system (though the system elements, policies, and procedures are the same for FOCAS and its partners). This is due in part to the fact that the program has been successfully mentored by Curamericas; the suggestion of USAID/Washington Child Survival advisors. Common to both partners is the recruitment, training and, supervision of resident home visitors known as community health agents (1:2000 population), who bring preventive services to their own neighborhoods and who are assisted by the FOCAS itinerant technical support team for service implementation and logistical support. The program works in collaboration with and is to some extent supplied by the local Ministry of Health authority, the District MSPP offices in Petion-Ville. The program currently employs three Haitian physicians and advises its partners in their support of four health centers, in addition to having helped them establish liaisons and referral systems with private and government hospitals nearby. Progress in the child survival program in the past six years has been significant under the mentoring program funded by USAID (see above). Ongoing monitoring data shows that the current level of malnutrition has decreased despite a change in nutrition interventions. The end￾of-project breakdown on immunization levels for children shows coverages between 73% and 97%. Growth monitoring/counseling is carried out with community participation. Over 100 traditional birth attendants have been identified and trained, and mothers are beginning to seek early antenatal care. CHAs (mostly women) have been trained in ORT and in pneumonia case management, and they visit each newborn within one week of the child’s birth to encourage the mother in breast-feeding and to review her needs, including family planning. Finally, ongoing vital events reporting has been expanded to include both Curamericas and the FOCAS headquarters staff. 5. Field strategy and related tools: In defined neighborhoods, CHAs organize their monthly neighborhood rally posts with the community for services such as immunizations, growth monitoring/counseling, and 32 contraceptive distribution. Home visits are prioritized to families that absent themselves from these posts, to families with newborns, and to families seeking help for respiratory infections. CHAs carry registers for under-threes and for women aged 15-45, kept in geographic order, with appropriate columns to record prospective longitudinal information. For each registrant, they record dates of immunization by type of vaccine, dates/weights from GMC sessions, and information from follow-up home visits. For women aged 15 –45, they record pregnancy status and child spacing information. The team observed that CHAs register pregnancies, births, and deaths in their well-defined neighborhoods and that they enroll in-migrants as soon as it is clear they have moved in with intent to stay. In these registers one can corroborate immunization, family planning, and the antenatal care record of each mother. In the under-threes register one can see the immunization and ongoing weight data records for each child under the age of three. Nutrition “monitrices” follow children who are at high risk for malnutrition and keep special registries on these. Each CHA updates his/her register monthly based on known in-migrations, out-migrations, births, and deaths. Vital events are reported in separate registries; deaths of under-fives and maternal deaths are investigated and a verbal autopsy form is completed. Thus, there has been and continues to be a wealth of data available to evaluators. During this evaluation, the evaluation team met with the CHAs and found them eager to supply answers about all the intervention areas. For example, the team leader, a physician, found CHAs familiar with counting respirations/minute along with detecting early signs of pneumonia and giving an initial dose of cotrimoxazole. They knew where to refer babies with diarrhea or possible pneumonia and the comparative cost to the parent depending on where they refer. 6. Progress report by intervention area: a. Nutrition ÿ Decrease the proportion of children 6–59 months of age who are malnourished by the end of the program. Indicator suggested: Proportion of children whose weight/age Z￾score is more than minus two standard deviations below the international std. median wt/age Z-score. (N.B.: Original program documents are vague on this indicator until the midterm. The indicator introduced by midterm evaluators is for children 6 – 23 months of age.) Present findings: Zones Children (6 – 23 m.) with Z-scores below –2 std deviations (includes – 3); moderate and severe malnutrition Children (6 – 23 m.) with Z-scores -3 std deviations or below; severe malnutrition MEI 6% 1% OBDC 15% 0% National Data (DHS 2000) 21.6% 5.1% 33 The midterm evaluators substituted a new (implied) objective: decrease the proportion of children 12-23 months of age whose weights are more than - 2 std deviations below the international standard median. By choosing this objective, since there was no baseline wt/age data, the evaluators were able to compare their findings to Haitian national data for the region, and to show progress. According to the midterm evaluation, the program had achieved this objective by midterm. The final evaluation showed progress in this area for MEI but not for OBDC, using national data for 12-23 month old children as a “baseline.” These data, due to the sample size, cannot logically be considered. The monitoring data shows progress but, since the end-of-project KPC survey did not distinguish between long-term residents and recent immigrants, the subsample size is too small to consider. A study by Tom Davis (Curamericas) from ongoing monthly weight/age monitoring data revealed that there had been continual decrease in the proportion of children in moderate and severe (“M2 & 3”) malnutrition during the program (see Annex 3). Because malnutrition in Haiti is strongly affected by seasonal variation, and using the same data, the final evaluation team leader looked at the ongoing monitoring data to see whether children had been protected during the three-month season of the year when kwashiorkor and marasmus are most prevalent according to hospital admissions. By 2002, the ongoing data showed that moderate and severe malnutrition were steadily declining. The evaluators also reviewed the ongoing records of weights from children being weighed regularly in “rally posts.” In the OBDC impact area (population 81,258 at June 2003 census), severe malnutrition is very rare and has almost disappeared.31 b. Breast-feeding ÿ Increase the % of mothers who breast-feed their babies within the first hour after birth from 52% to 80% (OBDC) and from 57% to 75% (MEI). Targets were modified in 2001. Result: OBDC: Baseline (98): 52%; end-of-project (03): 70% MEI: Baseline (98): 57%; end-of-project (03): 73% Comment: MEI was nearly at target; OBDC, influenced in part by greater population, mobility showed improvement but was slightly off target. ÿ Increase the percentage of mothers exclusively breast-feeding in the first 6 months of life from 4% to 64% (OBDC); from 3% to 69% (MEI). 31 Results of nutrition objective from DIP: “Increase the average wt/age Z-score of HEARTH/positive deviant (“Ti Foyer”) participants by one standard deviation, comparing WAZ at beginning of Foyer to WAZ 12 months later” is an objective never in the original proposal nor looked for in the baseline; it was added later and is inappropriate, on two counts: first, the project was never able to establish effective “Ti Foyers”(village-level rehabilitation/education units) and substituted “nutrition clubs” instead and, second, the stated objective even for programs that established successful “Ti Foyers” has never yet been achieved in Haiti at the time of the writing of this document. Hearth/positive deviant projects in Haiti are effective in improving average Z-scores and preventing deaths, but to increase by one standard deviation has been impossible. 34 Result: OBDC: Baseline (98) 4%, end-of-project (03): 44% MEI: Baseline (98) 2%, end-of-project (03): 50% Comment: Both fell short of the target but showed significant improvement. ÿ Decrease the proportion of children ever bottle-fed from 58% down to 42% (OBDC) and from 58% down to 42% (MEI). Targets modified in 2001. Result: OBDC reached 24% and MEI reached 30%, both exceeding their targets. Comment: Mothers living in marginal vs. rural zones show no difference in bottle use. Data from the calculation of the Odds Ratio show that literate mothers were five to six times more likely to use a bottle than were non-literate mothers. Perhaps literate mothers are more apt to have employment that takes them outside their homes without their infants. It is not clear whether using a bottle is something that most mothers who can afford it would do or if there is another third factor influencing bottle use that covaries with maternal literacy. This would be a good issue to explore with a focus group or with other qualitative methods. Good progress given that the indicator was added in 2001. Personal opinion of evaluator: Most Haitian mothers will answer what they think the interviewer wants to hear for this question. No bottles are ever seen at weigh-ins! So the mothers know the expected norm. The question should have been: Please may I see the baby bottle? The interviewer then notes the presence or absence of a baby bottle and its contents. c. Growth Monitoring/Counseling ÿ Increase the percentage of children who had their growth monitored for the first time during the first month of life from 54% to 64% (OBDC) and from 37% to 62% (MEI). Targets were modified in 2001. Result: OBDC over-reached target (end-of-project was 74%) MEI: over-reached target (end-of-project was 82%) Comment: Here is one place where the program strategy of the CHAs conducting home visits to the mothers of newborns in first week of life really pays off! This is as good a result as I have ever seen in this age group! ÿ Increase the percentage of children weighed at least six times in last year from 11% to 70% (OBDC) and from 32% to 83% (MEI). Targets modified in 2001. Result: OBDC: over-reached target (end-of-project was 81%) MEI: over-reached target (end-of-project was 92%) Comment: Evaluator observed active attendance at rally posts, high morale, and enthusiasm of parents. Notable was the presence of fathers. This is evidence of the 35 program’s ability to mobilize the community. Note also that absentees were sought out and weighed at home. The latter is a reflection of the program’s information system. d. Micronutrients ÿ Targets modified in 2001: Increase the percentage of children 6 –72 months of age receiving micronutrients according to MSPP protocol, two high doses of Vitamin A per year from 19% to 70% (OBDC) and from 45% to 80% (MEI). Result: OBDC over-reached target; end-of-project shows 78% MEI over-reached target; end-of-project show 83% Comment: This is a remarkable achievement since there were frequent stock-outs of vitamin A capsules at a district and national level. Distribution of iron supplements to children was tried and dropped early in the program history due to the expense of purchasing the product. Iron supplements and multivitamin distribution has continued for pregnant women. e. Diarrhea ÿ Increase fluid intake for children with diarrhea (children who are given the same amount or more liquids other than breastmilk) from 43% to 72 % in OBDC; from 46% to 67% in MEI. Targets modified in 2001. Result: Both OBDC and MEI over-reached target at 91% for OBDC and 85% for MEI. Comment: Excellent result but the fact that diarrhea remains the number one cause of death in this population is troublesome. ÿ Added diarrhea objective (2002): Increase proportion of children with diarrhea in past two weeks who have received ORT. Result: see below for new “baseline,” so that as program continues this can be tracked: {baseline in 2002 is 76% (OBDC); 85%(MEI)}. Comment: EOP survey serves as “baseline” for future program for these geographic areas. ÿ Increase % of children with diarrhea whose parents give them the same or more food during diarrhea (from 58% to 67%, OBDC; from 54% to higher proportion in MEI (54% was higher than target). Targets modified in 2001. Result: OBDC: 64% (almost at target) MEI: 82% (above target) Comment: Overall excellent results. 36 ÿ Increase the proportion of parents/caretakers who say they can find ORT salts package in community (from 46% to 70%, OBDC; from 46% to 70%, MEI). Targets modified in 2001. Result: OBDC: 91% and MEI 90%. Both were significantly above target. Comment: Overall excellent results. ÿ Newly suggested indicator: Proportion of mothers who say they wash or use ashes to clean their hands before food preparation, before feeding children, and after defecation. Comment: EOP survey got some valuable baselines; nothing to compare with here to measure progress. f. Respiratory Infection and Pneumonia Case Management ÿ Increase the percentage of mothers who seek assessment or treatment for child with cough and rapid breathing from 53% to 60% (OBDC) and from 44% to 50% (MEI). Targets modified in 2001. Result: End-of-project survey result: OBDC: 53%; MEI: 36%. Comment: Evaluator is puzzled by this result because CHAs appear to have been well￾trained and parents interviewed in focus groups seemed to understand. Focus group findings differ from survey findings. CHAs have instruments for counting respirations/minute and parents in focus groups seemed appreciative of the fact that cotrimoxsazole is available through CHAs. This led evaluator to wonder if parents understood the question in the KPC survey. g. Immunization ÿ Increase the percentage of 12 –23 month old children who are fully immunized from 31% to 83% in OBDC and from 67% to 92% in MEI (target modified in 2001). Result: OBDC: 70%; MEI: 67%; below target but excellent in face of frequent stock￾outs of vaccine supply in Haiti, although neither area met target. Comment: Evaluation team went to WHO/PAHO, current supplier of vaccine to Haiti. The person in charge informed the team that all the vaccine that Haiti needs is now in￾country and available to all NGOs through their local Ministry of Health District supplier. However, records from the program show that the district supplier reported “stock￾outs” again and again. The program even offered transport, which seemed to be a bottleneck, to no avail. CHAs are trained to vaccinate and carried out their activities according to protocol, but rarely had all vaccines available. In particular, tetanus toxoid 37 was in short supply during the life of this program. The evaluator checked with other NGOs in Haiti and discovered the problem was pervasive. ÿ Decrease the vaccine coverage drop-out rate from 39% to 6% in OBDC and from 16% to 6% in MEI. Result: OBDC: 22%; MEI: 18%. Comment: See comment above. Did not meet target but excellent result in face of frequent stock-outs of vaccine provided from Haitian MOH (MSPP) district sources. Evaluator visited MSPP district headquarters and confirmed that there had been frequent “stock-outs.” The district officer explained there had been a lack of transport available to the district, communication with national storage area was poor, etc. ÿ Increase the percentage of women who received two doses of TT or more before the birth of their youngest child. Target modified in 2001 as follows: OBDC: 55% of all women; MEI 42%. Result: Below target: OBDC reached only 26% of women MEI reached only 31% Comment: See comment above on stock-outs. Evaluation team called another PVO active in child survival program in urban area and found they had same problem. There was also an international shortage of TT, confirmed by CDC, during the life of the program. Program focused on pregnant women rather than “all women” due to vaccine shortage. N.B.: In verbal autopsies, by 2003, in the OBDC area, one begins to see babies dying from 5th to 15th day of life due to “maladi machwa serre ak kor red” (Creole for tetanus of the newborn). h. Child Spacing ÿ Increase the percentage of non-pregnant mothers of children 0 – 23 m of age who desire no more children in the next two years, and who are using modern contraceptives (MCs) from 12.5% to 40% (OBDC) and from 15% to 40% (MEI). (N.B.: Targets modified upwards in 2001 because program had already over-reached their original goal). Result: Both areas over-reached targets: OBDC now has 57% of mothers (above definition) using MCs; MEI has 50% using. Comment: Haitian women have overwhelmingly approved Depo-provera, (“methode piki”) the most popular method. With acceptance rates like this, the crude birth rate should be affected. 38 ÿ Recently added “Rapid Catch Indicator.” Increase the percentage of children 0 – 23 months of age who were born at least 24 months after last surviving child. Result: (EOP survey looked at this in order to have a baseline for future.) OBDC has 72% and MEI has 62% of mothers who have waited at least 24 months after the previous surviving child to give birth again. Comment: Breast-feeding by itself (usually for 18 months or more in Haiti) will give a two-year child spacing interval. However, there is a trend toward weaning babies earlier (12 months vs. 18 months) according to mothers in focus groups, so that unless women adopt modern contraception, this birth-spacing interval could decrease. i. Maternal and Newborn Care ÿ Increase the proportion of mothers who had a qualified “health professional” attend their last birth: from 77% to 89% (OBDC) and from 74% to 82% (MEI). Targets modified in 2001. Result: both below target according to EOP survey, but see comment below. OBDC reached 80%; MEI reached 76%. Comment: Question may have confused some respondents. Mother was asked if her attendant had a clean birth kit (“bwat”), indicating she was “trained.” But some trained TBAs have not yet received their kits (they are held up in customs). Training of some of the active “matronnes” (TBAs) has been relatively recent in program (last group of trainees was last year). ÿ Recently added objectives/indicators (EOP survey provides these as new “baseline” for future comparison). Percentage of mothers who received at least 2 prenatal consultations during last pregnancy. Result: 78% (OBDC); 74% (MEI). Comment: These are based on mothers’ recall and not on documentation. ÿ Percentage of children 0 – 23 months of age whose delivery involved use of clean birth kit or new razor to cut umbilical cord. Result: 94% had clean cord cut (OBDC); 89% had clean cord cut (MEI). Comment: “New razor blade” isn’t as good as presterilized clean cord cut kit; program can easily acquire these; understand some are held up in customs. ÿ % of mothers of children 0 – 23 months of age who can mention 2 danger signs that warrant emergency care for mother or newborn. 39 Result: OBDC: for mother: 65%; for child: 81% MEI: for mother: 64%; for child: 84% Comment: Already excellent result reflects training received. ÿ Increase proportion of newborns seen within first 7 days of life by health worker. Result: OBDC: 84%; MEI: 85% of mothers said they had received visit during first 7 days of life of their last-born child. Comment: This excellent result may reflect result of training not only by FOCAS but by CARE-MoRR project. 7. Factors affecting achievement: · FOCAS has established administrative capacity beginning with almost none in Haiti at the outset of this grant. It has hired staff, created policies and procedures to guide administrative activities, and given its Haitian counterparts ongoing training. The administrative offices are now supporting field activities for a population of approximately 100,000. There have been at least three training workshops or tutorials related to administration; improved record keeping has led to appropriate accountability. · FOCAS and its NGO partners have reaffirmed their missions and broadened them. For example, there is real commitment to combating HIV/AIDS; proposals to do so are in process. FOCAS has created the possibility for integrated care dovetailing curative services at health centers with preventive services and follow-up. · Leadership capacity has apparently been strengthened. The organization was seen as being nascent and had not yet demonstrated strength in 1997. Now Dr. Micheline Baguidy, Child Survival Project Manager, is highly respected in Haiti, clearly knows how to administrate with attention to details, and has given strict attention to financial accountability. Mme. Dilia David in OBDC and Dr. Vernet in MEI, program managers at the NGO partner offices, are able to critically evaluate their progress toward meeting defined objectives, and to make midcourse corrections. The three (Dr. Baguidy, Ms. David, and Dr. Vernet) are learning about evaluation, and about proposal writing and can supply any information needed to justify the strength of FOCAS to carry out newly proposed initiatives and activities. · Haiti’s highly mobile marginal populations: CHAs register new residents as soon as it is apparent they have moved in with intent to stay and invite them to rally posts. The turnover or migration is high in the marginal zones; for example, up to 28% of children selected for interviews in some of the marginal zones had either moved away or were outside of the zone for another reason at the time of the 2003 KPC survey. This complicates the ability to compare “baseline” with “final evaluation” data; the program has nevertheless been able to meet most objectives. 40 · FOCAS is on the road to institutionalization of KPC surveys and the capacity to carry out evaluative studies and can carry out implementation. However, external consultants from FOCAS/USA are still utilized for design and interpretation of KPC surveys. 8. Contributing factors to objectives not fully achieved: · Haiti’s highly mobile marginal populations: See above. Families present at the beginning of the program are not there at the end and are replaced by immigrant families that have had little time to be exposed to program activities. The program has nonetheless been able to meet most objectives. · Logistics: Frequent stock-outs of vaccine, Vitamin A, and growth monitoring cards at the district level Ministry of Health (MSPP) posed a major constraint. The program was dependent on the MSPP for a constant supply. With persistent, repeated demands, the program procured enough vaccine to maintain reasonable vaccination levels for children. Often, the program asked written permission from district officers to bypass the district level and go directly to the central level for supplies. This proved difficult. Tetanus toxoid however is another matter, and immunization levels in women have fallen behind. The Haitian Ministry of Health finally distributed DT (diphtheria tetanus) toxoid instead of tetanus toxoid, but only in small amounts. Program leaders had to prioritize its use to pregnant women; targets were not met, either in the MEI or OBDC zones. This evaluator contacted other PVOs in the area and discovered that this problem is pervasive. One PVO with a Washington DC funded child survival program in the Port-au￾Prince area explained they had kept a record of all their orders and the infrequent and inadequate filling of such orders, in order to explain their lack of coverage. A visit to the WHO/PAHO office in Port-au-Prince revealed that this organization is aware of the problem and that there is now enough vaccine in Haiti to fulfill the need, but that the only way it can be procured is through the district health offices of Haiti; however, these sources often lack transport and refrigeration for storage. · Financial sustainability remains a problem especially for paying the personnel, and most notably the CHAs who are the backbone of this program. The same is true for other child survival programs in Haiti. Attempts at getting the CHAs to “sell” their wares and thus improve sustainability has raised suspicion and mistrust between CHAs and their constituencies. CHAs in this program, when asked by a donor to “sell” rather than give away their packets of ORT salts, found enormous resistance. Families served by this program are down to one meal per day; purchasing medicine of any kind, in their eyes, takes food from their tables. The fact that many medications are “prescribed” and must be purchased is causing enormous hardship for the poorest of the poor in Haiti. CHAs report taking their own funds, again and again, to help the poor families they serve to buy prescribed medicines. 41 Evaluator’s Comment: It is not only sustainability but equity that should be a concern in urban slums, where the most marginalized families live. This program has worked hard to maintain the latter.32 Comment from Dr. E. Anne Peterson, USAID, in Global HealthLink, Jan-Feb 2003, #119: “Sustainable financing for routine health interventions is always a problem. Think about it, whether it’s brushing our teeth or washing our hands here [in the U.S.], it is hard to get people to keep doing it over and over again. We struggle with that for sustainability in the international health arena. That means re-invigorating things that we know really do make a difference. But frankly, they get forgotten and under-funded.” (page 2 in USAID’s Global Agenda) “Remember, if there is a moral mandate having to do with HIV/AIDS, then there should be the same mandate deal with the millions of children and women who are dying of all the other health issues.” ( p.3, Ibid). · Is the program addressing the issue of sustainability? The following steps are being taken to continue this program: FOCAS has written proposals to at least six organizations, some of them local in Haiti, for continued funding. According to government workers, there is a need to expand child survival activities into the newly forming urban slums around Port-au-Prince and Petion-Ville. The “FOCAS partnership” would be an ideal organization to do so. 9. Main lessons learned: Nutrition · With appropriate community mobilization, the HEARTH/Positive Deviance (“Ti Foyer”) intervention, might have succeeded. It appears that its failure was apparently due to inadequate training and improper implementation. According to implementers, the program tried to establish “Grand Foyers” (for 20 children) instead of “Petite Foyers” (for 4 – 5 children in a defined neighborhood). Parents saw the intervention as simply a day care and “feeding center,” and they wanted to send all their children, malnourished or not, and did not understand that their “ticket” to participation was some kind of daily contribution (a bit of food, fuel for cooking, water, etc.). According to Jerry and Monique Sternin (Tufts University School of Nutrition; successful users of the method), 32 Historic note: Although CHAs are recognized by the MSPP as important in Haiti, and a few are on the MSPP payroll, government funds are generally not available to sustain enough of them to make a difference. For example, government paid CHAs in the South reported to this evaluator during a nutrition survey that each was responsible to do home visits on populations of 5,000 people or more. A rare exception, during the 1970’s and 80’s in urban Haiti, was the use of CHAs by the MSPP’s Division d’Hygiene Familiale (DHF) under the direction of Dr. Arey Bordes. The DHF currently has few funds to continue such activities. So far, this evaluator knows of no PVO project in Haiti that has nothad to find funding for CHAs or their equivalent. Catholic Relief Services (CRS) identifies and recruits “collaborateurs volontaires” (volunteers) and recompenses them for their services on the day they help with food distribution. According to local TBAs interviewed, local citizenry is not very willing, either, to pay fairly for their services. Most of them say they have done many deliveries for nothing or for a few “goods in kind”. When confronted with the need to call a trained TBA who will expect to be paid, many couples resort to calling the “auntie next door” or a relative. This was the case in up to 25% of deliveries according to birth reports in the 1990’s in the district of the Albert Schweitzer Hospital near Deschappelles. 42 community mobilization may take as much as three months. Parents and community leaders must reach consensus on the merit of sustainably reducing malnutrition. They must support mothers to attend the two week training and rehabilitation sessions and participate in the two week follow-up, supporting the mothers to attend, and helping them to make their contribution. · Parents (including fathers), caretakers, and community leaders must see the value of and reach consensus on the value of sustainably reducing malnutrition before any intervention can succeed. · Real “catch-up growth” rarely occurs in children whose parents are in Nutrition Clubs that replaced HEARTH/Positive deviance methods. The EOP team carried out an evaluation of records of monthly weight gain of children in the OBDC area “nutrition clubs” and found that more than 60% of children in the clubs grow as fast or faster than the international standard median rate at four months after entry. This is not a bad result, but few grew at a rate faster than the international standard median for wt/age (“catch up growth”). · Nutrition messages appropriate to the underlying causes of malnutrition in Haiti should be substituted for the “three basic food groups” message. Messages should address the fact that children need more than one or two basic meals per day along with nutritious snacks. Replacement with appropriate messages could help bridge the knowledge gap. For example, instead of the “three basic food group” message, teach and demonstrate refeeding children who are anorectic or are not gaining weight; demonstrate use of calorie-dense snacks so that children eat five times a day. · Children who fail to respond (inadequate weight gain after two months) in a nutrition program need referral to rule out chronic disease such as hidden TB. · Nutrition intervention for the poorest families needs to be combined with income generating programs and financial lending initiatives. Micronutrients · Iron and iodine supplementation in addition to Vitamin A are needed. More than 30% of children are anemic in Haiti and mountain children, according to UNICEF in Haiti, need iodine supplementation. The program did not plan to have resources to combat anemia; the program taught about the use of iodized salt, but it has disappeared in village market places. Diarrhea · Deaths from diarrhea are still occurring (according to death reports) and may continue unless there is a continuum of services. Caretakers must not only give increased fluids and food and continue breast-feeding, they must understand and utilize ORT; then, be ready to carry the child to a health center where there is a “rehydration corner.” This 43 must include a cot (for the child) and chair for the mother, rehydration salts, wherewithal to administer the ORT, and a doctor and/or auxiliary who will supervise and encourage the mother. In addition, IV fluids must be available for the child no longer able to drink. Respiratory Infection and Pneumonia · Training CHAs to detect and treat pneumonia is not enough to guarantee services will be utilized. There needs to be community mobilization in this regard. According to the survey many mothers whose children suffer respiratory infections do not yet refer themselves to the nearby qualified health agent. Immunization · Offering tetanus toxoid immunization at rally posts is not sufficient to get coverage of women in the reproductive age group against tetanus. Widespread education about immunization of women and the reasons for it is needed. Pastors could help spread the word. Every case of tetanus of the newborn should be investigated and the family asked, “What could we have done to see that the mother of this baby was immunized?” This creates gossip about the case, and the news spreads, as revealed by experience at the Hopital Albert Schweitzer and other places. Child Spacing · Persistent education by CHAs during home visits, and offering of methods of modern contraception gratis at rally posts pays off! The fact that most mothers/newborns are visited within the first week of life has no doubt played a role; mothers are apt to be receptive about child spacing messages at that time. Maternal and Newborn Care · Partnering with a sister organization (in this case CARE and the CARE-MoRR project), qualified to detect needed services and train staff, gives a “jump start” to efforts in maternal and newborn care. Overall Lessons Learned · Door-to-door registration of families, prioritization of home visits to those absent from rally posts, to new members of the population, and to mothers of newborns within the first week of life, has greatly contributed to the success of this program. · “Everyone deserves supervision” and most receive it in this program. · Ongoing staff training and supervision from headquarters is essential. 10. Special Outcomes; unexpected successes and constraints: The program had several special outcomes: 44 · Well-documented and implemented training, in part through a mentor, Curamericas; identified training needs and carried out appropriate education with improved implementation (see Annex 1); · Overwhelming acceptance of modern contraception by mothers of under-twos who desire no more pregnancies in next two years; · Excellent progress in immunization coverage for children despite stock-outs of the supplier; · Excellent AIDS awareness; causes and consequences known by most parents in focas groups (although this was not a program objective); · Documentation of the frequency of water sources (even supposedly “clean ones”) being contaminated with coliform organisms (evaluators observed increased use of sodium hypochlorite as a disinfectant in the home apparently as a result ); · For women in the reproductive age group : low immunization coverage against tetanus with reappearance of tetanus of the newborn; · CHAs apparently competent in early diagnosis and beginning treatment of pneumonia. Constraints: See above mention of constraints. They are: 1) population mobility in marginal urban slums, 2) stock-outs of essential items from trusted supplier, 3) inability to get supplies out of customs, 4) lack of transport, and 5) economic crisis in Haiti and inability of persons to pay for care. 11. Plans for lessons learned to be applied to future activities: · Surprised by EOP survey results concerning the low tetanus immunization coverage, the program leaders plan to ask the MSPP to assist them for an immediate campaign for tetanus immunization of all women. · Program leaders plan contact with partners who can assist with nutrition intervention; they are planning to contact Food for the Poor about possible dry rations to be prescribed for malnourished children along with nutrition education. · Program will reintroduce demonstration-education in nutrition at rally posts and clubs; program leaders plan to resurrect training about Akamil, a traditional weaning food developed by Haiti’s Bureau of Nutrition. · Since diarrhea remains a major cause of death according to “verbal autopsy” forms, the program hopes 1) to improve access to clean water and appropriate use of same, and 2) re-education of mothers about how to procure and safely store clean water. D. New Tools and Approaches; Operations Research and Special Studies New tools developed by the program and Curamericas are the directed at quality improvement; Curamericas’ training materials developed by Tom Davis and others (see page 53 for example) include checklists for supervisors, ways of scheduling spot-checks that are truly random, and regular reporting. A complete description and guide to the program’s information system was developed and is available. 45 New to Haiti is the consistent use of birth and death reporting and the verbal autopsy, carried out by CHAs. These instruments can be accessed at any time by program leaders. The teams review of deaths in under-fives, for example, revealed that in the OBDC area, during the past six months, infant deaths due to “malai machwe sere ak ko red” (tetanus of the newborn) is beginning to reappear. This finding is consistent with low levels of immunization of women documented in the EOP survey. It is already being used to change program direction. Special studies include the study of ongoing nutrition monitoring data, the mortality study, and the water quality studies (see Annexes 2, 3, and 5). In addition, the midterm and final evaluation applied LQAS techniques to determine which program service areas were below par. See notes in the report of the EOP survey. 1. Results in terms of cross-cutting approaches such as community mobilization, communication for behavioral change, capacity building, sustainability strategy, and program management: a. Community Mobilization (i) Effectiveness: Good indirect evidence for community mobilization comes from the fact that the program met or was near target for childhood immunizations in spite of frequent “stock-outs” from the supplier. This meant that mothers and/or fathers were mobilized to attend many rally posts only to find that one time there might not be DPT, the next time, no measles vaccine, and the like. Focus groups at rally posts revealed that caretakers were very aware of the value of immunization, and knew how to “spread the word.” Attendance at rally posts includes many fathers as well as mothers. This speaks to the fact that there are effective efforts to get families mobilized to attend them. Some volunteer their own places for rally posts, and help with the set up. Result is that children are weighed and their mothers counseled regularly; Communities have also participated in identifying and recruiting traditional birth attendants (“matronnes”) and volunteers to assist in rally posts. On the other hand, lack of community mobilization seems to have contributed to the program’s decision to drop the “Ti Foyer” (Hearth/positive deviance) nutrition education and rehabilitation program. Parents did not understand the goal and thought program was a “feeding center.” (ii) Objectives met: Program was at or near target in a number of objectives that depended on community mobilization (see above). A recent analysis of program monitoring data showed an improvement of children classified as moderately or severely malnourished from 10.9% to 5.86% for MEI and from 10.5% to 3.78% for OBDC. One of the added objectives was to mobilize women to participate in the financial lending program sponsored by World Relief as a partner in the area, with the eventual formation of community banks. CHAs were able to quickly contact women and were able to play a role in their mobilization. 46 (iii) Lessons learned for future mobilization efforts: · Special training or retraining in community mobilization is needed in order to keep it going. New staff members, such as recently hired doctors, will need training. · Community mobilization takes time; it may take as much as three months and many visits for the constituents of a community to be willing to reach consensus about the importance of a given activity that will train families in behavioral change. They must understand the benefits and be willing to “buy in” to the activity. For example, in a program like “Ti Foyers” to sustainably reduce malnutrition, fathers may go along with the program if they realize that better nourished children will be ill less often, thus freeing the mother to do her work, and costing the father less since he does not have to travel to or pay for a consultation. Grandparents who often do the babysitting must see the benefit to them as well. This implies meetings with village leaders, with caretakers, and with influential extended family members like grandparents. · Similarly, for a “safe water” program, there must be time spent by program leaders in community mobilization or any newly realized clean sources could quickly fall into disrepair. (iv) Is there demand to continue? Yes, there seems to be a high demand to continue at all levels from families in the impact areas: · Findings from focus groups conducted by the evaluation team found that families expressed gratefulness for the program, especially immunizations, and wanted them to continue. Many demanded why the “Ti Foyers” (Foyers d’Apprentissage et de Rehabilitation Nutritionelle) were dropped. Mothers expressed that they now feel the need for antenatal care, and know the importance of using a trained birth attendant at the time of childbirth. · Beneficiaries in the MEI programhave approached their leaders to demand more health centers and continuing services. · From administrative staff in Haiti and in USA: All staff interviewed wanted the program to continue; USAID personnel told the team they were doing all in their power to find additional funding for the program. b. Sustainability 47 Even though there are temporary cutbacks in the program, program workers are continuing even with some salary loss. Additional funding is being sought through the MSPP, the local USAID mission, and others. FOCAS has written several grants; it is expected that one or more of these will come through. MARCH has sought funds for this program and others for AIDS prevention and treatment that is likely to come through. Are sustainability plans realistic? (i) It is realistic and necessary for the program to continue to seek external funding. The MSPP (government health programs) in Haiti are stretched to the limit; there is at present little hope that, except in the area of AIDS, they will have more resources soon. The program can therefore expect continued “stock-outs” and difficulties in raising capital for salaries for the CHAs. Salaries in Haiti are at an all-time low in terms of purchasing power; this includes salaries for medical professionals. (ii) The program’s entry into financial lending programs that would help at least some families out of abject poverty is a realistic approach to beginning to solve the problem. But for now, it is unrealistic to think that poor Haitians will be able to pay for their own “agents de sante” or other health benefits very soon. Program beneficiaries for the most part are so poor that they are down to only one meal per day; they must eek out their living from small farms or “petit commerce;” many women buy and sell small items like soap and matches and try to live on the small profit. To purchase prescribed medications, mothers described going without food. Since prenatal visits cost something, most confine themselves to only one prenatal visit during a pregnancy. TBAs complain that lately even a few dollars is more than they can expect from families who call them to assist in a delivery. More and more families call the “auntie” next door just to save a few dollars. It is unrealistic to expect that poor Haitians will be able to pay for care. (iii) Given the fact that Haiti is the poorest country in the western hemisphere, with the lowest per capita income, and that there is increasing publicity about the problem, it is realistic to hope for and expect some external funding to come through. As mentioned above, with some external funding, the sustainability plans are realistic. No country in the world has successfully demanded that the poorest of the poor pay for their preventive or even most of their curative services; Haiti is no exception. The MSPP is currently not able to provide much more, though they are seeking to reorganize and become more responsive. Changing the Ministry of Health’s ability to support community-based programs lies beyond the scope of this program. I would, however, expect the parents of the poor children who are beneficiaries, to demand services if they were suddenly dropped. They would first turn to local NGOs because they know about the economic crisis, and that their government is unlikely to be able to respond. 48 Were the sustainability plans that were articulated in the DIP met? There were few sustainability plans mentioned in the DIP. Concerning the status of the phase-over plan, continuing technical management, and assistance, FOCAS plans to continue involvement with the two partner NGOs. Both existed before the FOCAS program, and both are currently falling back on previous sources of donations to assist them during Haiti’s present economic crisis. Approaches to building financial sustainability: · Local level financing: The business community might be convinced to donate more; OBDC is exploring this. Health insurance schemes are beginning to emerge in Haiti through employers, but there is an all-time high in unemployment. As mentioned above, Haiti’s financial crisis has put many families in very precarious situations. Families have cut back to one meal per day in the program areas. · Cost recovery: Only the financial lending program offers any hope. The sale of products (such as ORT, medicines, and family planning materials) and services must be balanced with local economic development initiatives. · Resource diversification: MEI offers school and training programs that could eventually, through increased tuitions, perhaps become a resource. Efforts are underway to improve clinic operations through the expansion of lab services and equipment resources. · Corporate sponsorships: FOCAS, MEI, and OBDC staff are exploring this. Has the program built demand for services? There is an apparent increased demand for services at health centers, but people living in the area have a number of choices. Utilization of clinics is spread out among a number of providers. There is a high demand for field services. A consistent comment at the focus groups was the mothers’ appreciation of the availability of local services and the absence of long lines and hours of waiting to receive care. However, the ability of the community to sustainably pay for the provision of these services in negligible. c. Behavioral Change: Has the Program Been Effective? (i) Effectiveness: The approach has been effective in several ways. The program has preserved equity by insisting on door-to-door registration of under-fives and mothers in the reproductive age group, and on seeking out women and children who might absent themselves from services (usually the poorest of the poor, mothers who are too sick and/or hungry to come to the rally posts, or who are depressed). This requires a behavioral change on the part of the medical professionals and others who manage such programs, as well as on the part of the CHAs. (ii) Were community health behavioral change objectives met? 49 Abundant evidence is presented above that most were met. Has there been effective communication that has brought about behavioral change? Some observations (out of many) by the team in the area of nutrition and family planning are as follows: · A rapid review of weight gain for children of mothers in the nutrition clubs shows that four months after entering the club, nearly 2/3 of malnourished children are growing as fast or faster than the international standard median is indirect evidence that mothers are responding to nutrition education and adopting better child feeding practices. · Mothers in focus groups knew the value of colostrum and insisted that they no longer discard it; thus they have abandoned a traditional practice that had adverse consequences; the EOP survey confirmed this, and TBAs described their teaching about this. · Fewer mothers are bottle-feeding. · Mothers who wish no more pregancnies are adopting the use of modern contraception, a significant increase over baseline. · Most important is the perception by the beneficiaries (as revealed in interviews) that the CHAs and the program staff care about the people, and that people respond when there is communication in that light. (iii) What were the lessons learned? · “Cared for mothers can care”! This was not “just a job” for many actors in the program. CHAs described reaching into their own pockets to help the poorest mothers. The mothers, in turn, were very receptive to the CHA messages. · Repeated messages, sometimes re-enforced with songs, eventually “get through” to mothers who participate. · Community mobilization is a necessary predecessor of getting families and communities to be open to communication about behavioral change, and to be committed to helping with it. (iv) How will these behaviors be sustained? · At the personal and family level: I would expect that many of the behaviors would be sustained, such as use of colostrum, handwashing, feeding more liquids during diarrhea, and the like. Adult participatory education theory teaches us that an adult must be exposed to a message seven times before adopting a new behavior. Messages were repeated 50 again and again in this program, with consistency. Adults must recall, recall and appreciate, and then will try a new practice. But recall, and recall with appreciation, and practice are necessary again and again. Health behaviors will need recall and encouragement. Continued rally posts with more demonstration education related to the messages will help sustain the behaviors. · At the community level: With the action of the CHAs, or a replacement for them, such as mothers care groups, behaviors such as gathering at rally posts for immunization, distribution of Vitamin A and the like, can be expected to continue. · If everything reverts to the health centers and self-selection, the behaviors will be more difficult to sustain. Studies in Haiti show that one mile out from a health center, the coverage (for immunization and the like) drops to 30%. d. Capacity-Building Approach (i) Strengthening the PVO · Assessment of the capacity-building effects of the program; U.S. based as well as field operations: When FOCAS began their partnership, neither of the two indigenous NGOs, nor FOCAS itself, had experience with community based primary health care programs. Now there is capacity on the part of the NGOs to design, implement, and evaluate their effective child survival programs. Team participants explained that prior to the CS program, there was little comprehension of “medicine without walls.” Health activities, almost all curative, were related to a health center/clinic. Often cases came too late to be helped. Now communities with their CHAs are taking responsibility for much of the preventive and some curative health care, with good results, with far more equity in the system. Currently, CHAs active across the catchment area can and do offer preventive services and health education. Preventive services such as immunization and ORT training and packets of salts for ORT are available in the catchment area even at its periphery. CHAs also initiate treatment and refer cases of respiratory infection and diarrhea, two of the major killers of children. · Effects of this grant in influencing other programs operated by FOCAS: The institutional capacity of FOCAS has benefited in many ways as a result of its participation in the CSP grant. Fiscal accountability and general bookkeeping procedures have improved. Quickbooks software has 51 been installed and is utilized at both headquarters and field offices. Daily program expenditures are documented by detailed accounts and utilized for future budget projections. Annual audits of the organization have been successful and FOCAS is now a member of the Evangelical Council for Financial Accountability. FOCAS has also increased its Board development activities. At the headquarters’ office, it has expanded and increased committee participation. At the field office, a functioning Board of Directors is now active and meeting regularly as an advisory body. Finally, an agency-wide assessment was completed as a direct result of CSP grant capacity-building requirements. The tool used for this self￾assessment was the McKinsey Capacity Assessment. As part of this effort, FOCAS has reviewed, revised, and established new vision and mission statements. The organization has defined the Critical Success Factors that are essential to the future direction of program initiatives in both its Cincinnati, Ohio and Haiti programs. (ii) Strengthening local partner organizations Organizational capacity with local partners: both OBDC and MEI were greatly helped through multiple trainings by FOCAS staff from USA and by Curamericas, the organization that mentored them (see Annex 1). FOCAS has established a field office in Haiti with administrative, secretarial, and accountant services. Communications through telephone services are possible with outlying field staff. Repeated trainings in management have occurred through Curamericas; they sent trainers in management and supervision and supervisory instruments. USA/FOCAS visits have enriched the program greatly with tools that lead to accountability. MEI and OBDC are running a successful child survival program. Coverage for key child survival services such as immunization, Vit. A distribution, and growth monitoring are good. (iii) Outcomes of assessments conducted at the outset of the program and at midterm to determine the organizational capacity of the partners: FOCAS had a long association with MEI and its pastoral leadership in Haiti. In addition to church planting, MEI had successfully established and conducted primary schools across its impact area. The program had already maintained a health center, and owned many buildings and tracts of land for expansion. FOCAS assisted the leadership in many ways, while observing its capacity to carry out various services. FOCAS founder Dick Taylor made many trips to Haiti as follow-up to see that the program kept its word and maintained financial accountability. The MEI management structure is 52 flexible; there is a church council at the top with various committees, and the council oversees the operation of the health program. Mr. Taylor determined that OBDC was a reliable partner through his own investigation, assisted by Haitian staff that were brought onto the program early on. At FOCAS’ request, the midterm evaluators did a management assessment using MOST (Management and Organizational Sustainability Tool) in four major management functions (mission, strategy, structure and systems). Their findings were that, for MEI, “the presence of a constant leader over the years has given the MEI council and MEI a measure of stability, guidance, and program consistency.” Overall, they found that the MEI management structure is appropriately flexible. They found that operational planning is carried out by staff of the community health program, so that the community health staff is well aware of its tasks as well as of planned activities. The midterm evaluators found that OBDC was a logical partner (MEI land abuts the land of this indigenous NGO). The two programs had previously been willing to join hands for certain activities, such as participating in immunization campaigns. OBDC has a lay board of directors and, although not a faith-based organization, its leadership is philanthropic and has adequate commitment to outreach and development efforts. The OBDC program director is a business woman with training and experience in accounting and management, and is well-known to the business community in Haiti (those interviewed trusted her). The program director seems to enjoy more authority and responsibility than her counterpart at MEI, but the two coordinate very well together. Recommendations from the midterm evaluators included getting a firm mission statement from both institutions and from FOCAS, and adopting organizational strategies in conformity with the mission. It was unclear to the evaluation team whether all the recommendations of the midterm team had been met. Constraints included rapid turnover of staff at FOCAS’ headquarters and field offices. (iv) Organizational capacities of the local partners that have changed since the beginning; interventions that most contributed: · Curamericas, in its mentoring role, has contributed greatly to the organizational capacity (see trainings in Annex 6). · Headquarters staff at FOCAS have played a key role, and have built up the organizational capacity despite the fact that they have experienced staff turnover. The program has suffered to some degree because of local staff turnover. For example, Dr. Ferrus, a well-trained Haitian public health physician, left the program about three years ago, and his replacement, Dr. Baguidy, now doing an excellent job, had to take over his responsibilities 53 but had not had the opportunity to benefit from all the training Dr. Ferrus had. · FOCAS staff, MEI, and OBDC have benefited from many trainings by the MSPP, in cooperation with other organizations such as AOPS. (v) Lessons learned in capacity building of the local partners: · Technical assistance, especially for management and capacity building, should be emphasized from the beginning. MEI and OBDC both still needed extensive technical assistance at midterm, when evaluators found them still at “Level 1” (MOST). · Establishing liaisons as early as possible with other PVOs, NGOs, and government facilities active in the geographic area or nearby is crucial to program capacity building. Cross-fertilization and sharing of “lessons learned” and the ability to refer patients appropriately enables programs such as this one to avoid problems, and to succeed. Example: The capacity to detect early childhood tuberculosis is enhanced by physicians’ review of growth monitoring instruments and finding children with prolonged growth faltering. Physicians then need to be able to refer children to an institution such as Grace Childrens’ (TB) Hospital in Delmas (not far away). To do so means that Grace Children’s Hospital must have had an agreement that children referred from FOCAS partners will in fact immediately have appropriate exams to rule out TB. · Capacity to overcome distance and geographic dispersion with organizational skills demands imagination, determination, motivation, and discipline on the part of the staff. Staff recognition for a “job well done” is therefore important. 2. Health facility strengthening: · Effectiveness of the approach for improved management and services: Health centers under program direction have apparently become better equipped and staffed. CHAs have been encouraged to refer patients appropriately and patients are more willing to go to centers even though they pay a low cost fee. · Health facilities and their services were strengthened by the CARE-MoRR (Management of Reproductive Risks) project, with which the program collaborated, and which offered trainings where CHAs develop new skills. · New CHA skills (such as counting respirations in an ill child to detect or suspect pneumonia) need to be recognized and followed-up by the medical professionals in the program; a CHA who refers a patient for pneumonia deserves to be considered by the physician as “part of the team”! 54 · Although diarrhea remains a big problem and a killer of children, no health facility in the program has a “rehydration corner” where mothers under supervision can begin to rehydrate their children; this is needed. · Doctors in the program complain they need otoscopes, better stethoscopes, and the like for examining and treating pediatric patients. · There are stronger linkages between the facilities and their beneficiaries, largely due to CHAs acting as liaisons. 3. Tools the program used; effectiveness in measuring change: · The program used an end-of-project survey following a well-done baseline and midterm evaluation study. The ability to compare “before” and “after” rates of coverage for immunization, contraceptive acceptance in mothers of under-twos and the like is useful to program planners. · Curamericas assisted the program to develop many supervisory tools and quality of care assessment tools. An example follows: Checklist for Clinical/Hospital/Health Center Management of Severe Malnutrition History YES NO Was the parent asked about . . . 1. ...the child’s usual diet before the current episode of illness? .....................................................................................................................o o 2. ...the child’s breast-feeding history? ........................................................................................................................................................o o 3. ...the food and fluids taken during in the past few days? ............................................................................................................................o o 4. ...recent sinking of eyes? .......................................................................................................................................................................o o 5. ...the duration and frequency of vomiting, and its appearance? ...................................................................................................................o o 6. ...the duration and frequency of diarrhea, and its appearance? ....................................................................................................................o o 7. ...when urine was last passed? ................................................................................................................................................................o o 8. ...any recent contact between the child and persons with measles or TB? ....................................................................................................o o 9. ...any deaths of siblings? ........................................................................................................................................................................o o 10. ...birth weight? .....................................................................................................................................................................................o o 11. ...milestones the child has reached (sitting up, standing, etc.) .....................................................................................................................o o 12. ...immunizations that the child has received? ...........................................................................................................................................o o Examination 13. Was the child weighed properly? (fill in details) ......................................................................................................................................o o 14. Was the child measured properly? (fill in details) .....................................................................................................................................o o 15. Was the child’s weight for height (or length) calculated? ..........................................................................................................................o o 16. Was the child’s height (or length) for age calculated? ...............................................................................................................................o o 17. Was the child assessed for edema? .........................................................................................................................................................o o Was the child assessed for . . . 18. ...enlargement or tenderness of the liver? .................................................................................................................................................o o 19. ...jaundice? ..........................................................................................................................................................................................o o 20. ...abdominal distension? ........................................................................................................................................................................o o 21. ...normal bowel sounds? ........................................................................................................................................................................o o 22. ... “abdominal splash” (a splashing sound in the abdomen)? ......................................................................................................................o o 23. ...signs of circulatory collapse: cold hands and feet? ................................................................................................................................o o 24. ...signs of circulatory collapse: weak radial pulse? ...................................................................................................................................o o 25. ...signs of circulatory collapse: diminished consciousness? .......................................................................................................................o o 26. ... temperature (fever or hypothermia)? ...................................................................................................................................................o o 27. ... thirst? ..............................................................................................................................................................................................o o 28. ... eyes: corneal lesions indicative of VAD? ............................................................................................................................................o o 29. ... ears, mouth, throat: evidence of infection? ..........................................................................................................................................o o 30. ... skin: evidence of infection or purpura? ................................................................................................................................................o o 31. ... respiratory rate and type of respiration: signs of pneumonia or heart failure? ..........................................................................................o o 32. ... appearance of feces? ..........................................................................................................................................................................o o 55 If the child was very ill: 33. If clinical specimens were taken, did the staff avoid frequent handling by having the child remain in bed during the collection of specimens? .......................................................................................................................o o 34. If x-rays were needed, did the staff postpone taking them until the child was in better health? ......................................................................o o Decision on Treatment / Treatment: 35. If the child had a weight for height below –3 SD, less than 70% of the median, or bipedal edema, was the child admitted to the hospital or another location where the child could be observed, treated, and fed day and night? ......................................................................................................................o o 36. Did the health worker avoid giving the child iron supplements during the first two weeks of rehabilitation? ...................................................o o 37. Did the health worker use a proper oral rehydration solution (e.g., ReSoMol) if the child needed ORS ? ........................................................o o 38. Did the health worker avoid giving diuretics to treat edema? .....................................................................................................................o o 39. Was high dose vitamin A given to the child? ...........................................................................................................................................o o 40. Did the health worker avoid giving intravenous albumin and amino acids? .................................................................................................o o 41. Was a broad spectrum antibiotic given to the child? .................................................................................................................................o o 42. Was food intake monitored during treatment? ..........................................................................................................................................o o 43. Was the child fed at night during treatment? ............................................................................................................................................o o 44. If hypothermia was a risk, were blankets provided for the child? ................................................................................................................o o If the child had stunting only, but did not meet the weight for height criteria for severe wasting or edematous malnutrition (see #6): 45. Was the child managed in the community? ..............................................................................................................................................o o 46. Was the diet suggested to the parent one in which the child would receive at least 110kcal/kg per day, and sufficient vitamins and minerals to support cont. growth? ........................................................................................................................................................................o o 47. If the child has not yet been weaned, was the mother told to continue breast-feeding? .....................................................................................................................................................................................o o 48. Was the parent told to add 1-2 tsp. Of oil to each (100g) serving of food given to the child? .............................................................................................................................................................................................o o 49. Were the parents told to feed the child at least five times daily? .................................................................................................................o o 50. Was the parent told to give the child extra foods between meals? ...............................................................................................................o o 51. If the child needed vaccines, was the child immunized? ............................................................................................................................o o 52. If the child received vaccines, was the mother told where and when to bring the child for any additional doses of vaccines that the child requires? ..............................................................................................................o o 53. Was the child given a 1 week follow-up appointment (preferably at a specialty facility)? .............................................................................o o 54. If possible, did the staff member arrange for a CHW to visit the child in their home to provide practical advice on health and nutrition? ..................................................................................................................................o o 55. Was the child fed under observation using this new feeding pattern before discharge? .................................................................................o o 56. Did a staff member assure that the parent was able and willing to look after the child? .................................................................................o o 57. Did a staff member assure that the parent understands how to prepare appropriate foods and feed the child? ......................................................................................................................................................o o 58. Did a staff member assure that the parent knows how to make appropriate toys and play with the child? .........................................................................................................................................................................o o 59. Did a staff member assure that the parent knows how to give home treatment for diarrhea, fever, and acute respiratory infections? ..................o o 60. Did a staff member assure that the parent knows how to recognize the signs that mean s/he must seek medical assistance? ..............................o o 4. Integrated care; linkages between facilities and communities: · Through church-related activities, MEI has strong links to communities. Through community activities (water source improvement, etc.) OBDC has strong links. The program needs strengthening in community mobilization (mentioned above in relation to low levels of tetanus immunization in women). · In this program, CHAs’ work (mostly preventive) is in tandem with the curative services. The program has had from the outset one health center staffed by auxiliaries, a reasonably stocked pharmacy, and some limited instruments. New dispensary/health centers have recently been added. A physician hired by the program is present for a part of most days in the major dispensary, and for two days/week in the new, smaller center. He also participates in the ongoing training and supervision of CHAs, reviewing their records including their verbal autopsy reports on each death and is ready to use these for decision making regarding the program. Thus, the physician or 56 auxiliary seeing a patient for an illness in the health center is able to refer the patient back to his/her community health worker for follow-up. 5. Strengthening health worker performance: · The trainings and activities of the Curamericas trainers were geared to address these issues. See the mentoring partnership report in Annex 1. · Performance objectives were created to achieve program objectives and in general were reached. CHAs were expected to conduct and report on monthly rally posts; their achievements were checked out by supervisors. Curamericas trainers created appropriate forms to trace and to address these issues. · Lessons learned: At the clinic level, integrated care is important, but clinical treatment and quality of care in the clinics could lag behind other program achievements unless there is ongoing medical education/review and better equipment for doctors. At the community level, the community outreach system is the backbone of the program; the CHA network has made it possible for the program to achieve coverage, but CHAs must be carefully and constantly supervised. · Plans for sustaining health worker performance: LQAS, as used by the midtermand final evaluators, provides useful information as to which health worker areas were under-performing. The program can use this information to target those in need of retraining. 6. Were tools to assess results sensitive enough to measure change over the life of the program? The program had 1) ongoing monitoring data (e.g., see the mortality and nutrition reports by Tom Davis in Annexes 2 and 3); and 2) the program benefited from assessments and surveys carried out at the beginning, midterm, and EOP. The questions appeared sensitive enough to track progress and measure change. 7. Did the program have gaps between performance standards and actual performance? This was rare, but there may be some instances of this in regard to the performance of the mothers and other family members as a result of a shortage of opportunities to practice behaviors. For example, to combat diarrhea, there is a lack of “training by doing” (mothers are told about ORT, but are not mixing up the solution at the training sessions). The same is true for the nutrition interventions that ended up with little if any training through application (e.g., preparation of nutritious meals and feeding children at the club meetings). 57 E. Training This subject is well-covered in the mentoring report completed by Julie Mobley in 2000 (see this report in Annex 1). 1. Staff Training: Resources were adequate to accomplish what appears to be a change in knowledge, skills and competencies of the program and staff (see Annex 6 for sample training schedules). Training that has occurred over the life of the program includes: training in accounting and fiscal responsibility was provided by the FOCAS Financial Director to the accounting team members at FOCAS Haiti, MEI, and OBDC; physicians and administrative staff received ongoing training through Curamericas, MSPP, and other partners, such as a recent training provided by MSPP on the methods for AIDS awareness, treatment, and prevention. The present Haitian physician/project manager at FOCAS Haiti, Dr. Micheline Baguidy, and the program manager at OBDC, Ms. Dilia David, are among the best this evaluator has seen in terms of competencies. Strengthening is needed only in the area of use of incoming data for changing program direction (as in the case of death reports, recently documenting some cases of tetanus of the newborn). 2. Team evaluation of training results: · Effectiveness of training strategy: excellent · Training objectives: clear and well met · Evidence suggests that the training program resulted in new ways of doing things, as well as increased knowledge and skills: the EOP survey showed change in many behaviors; the use of modern contraception has been adopted very rapidly for example. · Lessons learned: Having a contract with an outside trainer such as Curamericas has resulted in well-trained personnel. · Plans for sustaining training activities: the program is seeking external funding through grants for this; the program will also take advantage of trainings offered by UNICEF, PAHO, and the MSPP. · Are the sustainability plans realistic? Yes, provided external funding is found to supplement partner-provided training opportunities. F. Program Management The inclusiveness incorporated in program planning has had a positive effect on the implementation process. Annual plans are worked out with all three partners participating. There have been many visits to the Haiti field site and technical consultants to help with planning. The DIP had some unrealistic goals, but these were corrected at midterm. In Annex 4 a table with preliminary results shows how many of the objectives had to be modified. After modification, most were realistic. Gaps in the DIP included lack of a realistic objective in 58 nutrition; one was added at midterm. Similarly, the indicator adoption of modern contraception was refined. Lack of tetanus immunization coverage for women may be attributed in part to the lack of an objective for all women, rather than only for pregnant women (the norm is for all women to get immunized). Coverage expansion efforts in this area are just now being addressed by the program. 1. Supervision of Program Staff: The supervisory system is adequate and appears to be fully institutionalized. Evidence that the program’s supervisory system approach is both efficient and effective may be found in that the CEO of the Albert Schweitzer Hospital (HAS) recently expressed interest in the approach and its possible use at HAS. 2. Human Resource Policies and Services: Essential personnel policies and job descriptions need to be revised. A personnel handbook has been developed and is in use. Basic staff functions and responsibilities have also been documented in accordance with specific job titles or positions. However, these materials need to be expanded and updated. This could help in further institutionalizing the system. Morale, cohesion, and working relationships in the program appear to be excellent. The impact of this can be seen at this juncture when some staff are continuing despite the fact that they must temporarily accept only half-time salaries. The level of staff turnover may have had a negative impact on the program. While turnover at the worker level has been virtually nonexistent, this has not been the case at administrative levels. FOCAS has had three different persons in charge (i.e., CSP Directors) at headquarters, all with differing qualifications, over the life of the program. At FOCAS Haiti there have been two CS project managers in six years. At MEI and OBDC, the physicians, especially, have had a high turnover rate insofar as working in the clinic is concerned. Constant ongoing reorientation and education are necessary in this program. Plans to facilitate staff transition to other paying jobs need strengthening. This should be addressed in any future phase-over plans. 3. Financial Management · Adequacy of the PVOs’ and Partners’ financial management and accountability and budgeting: An independent audit of FOCAS’ financial position is completed annually. The audit report dated June 16, 2003 documents that the “financial position of FOCAS, as of Dec., 2001, and as of Dec., 2002…[is] in conformity with accounting principles generally accepted in the United States.” Further, FOCAS has been accepted as a member of the Evangelical Council for Financial Accountability (EFCA), having “met all standards of financial integrity and Christian ethics.” 59 Budgeting skills of the local program have improved; they submit monthly reports to headquarters in Haiti and to FOCAS/USA. These accounts report, by budgeted line item, for all grant expenditures. The accountants at MEI and at OBDC have proven themselves responsible for maintaining copies, journal records, and documentation of all receipts, contracts, and agreements related to program expenditures. The Financial Manager at FOCAS/Haiti was offered special training in the use of Quickbooks, as well as on general accounting and bookkeeping principals. He reports monthly on all OBDC and MEI-related activities as well as on the operations of the FOCAS Haiti office, at 41 Rue Borno in Petion￾Ville. · Are resources adequate to finance operations and activities that are intended to be sustained beyond this cooperative agreement? At the time of this writing, resources are inadequate to finance operations that are intended to be to be sustained. This is in part because the staff counted on the program funding being extended by USAID for the coming fiscal year; nevertheless, they have written and submitted other grant requests. The news that nothing was centrally funded by USAID/CS in Haiti this year sent the staff into action. Staff in both USA and Haiti have agreed to cutbacks in salaries until, it is hoped, the program will hear from some of the many grants that have been submitted (at least six of them). The local USAID mission in Haiti is impressed with this program and have promised to do all in their power to help find other funds. · Was there sufficient outside technical assistance available to assist the grantee to develop financial plans for sustainability? The local team in Haiti seemed satisfied with the outside technical assistance, but opined that there were so many grants that had been written perhaps “too late” to salvage the situation for the next few months. It is a tribute to the staff that everyone, down to the health agents, have agreed to salary cuts in order to “carry over” the program until more funding becomes available. 4. Logistics: · Impact of logistics on the implementation of the program: Through no fault of the program, logistics have had an unanticipated negative impact. The program was assured by representatives of the Ministry of Health of Haiti (MSPP) that they could count on government assistance for cold chain equipment, vaccines, immunization cards, and other supplies, but this proved not to be the case. WHO/PAHO assured the team (9/03) that there would be no “stock-outs” in the future, as long as the district level offices asked for adequate supplies. When program members reported the above to the local USAID mission, they were told that the “logistics” problem for vaccine, Vitamin A, and other supplies was pervasive. One NGO 60 serving an urban slum has been unable to get condoms for more than a year, although they have made repeated requests. Another problem with logistics is that it is difficult for all NGOs in Haiti to get medications and equipment out of customs, and FOCAS is no exception. FOCAS is a member of AOPS, an organization that is working on this problem. · Is the logistics system sufficiently strong to support operations and activities that are intended to be sustained? For most functions, the logistics system seems adequate. This is due in part to the fact that many staff members provide their own transportation. More transportation will be needed as the program expands. 5. Information Mangement: · Effectiveness in measuring progress toward program objectives: The information system in this program is excellent at every level: population census, registries of children under five and women of child-bearing age, monthly reports on rally posts, supervisory forms, etc. Midterm evaluators reported, “ the system provides reliable information on monthly volume of services being provided.” In addition, the system also provides adequate financial reporting. · Was there a systematic way of collecting, reporting, and using the data? The program staff appears sufficiently skilled to continue collecting program information and to use it for revisions and strengthening of program operations. An example follows on a study of pneumonia deaths in the program area (guided by leadership from Curamericas, January 2003). A review of verbal autopsy data indicated: ÿ In the OBDC area, there were six pneumonia child deaths, two of which were in the 0-7 day period, one in the 8-29 day period, one in the 1-11m period, and 2 in the 12-59m period. ÿ In the MEI area, there were four pneumonia child deaths, two in the 1-11m period, and 2 in the 12-59m period. ÿ In general, the children who died were not malnourished. ÿ No relationship to gender. ÿ Deaths do not seem to be related to CHAs who are unauthorized. (We did a 2x2 table to look at deaths by authorized and unauthorized CHAs, and the Odds Ratio was 1.0 – no relationship.) (N.B.:CHAs authorized are those qualified to detect and begin treatment of suspected pneumonia with Clortrimoxazole.) ÿ Pneumonia deaths may be related to transportation issues – pneumonia deaths were slightly more common in marginal than rural areas (6 vs. 4), 61 and two of the mothers said that they did not take their child to the CHA or health facility during the illness preceding the child’s death. Only 60% went to a health facility. ÿ Community Health Agents only saw 6 of the 10 children who died during the illness preceding the child’s death. A review of 58 ARI Case Management forms showed that: ÿ Only 19% of children with pneumonia, severe pneumonia, or severe disease were followed up within two days. Over half (52%) either had follow-up at one month or later, or had not follow-up documented on the form. ÿ 43% of children (3 of 7) who should have been referred during the initial contact were not referred. ÿ 33% of children (2 of 6) who were the same or worse at follow-up (according to what the agent marked on the form) were not referred. ÿ 40% of children whose respiration rate had increased were not referred. Specific recommendations were developed and applied to program operations upon completion of the study. See Annex 2 for the full report. Additional examples of special assessments: 1) The midterm evaluators used an LQAS approach to discover whether or not there were health worker areas that were underserved. The information was used for “retraining” CHAs in areas where underservice was reported. 2) A number of partners such as CARE-MoRR have used focus groups to discover whether or not there has been any project-related impact on behaviors that would protect the lives of mothers. 3) Under the leadership of Judy Gillens, FOCAS partnered with the University of Miami to assess water quality throughout the impact areas. · To what extent did the program strengthen other data collection systems? When visited by the evaluation team, the MSPP district office and the WHO/PAHO experts reported how grateful they were to the program for providing them with data. Thus, there is indirect evidence that the program has shared information. · The program staff seem to have a clear understanding of what the program has achieved: The staff is proud of the levels of immunization coverage achieved through the program; they are also aware of their achievements in growth monitoring/counseling (GMC). In focus groups, mothers expressed genuine approval of the family planning methods, and were aware that many of their friends had accepted appropriate methods. · The program’s monitoring and impact data have been used beyond the child survival program in the following ways: Microenterprise programs newly available to program beneficiaries have relied on program data to target mothers in need. Further, the program has shared data on immunization 62 coverage with government authorities and fills out and sends in all forms required by the MSPP. 6. Management Lessons Learned: · An outside technical training team is needed as soon as possible in the program cycle. · “Everyone deserves supervision” (Dr. Rex Fendall, 1974, Liverpool School of Tropical Medicine and Hygiene). Frequent formative supervision from headquarters staff is crucial; similarly, formative supervision of field staff has been a key to success. · Training in how to keep accounts and maintain fiscal accountability is needed and appreciated by partner indigenous NGOs. · Communication and transport are essential to continuing good management practices. G. Technical and Administrative Support · As mentioned previously in this document, the mentoring of the program by Curamericas has been appropriate, timely, and appreciated. Among others, external technical assistance has come from CARE (the CARE-MoRR project) on reproductive health and family planning, Save the Children (nutrition), World Relief (microenterprise), and the Ministry of Health (AIDS and other topics). · It seems that most technical assistance that was needed has been provided. Program staff expressed the need for even more training in, for example, nutrition, and in the role of parasites and appropriate deparasitization. · Quality of care in the clinics visited remains an issue (see below) · Staff turnover and continued training or retraining of new staff members remains a problem. For example, each time a new physician joins the program, he or she requires additional training in community-based primary health care and their role when malnourished children are referred to them. Other key issues include: interaction with CHAs on death reports/verbal autopsies, appropriate use of antibiotics, role of parasites and their prevention and treatment, appropriate protocols for treating severely malnourished children, and the causes and modern treatments to cure duodenal ulcers (often caused by Helicobacter pylori, a common complaint among adults). Further, doctors lack some essential equipment such as otoscopes and training in their use, yet otitis media is a common illness. · Grannie midwives are trained by the program but are not consistently equipped, and how they are supervised is unclear. (The program has clean cord cut kits in customs that have been held there for more than a year.) · Death reports and cause-of-death reporting (use of verbal autopsy) should be reviewed by administrative staff and physicians on a quarterly basis; these data should be used to modify priorities (e.g., when tetanus of the newborn reappears as a cause of death, the 63 program could make tetanus immunization of women, especially new immigrant women, a priority). · FOCAS headquarters staff, as well as Curamericas, have given a great deal of support and mangement training to this program. There were, on average, at least four trainings per year. In addition, their mentor, Curamericas, made many trips to Haiti for trainings (see Annex 6). H. Other Issues Identified by the Team · Improved quality of care in the clinics is needed, with review or refresher sessions for doctors new to the program. · Attention to sustainability issues early in the program is essential. For example, MEI and OBDC could benefit from USA church sponsorship of some staff members, thus guaranteeing some ongoing salaries. Local merchants or industries might contribute, since they hire from the program area. Such a staff sponsorship effort has already been launched by FOCAS, and several of the staff have obtained annual sponsors. · Transportation to program sites for supervision demands vehicles; one is not enough for a program that now covers over 100,000 persons. · Frequent stock-outs at the MSPP district level should be anticipated and an emergency supply alternative should be identified; budgeting should have a special fund for such emergencies. · Water and sanitation programs are needed to accompany the program’s efforts to combat diarrhea. I. Conclusions and Recommendations · Most objectives were met, and this program is a success in spite of enormous constraints. · FOCAS has developed the administrative capacity to implement, monitor, and maintain accountability for a successful child survival program in partnership with two indigenous NGOs that have also been strengthened. · This census-based program has given attention to equity; it makes sure that, to the greatest extent possible, no child (or mother) is left out; this a major accomplishment given the mobility of the marginal urban populations served. · The program has maintained a reasonably good level of immunization coverage and Vitamin A distribution despite having to cope with frequent stock-outs from the MOH supplier 64 · Recommendations for USAID/GH/CSHGP, the program staff, and collaborating partners regarding future work: ÿ Continue to fund or seek funding for mentoring organizations in programs like this one that have little previous experience with field operations in child survival activities. ÿ Continue to insist on involvement with local USAID staff; their awareness of new funding sources and programs that can benefit CS programs is essential (for example new funding in AIDS prevention). ÿ Continue to fund ongoing training and retraining of local professional personnel; many PVOs and NGOs could be trained at once. ÿ Anticipate the lack of reliability of suppliers in-country, especially the MOH, and allow for emergency funds to fill in the gap. ÿ Transportation and communication needs will grow as the program expands; budgeting should allow for this. ÿ Revamp guidelines for evaluating programs in highly mobile marginal urban populations, as opposed to stable rural ones. Baseline, midterm, and EOP surveys will each involve interviews with “new” populations who have not yet had a chance to be impacted by the program. · General recommendations for the FOCAS team: Sustainability: ÿ Continue laudable efforts to find more funding both in US and Haiti (particularly business donors). Consider health insurance plans, but do not forget that the poorest will not be able to pay for these. ÿ Given the extreme level of poverty, seek partnership with institutions that can help with dry ration distribution (contact Food for the Poor). ÿ Have a workshop on strategic planning in Haiti with sustainability in mind. Maternal Health (some taken from Tom Davis’ recommendations): ÿ Identify, train, and equip more TBAs. Have the birth report form name the TBA in attendance, use this to identify the most active ones. ÿ Assure that all mothers know the EOC danger signs. (TBAs have now had training on these.) Develop flipcharts for this purpose. ÿ Make changes to the maternal verbal autopsy form to capture more of the delays (delays in recognizing need for care, in making a decision to seek care, in getting to a health facility, etc.). Include a narrative section near the beginning where the Supervisor can document the flow of events in detail (including times) of the case. Teach Supervisors to take good histories on this. ÿ Have CHAs do quarterly external checks of the mortality data to see if local leaders know of any other maternal deaths not included in the Vital Events Register. ÿ Do verbal autopsies on all deaths where the cause of death is listed by the Health Agent has “not known.” Program physicians should participate. 65 ÿ Make a change to the Vital Events Register: Add a column to easily identify women who have died from causes related to pregnancy and delivery (during or before 45 days following delivery). Diarrhea: ÿ Add a rehydration corner with a cot for the child and a chair for the mother in all health centers. Make sure staff know how to supervise these cases, and have ORS salts available. Have IVs in reserve if ORT doesn’t work; have physicians trained in their use. ÿ Begin doing one-day follow-up of children with diarrhea with signs of dehydration, severe disease, diarrhea with vomiting, and all children referred. ÿ Develop a form to document case management, referral, and follow-up of diarrheal cases in the same way that it is done for ARI cases. Make it a register￾type form where more than one case can be put on each sheet of paper. ÿ Refer all children who have diarrhea and vomiting if the child is unable to keep any liquids down. ÿ Explain to all mothers how to give ORS if a child vomits it up (waiting 15 minutes and then giving the ORS more slowly with a spoon). ÿ If a child has had vomiting recently, stay and observe the mother rehydrating the child to see if the child can drink the ORS and keep it down. ÿ Improving the water sources and purification of water would probably help decrease diarrheal incidence. Pneumonia: ÿ Have a campaign (community mobilization, including churches) so that there is widespread knowledge about qualified CHAs who can detect, treat, and refer patients with signs of serious respiratory infection. ÿ Make changes to the Pneumonia Case Management form. Remove elements that are not analyzed. Add to the form if the mother was educated on pneumonia prior to the child’s illness (using the Child Register to find that data). Make this form a register-type form where more than one case can be put on each sheet of paper. Specific changes that should be made include: - Add a spot for showing vitamin A treatment. - Add yes/no boxes for documentation of danger signs. (Currently, leaving a box blank means that the child does not have a danger sign; but, when a box is left blank, you cannot tell if the child did not have a sign or if the CHA simply forgot to fill out that part of the form.) - Put the respiration rate cutoffs for rapid breathing under the age categories as a reminder to the CHAs. - Change “last date of Vitamin A” to “Dose of Vitamin A in last four months?” - Add “or vomits everything” to the “cannot drink at all” danger sign. - Add stock-outs to the Program Manager Monthly Report so that stock￾outs on antibiotics can be dealt with in a timely manner. (Done) 66 ÿ Develop a system for preventing stock-outs of cotrimoxazole. Consider giving CHAs additional stocks of antibiotics each month so that they will have those available for indigent families. Have them keep track of families who receive free medicines so that Supervisors can do spot-checks to assure that this system is not abused. · Other general recommendations for prevention of child deaths: ÿ Develop a system for doing follow-up of children who go to clinics/hospitals outside of the OBDC/MEI system. ÿ Make changes to the child verbal autopsy form to capture more of the delays (delays in recognizing need for care, in making decision to seek care, in getting to a health facility, etc.) Include a narrative section near the beginning where the Supervisor can document the flow of events in detail (including times) of the case. Teach Supervisors to take good histories on this. ÿ Have CHAs do quarterly external checks of the mortality data to see if local leaders know of any other child deaths not included in the Vital Events Register. ÿ Do verbal autopsies on all deaths where the cause of death is listed by the Health Agent has “not known.” ÿ Physicians on the team should have a monthly meeting to review all death and verbal autopsy forms and make decisions about whether the death was preventable, and how it could have been prevented. FOCAS, MEI, and OBDC will use the lessons learned and apply the recommendations to improve future program operations. FOCAS maintains an ethic of continuous quality improvement, and such changes are an ongoing process in the management of this program. For example, a regular supply of milk formula has already been negotiated with Food for the Poor for the care and treatment of malnourished children. Concerning the tetanus immunization issue, FOCAS has already changed its coverage policy to include all women of child-bearing age. FOCAS is also committed to sharing lessons learned and best practices with the broader service community. This is being pursued through such avenues as case study publications and response to such opportunities as the Diffusion of Innovations RFA extended by CORE. 67 J. Results Highlights Overall, the program measurably improved infant, child, maternal health and nutrition, and contributed to the reduction of corresponding mortality rates and unintended pregnancies. Key issues include learning how to deal with and adjust for a highly mobile marginal urban population (N.B.: Beginning program survey deals with an almost entirely different population than those who will be present at end-of-project). Others are: · Coping with MSPP stock-outs of vaccines, Vitamin A, and other instruments; · Increasing collaboration with the District Ministry of Health Office; · Facing sustainability uncertainties in the face of a national economic crisis; · Learning how to cope with the fact that it is nearly impossible (one year or more) to get equipment and supplies to clear customs; · Planning for continuing education in the face of staff turnover; · Prioritizing efforts based on incoming data; for example, efforts to reach all women instead of pregnant women with tetanus immunization. Quantifiable results: · Ongoing birth and death reporting pays off: Death rates have declined in under-fives (see Annex 2). This was a great morale booster to program participants. · Ongoing monitoring data shows apparent reduction in malnutrition; severe malnutrition dropped from 11% to 4-6%. · Some quantifiable and specific results: ÿ Behavioral change appears to have occurred; examples include: the use of fluids and oral rehydration therapy increased from 43% to 91% in the OBDC area and from 69% to 100% in the MEI area; use of colostrum increased from 57% to 73% (MEI) and from 52% to 70% (OBDC). ÿ Full immunization coverage rates for children 12-23 m are relatively high in the face of a highly mobile population and frequent stock-outs of vaccines (although not quite at target, OBDC increased from 17% to 70%; MEI from 51% to 67%). ÿ Negative result (one of the few) is low tetanus immunization coverage for women: Of the more than 30,000 women aged 15-45 served by the program, only 31% of them are immunized in the MEI area and only 26% in the OBDC area. Most of this is due to stock-outs, persistent in Haiti. ÿ Use of modern contraception for women wanting no more pregnancies has increased from 13 % to 57% (OBDC) and from 15% to 50% (MEI). New methodologies: · Integrated care: The program facilities and referral system facilitate integrated care in a way rarely found in Haiti. · Use of an ongoing monitoring system, especially death and cause-of-death reporting in children and women 15-45 years of age to guide policy. · Use of a mentoring partnership to provide training, set standards, and help innovate the inception of the program. 68 New policies enacted: · New methods of accounting were adopted and personnel trained; monthly and fiscal accountability were implemented. · Transition from curative care alone (prior to CSP) to preventive care in tandem with curative care will allow for integrated maternal and child care. · New supervisory norms and standards have been introduced by Curamericas and accepted. · Review of CHA vital event reporting and verbal autopsies led to policy changes and activities. New policies to be enacted: · Program will now include a plan for sustainability from the outset of any new program efforts. · A tetanus immunization campaign (with MSPP) will include all women in the reproductive age group. · The program will reconsider demonstration education efforts in nutrition and will seek partners toward that end. Overall program success: · Most indicators are at or above target (see table in Annex 4), and their determinants and consequences understood; for example, the disappearance of measles alone would be expected to have prevented many deaths from measles itself and from the concomitant malnutrition that almost always follows measles. Moreover, these services require no payment and reach the poorest of the poor near their own homes. · Key program innovation: CHA death and verbal autopsy reporting for under-fives and women in the reproductive age group is remarkable. This is one of the few child survival programs where there is an attempt to record all deaths and investigate the cause, and then use the assessment to innovate changes that save lives. This information helped the program decide to double its efforts to immunize women and to establish increased prevention, care, and supervision of childhood cases of pneumonia and diarrhea. 69 K. Evaluation Team and Survey Methodology 1. Team members and their titles: Gretchen Glode Berggren, M.D., M.Sc.Hyg. International health and nutrition consultant; Retired faculty member, Harvard School of Public Health Judith Gillens, M.S.W., Ph.D., Director of Haitian Development Programs, FOCAS Program director for the Child Survival Program (CSP) Adjunct Professor at Miami University, Oxford, Ohio Philip Moses, M.P.H., Public Health Specialist, FOCAS Tom Davis, M.P.H., Senior Program Specialist, Curamericas Micheline Baguidy, M.D., CSP Program Manager, FOCAS Haiti Dilia David, Program Manager, OBDC Keneth Vernet, M.D., Program Manager, MEI 2. Final evaluation methodology: · An end-of-project survey was applied and analyzed by Phil Moses. Comparisons were done with baseline and midterm findings. · Program documents were examined and interpreted. · Field visits were done to rally posts, clinics, and staff centers. · Home visits were carried out to the homes of some beneficiaries who were chosen at rally posts or who invited team members. · Results of special studies and surveys were examined. · Directors and administrators of the indigenous NGOs were interviewed. · PVO partners, such as World Relief, were interviewed. · Focus groups with CHAs and with program beneficiaries were carried out. · USAID, MSPP, and WHO/PAHO staff were visited and interviewed; debriefing was accomplished with the local USAID mission to get their input. 3. Final KPC survey methodology: FOCAS implemented a final survey in August of 2002 to obtain end of project information about the knowledge, the practices and the coverage that impact child health in the project areas served by MEI and OBDC . The survey covered neonatal and maternal care, nutrition and micronutrients, pneumonia, birth control, immunization and the control of diarrheic illnesses. The final questionaire drew heavily from the questionnaire used for the baseline KPC survey conducted in 1997. Some of the questions were adapted to reflect more current methods of 70 measuring specific indicators. Some new questions were added to reflect new non-project indicators that will serve as a baseline for future projects as well as to include RapidCATCH indicators developed by the Child Survival Technical Services (CSTS). FOCAS' FOCAS's KPC Core team consisted of Haiti Program Director (Dr. Judy Gillens), FOCAS Haiti's Child Survival Program Manager (Dr. Micheline Baguidy), FOCAS' Public Health Specialist (Phil Moses), OBDC's Project Manager (Ms. Dilia David) and MEI Project Manager (Dr. Keneth Vernier). The Public Health Specialist designed a preliminary draft of the questionnaire in English. The KPC Core team then made modifications to adapt it to the needs of the program. The questionnaire was translated into Creole. For convenience of handling, the questionnaire was divided into two sections. The first contained questions relevant to the health of the child, the second contained questions pertinent to maternal health. The survey covered the contractual indicators of the project as per the 1998 DIP and the 2000 Cost Extension agreement. In addition, the KPC Core team added a number of new indicators in order to establish a baseline for possible future intervention areas. The methodology of sample used for the baseline is Lot Quality Assurance Sampling (LQAS) , which is a type of a special stratified sample. The purpose of this type of sample is to be able to assess the coverage or the quality of health services, knowledge and practices in a particular area. LQAS allows the project to identify the areas which have levels of coverage that go beyond expectations in comparison to those which don’t reach the level of expectation. Using LQAS, it is possible to detect the differences in knowledge, practices and coverage between the different areas making it possible to determine the sections of the program areas which deserve attention due to poor performance/outcomes. For the final survey each project area was subdivided into Supervision Areas. The KPC Core team divided the OBDC area into seven supervision areas and the MEI area into four supervision areas. In the OBDC program area the sample size was 20 from each supervision area for a total sample size of 140 (20 children X 7 supervision areas = 140). In the MEI area the sample size was 25 from each supersion area (25 children X 4 supervision areas=100). The extra five mothers interviewed in each SA of the MEI area as to ensure that the sample size for the entire MEI area would be enough to ensure a confidence level of 95% for specific project indicators. Survey sites were determined within each SA using a sampling frame. OBDC’s sampling frame consisted of localities with a list of the total population based on OBDC’s census data (last updated in June 2003). MEI’s sampling frame consisted of sectors assigned to specific CHAs with a total population per sector. Care was taken to ensure that every mother in the project area had an equal probability of being selected. Staff used the child registries to randomly select children 0-23 months old whose mother would be interviewed. Children were selected alternately from the 0-11month and the 12 –23 month age groups in order to ensure a more even age distribution. For each SA in the MEI area a total of 31 children were selected. Six selected names were then randomly assigned as “reserve” names to be used any of the other 25 mothers were unavailable. For each SA in the OBDC area a total of 25 children were selected with five names randomly assigned to the “reserve”. The purpose of pre-selecting the reserve group was to reduce selection bias on the part of the interviewers in the event that the targeted mother was unavailable. 71 Classroom training was conducted for one day on July 31. Training was done in French and Haitian Creole with the PHS and Dr. Vernet acting as co-facilitators. All the participants understand French although most are more comfortable expressing themselves in Creole. (The PHS is fluent in French but not in Creole, Dr. Vernet is fluent in both). Training focused mainly on becoming familiar with the questionnaire, how to conduct the interview and to record data properly. The following day (August 1) interviewers and survey supervisors practiced in a nearby slum. Interviewers conducted several practice interviews each, filling out the questionnaire form. Survey Supervisors directly observed the practice interviews and filled out a quality improvement checklist for KPC surveys developed in Creole for the 1997 baseline survey by Tom Davis. Several modifications were made in the survey questionnaire based on problems uncovered during the practice day. Data was collected from August 4 – 8 and from August 11-12. The data collection process took longer than the originally planned four days due to a number of logistical and administrative issues. The PHS had originally planned on setting up and overseeing the data entry at the FOCAS office using Epi Info 2002 (a statistical package developed at the Centers for Disease Control and Prevention in Atlanta for the analysis of health survey data). However, due to the above-mentioned delays in data collection, unreliable power and a lack of reliable computers at the FOCAS office in Pétion-Ville, the KPC Core team hired Mr. Paul Bréa as a consultant to enter the data. Mr. Bréa’s team used and earlier version of Epi-Info (version 6) to enter the data since they were not familiar with the updated 2002 version. His data entry team used a system of quality control in which two individuals enter the same data and then the data sets are compared for discrepancies. Because of the above-mentioned delays the data entry could not be completed before the PHS had to leave the project area. The data set was sent by email to the PHS several days after his departure. There were some problems converting the dates from the version 6 format into the Epi-Info 2002 format. The Epi-Info help center at the CDC provided the courtesy of converting the dates into Epi-Info 2002 format thus making possible the analysis of the data. In some cases the conversion process gave garbled dates but these mistakes were easily identified and recoded. On September 1 the PHS sent a list of preliminary results on all the KPC indicators for both organizations the FOCAS Haiti staff and to Dr. Gretchen Berggren, the final evaluator for the project who was in the field at the time. Due to lack of experience with the Epi-Info 2002 software the PHS took longer than scheduled to complete data analysis. Colleagues Tom Davis and Julie Mobley donated much of their own time to give guidance on general analysis and on the use of Epi-Info 6 and 2002 versions. (Both Mr. Davis and Ms. Mobley worked on the baseline KPC for FOCAS in 1997. They co-authored the KPC 2000+ Draft Trainers Curriculum and Ms. Mobley is currently scheduled to conduct a training on Epi-Info 2002 in Nicaragua in November of 2003). Their help was invaluable to the data analysis and in many aspects of the KPC process. Limits of the KPC: The mobile nature of many families in the program area made measuring some indicators difficult. In one neighborhood, approximately 28% of the children initially selected for an interview had moved away from the area or were staying with relatives in the countryside at the 72 time of the survey. Many of the children selected for the KPC survey may not have lived in the area long enough to have benefited fully from program interventions. Other children who had benefited from these interventions may have been outside of the program area during the time of the survey. This is especially problematic when trying to measure the outcomes of interventions related to nutrition since such a child often has to have been enrolled in such an interventions over a fairly long period before showing improvement. Children living in urban slum areas were more likely to have moved or been absent than were children living in rural areas. The timing of the survey coincided with school vacations when many families take their children to live with relatives in the countryside where the cost of living is lower. (Although none of the targeted children were school age, many had accompanied older school aged siblings to the countryside). When selecting children for the KPC survey interviewers were not instructed to ask how long the child had lived in the program area. The anthropometry data from the KPC were not intended to be used directly to evaluate the program. Anthropometry was conducted rather to triangulate the program data generated from the ongoing growth monitoring conducted by the health agents and nutrition monitrices. Furthermore, there are no baselines for nutritional status against which to compare survey data. A recent analysis of program data showed an improvement of children classified as moderately or severely malnourished from 10.9% to 5.86% for MEI and from 10.5% to 3.78% for OBDC. It would have been impractical to try to determine such a change with the KPC survey since the required sample size would have been much too large. (To determine a change of 10.9% to 5.86% would have required a weighing approximately 784 children at random, while measuring a drop from 10.5% to 3.78% would require weighing about 306 children. Such large-scale surveys are well beyond the budget of this program). The Weight-for-Age Z scores from the survey show a higher rate of moderate to severe malnutrition than that shown by the program data. This may be due to the fact that many of the children weighed during the survey were very probably new to the area and therefore would not have been receiving services long enough to show improvement in nutritional status. 4. List of persons interviewed or contacted this evaluation: Haiti Ministry of Health Brignol Boulin, M.D. USAID Mission in Haiti Polly Dunford, Dr. Yves-Marie Bernard, John Burdick, and Alice Norton World Relief Haiti Dr. Hubert Morquette Save the Children/Haiti Marjolein Moreaux, Nutritionist MARCH/Mirebelais Mireille Henry, Nutritionist and Dr. Antoine Augustin 73 WHO/PAHO Miss Vernier, R.N., Hygienist (in charge of vaccine supplies) AOPS Dr. F. Jean-Louis, Mr. Ambroise Curamericas Tom Davis, mentor representative 74 ANNEX 1 – Mentoring Report Foundation of Compassionate American Samaritans (F O C A S) and Andean Rural Health Care (A R H C) A Report on the First Three Years of a Mentoring Partnership Prepared by: Julie Mobley, MSPH October, 2000 75 ACKNOWLEDGEMENTS The Foundation of Compassionate American Samaritans and Andean Rural Health Care would like to express their appreciation to the United States Agency for International Development, Bureau of Human Response / Office of Private and Voluntary Cooperation for making this report possible, through their Child Survival Mentoring Partnership Grant. gggggggggggggg The author would like to express appreciation to the following people who assisted in this report by agreeing to be interviewed, sharing all relevant documents, and assisting in the review of earlier drafts of this report. FOCAS: Richard Taylor, Founder and Executive Director Amy Metzger, Health Programs Director (Headquarters) Arsene Ferrus, Child Survival Project Manager (Field) ARHC: David Shanklin, Executive Director (and former International Program Director) Thomas P. Davis, Senior Program Specialist 76 Table of Contents Abbreviations / Acronyms I. Executive Summary…………………………………………………………………78 II. Background ……………………….……………………………………..…………80 A. Partner Organizations …………………………………………………………80 B. CS Mentoring Partnership Grant Submission Process …………………….81 C. Project Description……………………………………………………………82 III. The Mentoring Approach……………………………………………………………83 A. Philosophy and Objectives of the Mentoring Partnership ………………..83 B. Design / Approach of the Mentoring Process ……………………………….84 C. Mentoring Partnership Plan …………………………………………………87 IV. Project Progress and Achievements ……….……………………………………….89 A. Specific Achievements of the Mentoring Partnership ……………………..89 B. Progress since the Mid-Term Evaluation ……………………………………93 V. Strengths and Constraints of the CS Mentoring Approach ……………………..96 A. Strengths of the CS Mentoring Approach ..…………………………………..96 B. Constraints of the CS Mentoring Approach …………………………………97 VI. Summary Recommendations and Comments …………………………………100 A. Lessons Learned: Recommendations for Future Mentoring Partners …… 100 B. Points for Reflection and Further Discussion ………………………………103 Appendices A. Table of Mentoring Activities B. Training Schedule for Field Staff, Years 1-2 77 Abbreviations and Acronyms ARHC Andean Rural Health Care ARI Acute Respiratory Infections CS Child Survival CSPM Child Survival Project Manager BHR/PVC Bureau of Human Response / Office of Private and Voluntary Cooperation ED Executive Director FOCAS Foundation of Compassionate American Samaritans HA (Community) Health Agent HHF Haiti Health Foundation HIS Health Information System HPD Health Programs Director IEC Information, Education, Communication KPC Knowledge, Practice and Coverage (baseline survey) MEI Mission Evangelique International MOH Ministry of Health (Haiti) NGO Non – Governmental Organization OBDC Oeuvres de Bienfaisance et de Developpement Communautaire PD Program Director PHC Primary Health Care PM Program Manager PVO Private Voluntary Organization STS Senior Program Specialist TA Technical Assistance USAID United States Agency for International Development 78 I. Executive Summary This report documents the history, progress and accomplishments of the only Mentoring Partnership Child Survival Project to be awarded to date by the USAID Bureau of Humanitarian Response / Office of Private and Voluntary Cooperation. The report was written at the request of the two partnering private voluntary organizations (PVOs): Foundation for Compassionate American Samaritans (FOCAS) and Andean Rural Health Care (ARHC). It is not meant to be a comprehensive overview of the project’s activities, but rather documents those aspects which relate directly to the mentoring partnership and which distinguish this project from other “traditional” USAID-funded Child Survival projects. Background: FOCAS and ARHC formed a mentoring partnership which successfully competed for a Child Survival Project XIII grant in 1996, the first year that USAID BHR/PVC offered the Mentoring Partnerships category of grant awards within the Child Survival domain. FOCAS, the “Mentoree PVO,” had implemented health activities through a local Non￾Governmental Organization (NGO) in Haiti for over ten years, but had not yet competed successfully for a Child Survival grant before receiving a grant in this category. ARHC, as the “Mentor PVO,” brought to the partnership previous experience in managing five USAID-funded Child Survival Projects in Bolivia, and has recently received a USAID matching grant for work in Guatemala and Bolivia. In addition, its Senior Program Specialist (SPS) had lived and worked for two years in Haiti, and was also very familiar with ARHC’s Census-Based Impact-Oriented (CBIO) methodology. Project Description: The Child Survival Project implemented by this partnership seeks to provide community￾based child survival services for approximately 75,000 people in Haiti through nutritional interventions including promotion of breastfeeding and distribution of vitamin A, child spacing services, pneumonia case management, improvement of vaccination coverage, case management of diarrhea, and maternal and newborn care. Through the mentoring partnership, FOCAS personnel and their local NGO partners are learning to implement ARHC’s highly successful CBIO approach to primary health care, as well as gaining skills at administrative, financial, managerial and technical levels. The Mentoring Approach: The mentoring relationship of FOCAS and ARHC is based upon the goal of achieving eventual sustainability through a holistic approach to partnership. The mentoring in this project takes place at all program and administrative levels. Progress and Achievements: This report gives an overview of progress made to date, outlining many specific mentoring activities that have taken place at both field and headquarters (HQ) levels. Significant progress has been made in the three years of the project, with a focus on the transfer of technical skills at field level early in the project, and a shift to more emphasis 79 on managerial and administrative mentoring in the third year of the project. FOCAS field staff have censused over 71,000 people, reaching them through rally posts, home visits, and clinic services. The Executive Director of FOCAS believes that FOCAS and its local NGO partners are one and a half years ahead in their capacity building of where they would have been at this point had they received a “typical” entry-level Child Survival grant. Strengths and Constraints of the Mentoring Approach: A mentoring approach to implementing a child survival project brings an array of benefits, as well as presenting unique challenges. ARHC personnel have bolstered skills of FOCAS personnel in the administrative and programmatic aspects of their work, and at times have worked directly with the implementing partner NGOs in Haiti. ARHC has also worked with FOCAS at the level of strategic planning, board of directors development, and financial management. ARHC has also received benefits from the relationship, gaining valuable experience that will further its ability to provide supportive services to other NGOs and PVOs through a partnering approach. Constraints have sometimes arisen in the areas of personnel availability, communication gaps, and differences in fiscal procedures, and are detailed further in the report. The benefits of the partnership have greatly outweighed the constraints in this project. Summary Recommendations: Lessons learned through the first three years of this project are shared in the report, with recommendations that will hopefully be of value to other PVOs contemplating a mentoring approach. In this project’s experience, the mentoring partnership should be formed on the basis of shared philosophies and compatibility between partners, with realistic expectations and attitudes toward the relationship. A mentoring relationship should start with a comprehensive assessment of organizational capacity of the “Mentoree PVO,” highlighting areas of need and resulting in a detailed plan of mentoring activities to address those needs over the life of the project. Open communication and availability of personnel are key ingredients to maintaining a strong partnership, and a clear monitoring and evaluation strategy should be established to assess achievement of the mentoring objectives as well as the overall program goals. 80 II. Background: A. Partner Organizations 1. Foundation of Compassionate American Samaritans The Foundation of Compassionate American Samaritans (FOCAS) was founded in 1986 by Richard P. Taylor, as a Christian non-profit charitable organization. The primary objective of FOCAS was to help the desperately poor in remote areas of Haiti with basic needs such as elementary education, feeding and primary health care. FOCAS’ initial project was a child support program in a remote mountain community in Haiti, which then grew to include a medical (nutrition, hygiene, health education) ministry and supplemental food program. Over the subsequent years, FOCAS has expanded to include a vocational school in Haiti, and outreach programs to inner city youth in Cincinnati, Ohio. The mission of FOCAS is: “We earnestly seek transformed lives by proclaiming the gospel of Jesus Christ and assisting with crucial physical and spiritual needs. Ref. Luke 4:18-19. . .” 2. Andean Rural Health Care Andean Rural Health Care (ARHC) is a 501(c)(3) non-profit organization, incorporated in 1983. It began its work in Bolivia with community-based child survival and primary health care service delivery. ARHC has developed one of the leading public health models in Bolivia, using a “Census-Based, Impact-Oriented” (CBIO) approach. ARHC works through its Bolivian counterpart organization, “Consejo de Salud Rural Andino” (CSRA), delivering service to over 70,000 beneficiaries. ARHC has successfully competed for and been awarded five USAID BHR/PVC Child Survival Grants, and recently has been awarded its first USAID Matching Grant to support activities in Bolivia and Guatemala. Based on its success in Bolivia, ARHC has expanded its CBIO approach into other countries of need in the Americas, including Mexico, Guatemala, and (through the FOCAS/ARHC CS Mentoring grant) Haiti. In a USAID/PVC/CS external evaluation of ARHC’s program in 1994, the panel concluded that ARHC should share its experience and that USAID should support the CBIO approach in other countries. ARHC’s mission statement reads: “Moved by our faith, we are committed to the measurable improvement of health and the prevention of unnecessary suffering, sickness and death. We bring hope through health, working through self-sustaining local partnerships in communities lacking access to basic services.” 3. Local Haiti Partners: MEI and OBDC 81 In Haiti, FOCAS works with and through Mission Evangelique International (MEI) and Oeuvres de Bienfaisance et de Developpement Communautaire (OBDC). FOCAS has worked for many years with MEI, an indigenous non-profit, non-governmental organization (NGO) operating for over 20 years in church development, education, and health. Its health work had included the operation of fixed and mobile medical clinics, immunization, nutritional education and feeding, and training of midwives. OBDC, another indigenous NGO, was brought into a parallel partnership with FOCAS through the current CS grant. This organization had been implementing child survival and family planning programs for several years in another nearby rural area of Haiti. B. CS Mentoring Partnership Grant Submission Process FOCAS submitted its first application for a USAID Bureau of Human Response / Office of Private and Voluntary Cooperation (BHR/PVC) Child Survival Project grant in 1994. After being unsuccessful in that first attempt, it tried again in 1995. Once again, it received an unfavorable response from USAID. However, in 1996, USAID BHR/PVC offered a new category of CS grant, called “Mentoring Partnerships.” As the name implies, this category was specifically geared toward smaller, newer PVOs lacking the experience and field expertise in child survival to compete successfully for the USAID CS grant against larger or more established PVOs. For this reason, it seemed especially suited to an organization like FOCAS. Richard Taylor weighed his options once again for the new CS funding cycle, and decided to apply under this new grant category. He then had to decide which PVO might be the most appropriate one with which to form a mentoring partnership. He entered into discussions with ARHC as well as with World Vision International (WVI). Both established PVOs were interested in joining FOCAS in the mentoring partnership, but each proposed a very different approach and structure to the relationship. WVI offered a larger corporate structure, a larger body of resources, and shared a similar holistic, outwardly Christian approach to administering health projects. At the same time, however, the WVI concept would have had WVI as the primary grantee, with FOCAS as a secondary grantee. In addition, the FOCAS service area and number of beneficiaries would have been much smaller. ARHC, on the other hand, was a smaller PVO (with seven full-time headquarters staff at that time) but offered considerable experience with CS projects, and was also willing to allow FOCAS to be the primary grantee in the partnership. As described previously, ARHC also had recently expanded its mission focus and was seeking to partner with other health development organizations to promote the CBIO approach to primary health care (PHC) in the Americas. Of some concern to Mr. Taylor was that, while faith-based, ARHC did not have an evangelical approach to development work, which is at the core of the FOCAS mission. FOCAS eventually decided to partner with ARHC. The partnership has paid off well in many ways for both ARHC and FOCAS. Mr Taylor feels that by being the primary grantee in its partnership with ARHC, FOCAS’s learning curve has been steeper than otherwise might have been possible. He also feels that the personnel resources of ARHC, 82 while fewer in number than those of WVI, may have in some ways been more approachable and accessible than WVI personnel might have been. He feels they have been willing and able to work fairly closely with FOCAS staff, attending their board meetings and visiting their office in Cincinnati. The differences in mission between FOCAS and ARHC seem to have had no appreciable adverse impact on the partnership. Rather, the programmatic similarities of the two PVOs have greatly benefited the project. C. Project Description The ultimate goal of this child survival project is to reduce unnecessary morbidity and mortality, and improve the health of project area children less than five years and women of childbearing age, through the provision of key CS interventions. A secondary goal is to increase access to high-quality basic CS and primary health care services in the project areas. The tertiary goal of the project is to increase the capacity of FOCAS and its Haitian NGO partners to successfully plan, implement and evaluate sustainable, community-based CS services. There are six key CS interventions proposed for this project: ß nutrition improvement, including vitamin A supplementation and promotion of breastfeeding (25% level of effort); ß diarrhea case management (10%); ß pneumonia case management (20%); ß immunization (10%); and ß child spacing promotion (25%). ß maternal and newborn care (10%) These interventions are being implemented through the census-based, impact-oriented (CBIO) methodology developed by ARHC. The project plans to reach a total population of approximately 75,000 by the end of its four years. 83 III. The Mentoring Approach A. Philosophy and Objectives of the Mentoring Partnership Being a new experience for all parties involved, including USAID, the mentoring partnership approach to child survival projects has evolved and taken shape and direction throughout the initial three years of project implementation. From the beginning, FOCAS and ARHC entered into this mentoring partnership with several well-defined philosophical tenets that support their work. Three of these key tenets are as follows: 1. A Holistic Mentoring Concept A strength of the FOCAS-ARHC approach, noted by USAID in awarding the grant, was in conceptualizing the mentoring partnership in a holistic context. While other PVOs may have viewed a mentoring CS approach more as a technology knowledge transfer grant, the FOCAS & ARHC leadership had a much richer vision of what it could be. The concept was verbalized slightly differently by the various personnel interviewed from both organizations. A common theme, however, was that the Mentor PVO would bring to the partnership a proven organizational methodology and philosophy, and a commitment to a successful approach in implementing a child survival project. ARHC personnel would “shadow” FOCAS personnel at all organizational levels, including the board of directors, headquarters and field administration, technical programs, NGO relations, and overall program management. 2. Working in Partnership Richard Taylor (Founder and Executive Director of FOCAS) believed strongly in the importance of partnership even before the mentoring project became a reality. He presented his thoughts at the 11th Annual CS PVO Headquarters Workshop held at Lake Junaluska in 1999. The eight major points of partnership he stressed were: 1. The partners must have equivalent values. 2. The partners must have mutual respect and trust. 3. There must be a thorough partnership plan. 4. It must be a win-win arrangement. 5. Management must be committed to success. 6. The partners should have complementary skills and capacity. 7. There should be careful selection of project personnel. 8. There needs to be clear, written documentation of the agreement between the two partners. 84 3. Working Toward Sustainability As with any Child Survival project, sustainability in this case may be defined as the ability of the local implementing partners (in this case the two Haitian NGO partners with and through whom FOCAS works) to continue the process of developing and expanding basic CS and primary health care services after the end of this funded project. In this project, however, a first level of sustainability is necessary for the attainment of that goal, given the tiered mentoring structure involved in this approach. The first level of sustainability will be achieved as the Mentoree PVO, FOCAS, attains its own potential in independently administering, managing and implementing future child survival projects. Following that, but of ultimate importance, will be the more profound goal of sustainability at the field level. This presents a serious challenge in a country like Haiti, where making realistic progress toward sustainability is tempered by the realities of an unreliable and tenuous social, economic and political infrastructure at many levels. Because of the complex nature of these two tiers of sustainability, the project is still working on defining a clear sustainability plan, but holds sustainability at both levels as a key goal of this mentoring partnership. B. Design / Approach of the Mentoring Process The Mentoring approach taken in this project can be visualized on three different levels. 1. Mentoring Level I Mentoring at this level refers to mentoring at the PVO headquarters level, implemented through the relationship between FOCAS/USA and ARHC HQ personnel. a. Personnel Communications & Mentoring Structure at Level I b. Focus of mentoring at Level I ß Strategic Planning ß Development of Board of Directors ß Administrative Policies ß Fundraising ß Financial Management ß Personnel Management c. Characteristics of mentoring at Level I ARHC Exec Dir, Program Dir FOCAS Exec Dir, Health Pgm Dir 85 All partnered personnel at this level are based in the U.S., which makes communications easier than at the other levels. Telephone, E-mail and fax are the primary modes of communications used. Travel also has been a means of mentoring at this level, through in-person meetings and visits to partners’ offices, usually taking place twice per year. In addition, senior staff of both PVOs have attended the other partner’s board of directors meetings twice in the first three years. Partners from both PVOs have exchanged numerous memorandums and other documents focusing on areas of need and mentoring activities at the various project levels. They have worked without a formal timetable of scheduled activities such as is used at the more technical levels II and III. Rather, during the first three years of the project, mentoring has taken place often on an as-needed basis, sometimes planned months in advance – such as attendance at the partner PVO’s board of directors meetings – but sometimes happening in response to a more immediate need on the part of the Mentoree PVO. 2. Mentoring Level II Mentoring at this level refers to advice and counsel of a more technical and managerial nature, taking place between the partnered FOCAS/USA and ARHC technical and financial staffs, and the FOCAS/Haiti staff. a. Personnel Communications & Mentoring Structure at Level II b. Focus of mentoring at Level II ß Technical Interventions in Child Survival ß CBIO Skills Development ß Techniques in Training and Evaluation ß Project Planning, Monitoring and Evaluation ß Project Reporting, Including HIS Development ß Field-level Personnel Management and Training ß Resource Development, Grant-writing c. Characteristics of mentoring at Level II ARHC Program Dir & Sr. Tech Specialist FOCAS/USA Health Program Dir FOCAS/Haiti CS Program Manager 86 Clearly outlined timetables of topics and trainings were developed early in the project and expanded/modified on an ongoing basis by the ARHC Senior Program Specialist (STS) , FOCAS/HQ Health Programs Director , FOCAS/Haiti CS Program Manager and partner NGO Program Managers in Haiti. The ARHC SPS worked directly with FOCAS personnel as much as possible in planning, coordinating and training of the technical interventions from the beginning, and has gradually encouraged more initiative and leadership on the part of the FOCAS/Haiti CS Project Manager in taking on these roles. During the first two project years, the ARHC SPS made eight visits to Haiti, usually for several weeks at a time. During the third and fourth years, the SPS is planning to make only one or two trips to Haiti per year. Communications at this level are understandably more problematic than at Level I. When the HQ personnel are not in Haiti, communications between the mentoring partners at this level take place largely through E-mail, and sometimes through telephone calls and faxes. The ARHC SPS often initiated communication in the first two years of the project, based on his awareness of issues and concerns that needed to be addressed. Currently, the flow is more balanced, with both the FOCAS Health Programs Director and the FOCAS/Haiti CS Program Manager initiating communication with the STS. Communications have been a continuous challenge due to power and telephone / fax / E-mail breakdowns in Haiti. 3. Mentoring Level III Mentoring at this level refers to the relationship, including technology and managerial transfer, between FOCAS/USA - Haiti and the two indigenous NGO’s with whom and through whom they implement the child survival project. This level of mentoring and backstopping between the CS grantee PVO, headquartered in the U.S., and its implementing partner NGOs in the target country, is similar to a “typical” CS project. a. Personnel Communication & Mentoring Structure at Level III b. Focus of mentoring at Level III ß Technical Interventions in Child Survival ß CBIO Skills Development ß Techniques in Training and Evaluation ß Project Planning, Monitoring and Evaluation FOCAS Health Director, FOCAS CS Project Mgr (with TA from ARHC or other consultants) Haiti Partner NGO’s, clinics, field staff 87 ß Project Reporting, Including HIS Development ß Field-level Personnel Management and Training ß Resource Development, Grant-writing As can be seen, the major focus of mentoring at this level includes the same topics mentioned in Level II. At this level, however, the focus is primarily on field-level operations and delivery, and with expertise coming through FOCAS personnel, with minimal input by ARHC. ARHC has frequently provided TA simultaneously to the FOCAS/Haiti staff and NGOs by modeling techniques, especially during trainings. c. Characteristics of mentoring at Level III Communications at this level, mostly within Haiti, have been more problematic due largely to the inconsistent and under-developed Haitian infrastructure. Though the situation has improved, neither of the two local partner NGOs has consistently reliable radio or telephone contact with the FOCAS office in Petionville, Haiti. The two NGO Program Managers both have personal cellular telephones, but since FOCAS/Haiti does not, they still depend on unreliable phone service at the FOCAS office. (FOCAS/Haiti has applied for a cell phone in Haiti but has not yet received it). Transportation limitations also have hurt access among the NGOs. The three NGOs attempt to hold weekly meetings, but maintaining this regularity is often not feasible in Haiti. 4. Other mentoring inputs The personnel and issues highlighted above do not reflect the entire scope of the mentoring processes in this project. At times, other personnel from both PVOs have taken part in mentoring activities and idea exchanges, such as board members, financial and administrative support staff, and field-level CS staff (i.e., during FOCAS/MEI/OBDC visits to ARHC CS projects in Bolivia). Additionally, the schematics above represent the primary directions of communication, but in reality there is frequent overlap in communications among and between the three levels. C. Mentoring Partnership Plan One of the first documents generated by the mentoring parties was a “Memorandum of Agreement,” included in the project’s DIP. This presented the general terms and guiding principles of the mentoring relationship between the partner PVOs. Several other documents also were generated and proposed as reference guides at the beginning of project activities. During the first two years of the project, these guidelines were used and revised by personnel from both PVOs. Although the concepts outlined in these documents have been used to guide mentoring activities to date, they have not yet 88 been formalized into a comprehensive mentoring guide. Rather, an overall process has evolved throughout the project’s first three years, and still is viewed as a “work in progress” by project staff. The technical aspects of the mentoring plan have lent themselves to a more formal, structured, and documented process than the administrative / managerial aspects of the partnership. This is due, in part, to the decision that the technical mentoring was to take first priority, to “hit the ground running” with CS interventions in Haiti. The benefit of this approach was that, potentially, more lives would be saved as a result of the earlier initiation of project interventions. However, it also has resulted in some delay in the transfer of managerial and administrative skills, a constraint noted during the mid-term evaluation. These latter themes now are being given a stronger focus during the third and fourth years of the project, as detailed further in this report. The original grant proposal had anticipated that the mentor role played by ARHC would be strongest over the first two years of the project, but would extend throughout the four years of the grant period. At this point, nearing the end of the third year of project funding, progress in some mentoring areas, most notably the transfer of technical skills to field staff, has been very good. The role and time allocation of the ARHC SPS in this project has, in fact, been recently reduced. The other areas of managerial, administrative and financial mentoring are seeing gradual progress, somewhat delayed due to the resignation of the first FOCAS Health Programs Director in the second year of the project. FOCAS and ARHC will be seeking to extend the mentoring partnership for several more years, through the application of a follow-on grant. The details and management strategy for these add-on years is currently being developed by FOCAS and ARHC personnel. 89 IV. Project Progress and Achievements A. Specific Achievements of the Mentoring Partnership In keeping with the purpose of this report, this section is not intended to be a comprehensive listing of all achievements of the CS Project to date. Rather, its purpose is to highlight the progress and achievements that were accomplished largely through cooperative aspects of the mentoring partnership. These accomplishments are listed somewhat, though not entirely, in chronological order. A brief overview of mentoring activities throughout the LOP also can be seen in Appendix A. 1. Entry into the Mentoring Partnership The Program Director of ARHC had previously met the Executive Director of FOCAS, and had offered informal assistance through advice and the sharing of written materials during previous FOCAS CS grant submissions. After deciding to partner for the mentoring grant submission, they worked together to submit the successful grant application. ARHC’s previous experience in successful grant-writing enhanced FOCAS’ own efforts, and together they received the only grant to be awarded thus far in the mentoring partnership category of USAID BHR/PVC Child Survival grants. 2. Initial Planning After receiving notification of the grant award, a first step in the now-official mentoring partnership was to begin detailed project planning. To accomplish this, the Program Director and Senior Program Specialist of ARHC went to Haiti with the Executive Director of FOCAS, to meet with the newly-hired CS Project Manager of FOCAS/Haiti and the Program Managers of the two partner NGOs in Haiti, MEI and OBDC. At this meeting in September 1997, they sketched out plans for the first two years of the project. 3. Baseline Survey As in most CS projects, a Knowledge, Practice and Coverage (KPC) baseline survey was one of the first technical activities to take place in the field, in November 1997. The desired approach would have been for ARHC and FOCAS HQ health staff to work together in planning and conducting the survey, and in transferring the KPC survey skills to the local partners. In this case, however, FOCAS had not yet hired a Health Programs Director at the HQ level. Instead, the ARHC Senior Program Specialist worked together with the CS Project Manager to organize and conduct the baseline survey, still facilitating a direct transfer of skills to the FOCAS/Haiti level, and avoiding the need to bring in an outside consultant to coordinate the survey. 4. Writing the Detailed Implementation Plan 90 Another major partnering activity early in the project was the writing of the Detailed Implementation Plan (DIP) after results from the KPC baseline survey were available. The DIP was completed in April 1998, as a combined effort primarily among the ARHC STS, the newly-hired FOCAS Health Programs Director and FOCAS field personnel. As with the baseline survey, this was an example of the Mentor PVO providing the relevant abilities and experience to the Mentoree PVO, building the capacity of the partner NGOs in the process, and replacing the need for an outside consultant. The DIP was reviewed and approved by USAID in June 1998. 5. Sharing the CBIO Philosophy / Field Visit to ARHC CS Project in Bolivia Plans were made early in the project for key FOCAS/USA and Haiti staff to visit an ARHC / Bolivia Child Survival project. Unfortunately, illnesses and a death forced two postponements of the trip. It finally took place, however, early in 1999, and allowed FOCAS/Haiti and its partner NGO field staff to learn directly from their counterparts in a “prototype” CBIO-based child survival project, and to see first-hand the successes and constraints of their partner PVO at the grassroots level. Unfortunately, the trained FOCAS HQ Health Programs Director left the organization soon thereafter, but the second Health Programs Director visited the Bolivia project and received the same training during June 2000. 6. Child Survival Technical Trainings FOCAS and ARHC first emphasized providing technical assistance related to child survival interventions in the field, and then gradually phased-in program management as an additional area of mentoring. The specific areas and timing of CS technical assistance provided are detailed more completely in Appendix B. Most trainings were conducted by the ARHC SPS in collaboration with FOCAS’ CS Project Manager. The FOCAS HQ Health Programs Director later took a minor role in some trainings, and the FOCAS/Haiti CS Project Manager gradually took on more responsibility as well. Trainees were often pre- and post-tested to assess learning, another important training activity that was gradually taken over by the CS program manager. To date, training and implementation of most of the CS interventions have been completed in the field. Current and projected areas of technical mentoring and training now will focus on continuous quality improvement, and enhancement of management and supervisory skills and processes. Specific examples of the FOCAS/Haiti CS Project Manager assuming formerly mentored tasks: a. The FOCAS/Haiti CS Project Manager has repeated some topics/trainings that the ARHC SPS conducted initially in the project, and now is independently using pre￾and post-tests to assess the quality of the training and learning. The ARHC SPS still often reviews the content of the trainings and the results from the pre- and post￾tests, offering feedback. 91 b. In April 1999, following the DIP review, the Haiti project managers with the assistance of the ARHC STS, conducted a health education assessment: A general service checklist was developed by the CS Project Manager, with the ARHC SPS taking a secondary role, providing feedback on the checklist. c. Also in April 99, an ARI checklist was developed by the ARHC SPS and CS Project Manager jointly. The CS Project Manager shared the protocol from a REACH training management workshop he had just intended, the ARHC SPS drafted a checklist based on that protocol, and then the CS Project Manager offered feedback. The field staff and supervisors used the ARI checklist for awhile, then in February of 2000 visited another project in Haiti, after which time they modified and simplified the checklist based on their experience. The ARHC SPS acknowledged that he did not necessarily agree with all of the changes and simplifications made to it, but that it remained an appropriate tool. The end result was that the FOCAS/Haiti and NGO Program Managers had taken ownership in developing their own quality improvement tool based on what they had learned through the mentoring process. 7. IEC and M&E Materials Many technical training, supervisory and evaluation documents have been developed as part of the technical component of this project. In many cases, these tools have been developed through a mentoring approach, primarily relying on input and coordination from the ARHC STS, in collaboration with FOCAS, MEI and OBDC personnel. In some cases, responsibility for the development of the training and supervisory guides has shifted from the ARHC SPS to the FOCAS/Haiti, MEI and OBDC program managers. 8. Conducting the First Census As an integral part of the CBIO approach, conducting and maintaining a census has required extensive training and TA from the ARHC mentors. The project is expecting to reach about 75,000 by the end of the four-year project. The project was able to census 20,000 during the first year, and by the end of the third year of the project they had censused over 71,000 individuals. 9. Development of the Annual Implementation Plan (AIP) Following their participation with ARHC staff in developing the DIP, FOCAS HQ and Haiti staff, along with their partner NGO staffs, developed an AIP in August, 1998. This process was done almost completely by FOCAS and local partner staff, with only limited assistance required from ARHC. 10. Ongoing Management Mentoring 92 Development of project annual plans, mentioned above, should be an ongoing activity which reflects a shift away from ARHC direction and more toward grassroots input and leadership. In the field, processes in management and supervision of staff, originally a part of training sessions provided by the ARHC STS, have gradually been taken over by the FOCAS CS Project Manager and, to a lesser degree, by the NGO Program Managers. As mentioned above, supervisory checklists, an integral part of the supervisory protocol of this project, now are often developed or revised by project personnel in Haiti. Though progress has been made in this area, in his August 2000 visit to the FOCAS/Haiti CS project, the ARHC Executive Director noted several aspects of the managerial and supervisory processes that need further strengthening. These will be emphasized and addressed during a follow-on grant period. Health information systems management has seen some transition in “ownership” to Haiti personnel, and at this point is appropriately remaining a manual system. There are plans to improve and partially computerize it during a follow-on grant period, a process which will likely depend upon ARHC and other outside input at the initial stages. Project staff had expressed a desire for further management and financial management training during the mid-term evaluation. In response to this, the former ARHC Program Director (now Executive Director) outlined an overall scope of mentoring tasks for 2000 – 2001, and is gradually addressing these training needs. 11. Ongoing Administrative Mentoring Information has been shared freely between the two PVO Executive Directors in many areas of administration and finance. Specifically, mentoring has taken place in the following areas: a. Fundraising: The ARHC Executive Director has held many discussions with the FOCAS Executive Director regarding fundraising ideas and resources. This counsel has included fundraising of individuals, service clubs, churches, and foundations. Information shared by ARHC has especially focused on the best timing, content and direction of grant-writing to specific foundations. Though there have not yet been significant “fruits” of this mentoring, the FOCAS Executive Director has expressed an expanded vision and increased confidence in approaching potential donors. They also have adopted alternative fundraising strategies, such as ARHC’s efforts in organizing home meetings as a vehicle to fund raising. In addition, FOCAS has started a highly successful annual fundraising breakfast targeted toward the downtown Cincinnati business community. b. Strengthening the FOCAS Board of Directors: 93 Mutual attendance at partner PVO’s Board of Directors meetings has been taking place about once per year, and the ARHC Executive Director occasionally reviews FOCAS Board of Director meetings minutes, offering feedback. The ARHC Executive Director has also suggested the importance of broadening the FOCAS Board by recruiting new Board members, and FOCAS is following his suggestions by recruiting new members with increased diversity. c. Strategic planning: ARHC has shared its annual operational plans and five-year strategic plans with FOCAS HQ. This is a subject where the two organizations differ significantly in approach, but FOCAS is considering how to adopt selected ideas to its own organizational planning philosophy and priorities. d. Financial management: There has discussion thus far between ARHC and FOCAS regarding FOCAS’ financial and accounting systems. ARHC shared financial documents and audited financial reports with FOCAS early in the partnership. FOCAS’ financial growth has been significant over the past several years, and ARHC had suggested changes that were needed in FOCAS’ fiscal and accounting procedures. Ideas and memorandums were shared along these lines, with the ARHC Executive Director addressing the issue at FOCAS’ Board of Directors meeting in May 2000. Changes have been made, and FOCAS has satisfactorily completed two A133 audits by an independent auditing firm. More work is planned in this area, especially focusing on financial management capacity building at the field level. e. Personnel management: ARHC has shared information openly with FOCAS from its own personnel management system. It has shared written personnel policies, job descriptions and salary structures at both the HQ and field levels, annual personnel appraisal forms, and organizational charts. Some changes have been made at FOCAS as a result, including an HQ re-organization and the preparation of a new personnel handbook with new job descriptions. B. Progress Since the Mid-Term Evaluation The Mid-Term Evaluation (MTE), led by an external consultant in October-November 1999, formally documented many successes of the mentoring approach thus far33. At the same time, it highlighted areas of the mentoring process that still needed improvement and/or attention. 33 “FOCAS in Haiti 1997-1999; PVO Child Survival Program Mid Term Evaluation”; Bette Gebrian Magloire RN, MPH, PhD; November 1999. 94 Areas of progress highlighted in the MTE report included: · Successful process established for clinical and technical transfer from ARHC to FOCAS · Increase in technical and management capacity at FOCAS HQ · FOCAS/USA entry into CORE USAID group to benefit from expertise of others · FOCAS partner NGO staff provided with concrete tools with which to assess quality of care at level of individual beneficiary Areas of constraint and suggestions for improvement noted in MTE report: · More planning, implementation and evaluation of capacity building · Documented plan for TA in management (including financial management) · Further training and mentoring in fundraising strategies needed · Development of a sustainability plan ARHC and FOCAS personnel have made substantial progress acting on these recommendations in the past year. Some of the specific strategies and activities are summarized below, though many of them have also been addressed in greater detail in the previous section, IV A. “Specific Achievements of the Mentoring Partnership.” Capacity Building: The project has developed a strategy for organizational capacity building for 2000 – 2001. Although this plan is still undergoing final revisions, it has laid the groundwork for the remaining activities, some of which have already begun. Technical Assistance in Management / Financial Management: After the MTE, a plan for management training during FY 2000 and 2001 was outlined by ARHC and is still undergoing revision before complete adoption. Training in several of the proposed areas has already taken place, most notably in a series of trainings that took place in Haiti in July-August 2000, conducted by the ARHC Executive Director and the FOCAS Health Programs Director . As mentioned previously, ARHC and FOCAS have been working together on the financial management systems of FOCAS at both the HQ and field level. Capacity building in this area at the field level is an objective of regular visits to Haiti by the FOCAS Executive Director and Financial Consultant, and this will remain a focus of mentoring during the remainder of the project. Training in Fundraising: There has been ongoing mentoring at the HQ level in fundraising over the last three years, though somewhat less at the field level. However, fundraising was a topic covered with NGO and FOCAS/Haiti Program Managers during the latest visit of the ARHC Executive Director in July - August 2000. While there, along with the FOCAS Health Programs Director, he met with project managers, discussed alternative sources of local funding, and visited potential donors with the FOCAS/Haiti Project Manager. Partner NGO Program Managers also received ideas and input from ARHC field staff in raising local support during their trip to Bolivia. 95 Plans for Sustainability: Project personnel are currently developing sustainability plans for both levels of the project, as discussed previously. Sustainability in Haiti is a formidable challenge, due to the rugged nature of country, lack of infrastructure, and the ever-tenuous political stability. The Haiti field staff persevere admirably in the face of all of this, and progress continues to be made. Sustainability has a strong correlation with capacity-building. Technical capacity-building has been done well at the field level. The current and projected efforts in CQI mentoring should assist greatly in promoting sustainability from a technical standpoint. 96 V. Strengths and Constraints of the CS Mentoring Approach A. Strengths of the CS Mentoring Approach The previous section, IV, listed many tangible benefits and accomplishments of the project, all attributable to strengths of the CS mentoring approach. Additional strengths and benefits are listed below: 1. Benefits to the “Mentoree” PVO Clearly, a strength of the CS mentoring approach involves the many implicit benefits to the “Mentoree” organization. a. Entry into USAID BHR/PVC CS project funding: One of the primary benefits, by design, is the ability for a younger, smaller, but competent PVO to gain entry into a potentially ongoing series of USAID-sponsored Child Survival grants, an otherwise restrictively competitive process. Though at times there are unavoidable constraints and frustrations in trying to merge personalities and corporate philosophies of two dynamic PVOs to accomplish singular project objectives, the bottom line is that the mentoree PVO would likely not have been able to secure USAID CS funding without the input of the mentor PVO. Whatever constraints may arise are surely outweighed by this benefit. b. Navigating the USAID system: Many PVO’s implementing a USAID CS project for the first time are sure to be slow to attain competency in the level of accountability and reporting required by USAID. The assistance offered by a seasoned PVO in navigating the complexities of the USAID system is another benefit of mentoring. c. Technical assistance in a holistic context: As previously stated, a mentoring versus a consulting approach makes available a holistic philosophy and corporate “personality” of a successful PVO, along with a wealth of experience in a proven methodology. (This is described more fully in section III.A.1.) 2. Benefits to the Mentor PVO A perhaps less obvious strength of the mentoring approach is the benefit to the Mentor PVO itself. In the case of this partnership, personnel from the Mentor PVO were quick to acknowledge benefits they have received: Among those mentioned were: 97 a. A strengthening of their own technical skills as new or modified approaches were implemented. One example of this was the increased experience gained by the ARHC SPS in the Hearth Nutrition Model as part of this project. b. An increase in skill, experience and reputation of the PVO as a mentoring and training resource to assist other PVOs/NGOs. This is especially beneficial in the case of ARHC, where its mission is increasingly one of supporting and strengthening other organizations through similar partnerships. c. Experience may be gained in a new geographic and cultural context. 3. Benefits to the Local NGO Partners and Project Beneficiaries The benefits at the field level are many of the same benefits listed in section 1 above. In addition, field partners are able to benefit from a larger body of resources while being mentored by personnel from two distinct PVOs. As seen in this project, personnel from ARHC also were able to fill in potential backstopping “gaps” resulting from the turnover in the FOCAS Health Programs Director position. B. Constraints of the CS Mentoring Approach Most constraints voiced by personnel from both partner PVOs were difficulties inherent in the process of any inter-agency collaboration. Management and administrative issues, fiscal considerations, accountability and decision-making, and styles of communication all pose challenges within any single PVO, let alone between two PVOs collaborating in the implementation of a new, ambitious and time-limited child survival project. Specific issues highlighted by staff are noted below. 1. Shared accountability and control A constraint of this relationship likely to be experienced in other mentoring relationships is the issue of shared accountability and control. While on paper, final decision-making control and accountability for this project ultimately rests with FOCAS, personnel from ARHC are invested in the interventions and outcomes and certainly feel an implicit sense of accountability and responsibility for both the successes and shortcomings of the project. As in any partnership, there are compromises that must be made, and a certain degree of loss of autonomy. Not all joint decisions made or actions taken may be completely in keeping with each organization’s own corporate philosophies. However, in most cases, staff from both partner PVOs have collaborated in making optimal decisions for project success through open and honest communication, and with personal and professional flexibility. 98 2. Personnel Quality and Turnover In this partnership, both organizations are fairly small with a limited number of HQ personnel. Personnel turnover in both PVO’s has led to some difficulties, not so much in continuity of program activities, but in continuity of communications at both the HQ and field levels. Likewise, the loss and turnover of personnel of both PVOs seemed to cause periods of personnel unavailability, as attentions were necessarily directed more internally and workloads increased. At the same time, however, as noted above, there were benefits brought about by the mentoring relationship during these times (i.e., ARHC’s Senior Program Specialist was able to fill the technical and training “gap” in Haiti while FOCAS sought their first, then later a second, HQ Health Programs Director). FOCAS has had some difficulty in finding the “right” personnel for some positions, and the delay in hiring the first Health Programs Director, and his subsequent departure, undoubtedly caused significant delays in project implementation. 3. Personnel Availability Related to the previous section, the realities of two busy PVOs scheduling meetings together presents a challenge in the best of times. Unexpected turnover of personnel and urgent issues faced by PVOs resulted in personnel from both PVOs not always being available to the other organization. Although this was a shared constraint, it was felt more strongly by personnel from FOCAS, as their time is close to 100% with this project, whereas the two primary ARHC mentoring personnel have limited time allocated toward it. The ARHC team’s own schedules and workloads often required FOCAS to schedule ARHC’s time significantly in advance. This required that FOCAS personnel be proactive and far-sighted in scheduling time with ARHC personnel, as well as allocating their own time (an important skill which should also be developed through the mentoring process). Any unexpected changes to the schedule, naturally, cause additional difficulties. This has probably been one of the areas of greatest frustration within the partnership. 4. Communications Early in the project, computer platform differences between partners caused significant difficulty in E-mail communications as well as in sharing of documents. FOCAS/USA & Haiti now have both Macintosh and IBM-format systems, but FOCAS/Haiti had only Macintosh at the beginning of the project. Telephone services in Haiti are very problematic, with frequent breakdowns. During the first project year alone, the FOCAS/Haiti telephone line was at one point out of operation for more than two months. The personal telephone lines of the Haiti FOCAS and NGO partner staff, a potential backup system, were also out of service during the same period, some for as long as six months. Similar outages continue to the present. Even when the FOCAS/Haiti telephones are working, communication among and between the FOCAS/Haiti office and partner NGOs is difficult. It sometimes takes the 99 FOCAS/Haiti staff hours to get online to send or receive E-mail. Messages transmitted to the FOCAS/Haiti office often do not reach the NGO partners promptly. Apart from physical and logistical constraints to communications, differences in personal communication “styles” caused further difficulties. As in any organization, open and honest communication styles are essential, and communication between two collaborating PVOs requires prompt returns of calls/E-mails. Difficulties in maintaining consistent and prompt communications were expressed by both PVOs along these lines. 5. Resource Allocation Personnel from both PVOs have experienced some dissatisfaction with the “spending attitudes” of the other. FOCAS staff were sometimes encouraged by ARHC to allocate resources in ways that they might not have chosen to do independently, and were occasionally concerned by unanticipated or unknown costs of working with ARHC staff. ARHC personnel have felt that, at times, FOCAS personnel were disinclined to spend the money necessary for ARHC services, while at the same time FOCAS staff have felt that specific ARHC personnel haven’t always been available to provide mentoring services when needed by FOCAS. This important area of uncertainty and diversity in financial resource allocation and income has probably been one of the biggest strains within the partnership. 100 VI. Summary Recommendations and Comments As mentioned previously, the purpose of this report is to document the mentoring partnership of FOCAS and ARHC, as well as to make recommendations for other PVOs considering a mentoring partnership grant. A. Lessons Learned: Recommendations for Future Mentoring Partnerships 1. Constructive Mentor and “Mentoree” Expectations and Attitudes Clearly, a key in successfully implementing a mentoring relationship between the PVO partners is that both share common expectations and attitudes regarding the relationship. The Mentoree PVO must have key personnel at all levels who understand the mentoring process and their role in it, who are willing to be mentored, and who have the basic skills and training needed. The reality in some cases may be that the Mentoree PVO has agreed to a mentoring relationship as the most likely way, or as a last resort to attain USAID funding for its child survival activities, but without the individual or corporate will to accept guidance and change.. Another constraint may be that new CS project personnel will typically be hired after the grant is approved, and thus can not participate in initial planning and mentoring agreements. Therefore, a key aspect of the hiring process should be looking for personnel characteristics of skill, training, a willingness to be mentored, humility and “teachability.” At the same time, the Mentor PVO must reflect patience, tolerance and flexibility. By virtue of its role as “mentor,” it has accumulated experience and competence in administering specific projects in specific contexts. It certainly, however, could not expect, or be expected, to have an answer for every potential situation that may arise in a new project. The Mentor PVO personnel should be willing to advise, counsel and teach, but also must be willing to let the Mentoree PVO make its own final decisions and take its own independent actions. If the Mentoree PVO has some resulting failures, it can pick itself up, dust itself off, and learn from its mistakes, as would any PVO. At the same time, the Mentor PVO might, itself, learn something from the resulting successes of decisions made independently by the Mentoree PVO. 2. Assuring a Shared Philosophy and Compatibility Between Partners Certainly it is important to understand as thoroughly as possible the mission, vision, corporate philosophies and operational structures of the potential partner PVOs. Even best efforts at this may not completely alert the respective partner PVOs to potential areas of conflict, but forming a strategy of how future conflicts will be discussed and resolved is essential in keeping the partnership strong. In this partnership, directors of both PVOs had met in person and talked by telephone both formally and informally several times to discuss partnering possibilities, and to 101 familiarize themselves with the other. Differences were acknowledged and discussed early on, but were expected not to be major obstacles in the partnership. As in most relationships, it is virtually impossible to match corporate or individual philosophies and management styles in all key areas. But certainly the general philosophies of the two organizations should be compatible. This may be especially important in the case of faith-based organizations, where approaches to planning, fundraising, and conflict resolution, among others, may be very different. Fiscal compatibility in planning, making and recording resource expenditures can be a difficult issue for two PVOs to resolve when not clearly spelled out in the initial partnership agreement. As with most new projects, initial budget estimates are necessarily “best guesses.” Therefore, open communication leading to a clear understanding of the financial expectations of each PVO should take place, preferably as the partnership is being cemented, to assure that the two PVOs will be “fiscally compatible.” 3. Initial Assessment of Organizational Capacity Extensive planning must be done at the beginning of the project, and it should start with an evaluation of organizational capacity and expectations of both PVOs. An identification of existing and lacking skill areas of the Mentoree PVO should include technical, managerial, financial and administrative levels. Specifically, the Mentor PVO, with participation of the Mentoree PVO, should make a checklist of anticipated skills needed by the Mentoree PVO to competently implement a CS project. The list should address “skills” in organizational capacity as well as skill areas for individual staff positions. Ideally, the Mentoree PVO key staff should conduct a self-appraisal in the various skill areas, and compare their results to the external appraisal done by the Mentor PVO. The list should be detailed enough to avoid misunderstandings ( i.e., the Mentoree’s self-appraisal of growth monitoring skills based on less rigorous project experience may be a fairly “glowing” rating, while in reality his/her skills are less than satisfactory in light of the skill level needed in a USAID-funded CSP). If skill areas are detailed sufficiently in the checklist, many of these differences can be identified and addressed early in the project. The assessment should result in specific personnel being assigned to mentor in specific skill areas, at specific times throughout the project. An enormous sensitivity and openness is needed on the part of both PVOs during development of this mentoring plan, as the initial assessment may indicate, for example, a need for the Mentoree PVO to completely restructure its fiscal processes, or to enact other major reforms in its standard operating procedures. Mentoree personnel also must exhibit a high degree of sensitivity and honesty in assessing their own level of competency. Concepts of appreciative inquiry could be used effectively in this initial planning and assessment, determining the existing organizational capacity of the PVO, and carefully deciding on the next steps to be taken. 102 4. Detailed Mentoring Plan Upon completion of the organizational assessment, a formal, detailed implementation plan of specific mentoring tasks should be developed. This will give structure to the vision of the mentoring relationship, and will help to define the roles of key players on both sides of the mentoring relationship. The initial plan of the mentoring partnership should include establishing methods for joint problem solving, as well as activities in all identified areas of mentoring: technical, managerial and administrative. Having a formal, agreed-upon document detailing tasks, completion dates, and expected outcomes will assist both organizations in monitoring and evaluating the mentoring aspect of the project on an ongoing basis. 5. Evaluation of the Partnership The partnership should be evaluated in a holistic framework, much in the same way it was formed. The evaluation should consider the efficiency and quality of the partnership, not just the number of beneficiaries being ultimately served. Outcome and process indicators could be set after the initial needs assessment and mentoring plan, to allow both partner PVOs to monitor ongoing progress and to establish a basis for a formal evaluation of the mentoring objectives as a subset of project objectives. As part of the mentoring evaluation process, a formal external documentation/evaluation could be built into the project timeframe and budget. The first evaluation could be done as early as possible to allow for modification and improvement in mentoring and inter-organizational dynamics, and to identify any constraints to the mentoring process. 6. Communication Styles and Logistics As in any partnership or relationship, two-way communication is essential. Lack of communication, due to people’s time constraints, personalities, and technical breakdowns, can have a detrimental effect on the mentoring relationship. Trust and openness should be established early, not just between PVO directors, but at all personnel mentoring levels. This will lead to a greater sharing of information, and may help identify problem areas earlier in the project. Availability of personnel, including perceived availability, is a key to keeping channels of communication open. Promptness in responding to requests through E-mail or telephone, and to other inputs from the partner PVO is essential. Style of communication is important and, naturally, subject to individual personality differences, corporate cultural differences, as well as ethnic and cultural differences. Suggestions from mentors should always include not only what to do, but why to do something a particular way. Logistics of communications as well as style of communication are important elements of communication. Language barriers, telephone system breakdowns, and differences in computer platforms (IBM vs. MacIntosh), are just a few areas that may need to be addressed early in the partnership. 103 7. Integration of Technical and Managerial Skills This project found that technical assistance for specific project interventions should begin with mentoring in a few techniques / interventions in all project communities. Later phase-in of additional techniques and interventions is better than a later phase-in of the same interventions in additional communities. The project also started with a strong focus on field-level TA without simultaneous mentoring in management skills. Most personnel, when interviewed, stated that mentoring in management skills should come earlier in the project, possibly simultaneously with technical skills. This might be more possible if a phase-in of technical interventions is done, keeping in mind that a four-year project is very time￾limited, and interventions must therefore be phased in rapidly. It also will require an increased commitment of resources earlier in the project. The ARHC Senior Program Specialist suggests that the Mentor PVO should oversee and coordinate curricula of all field-level training sessions, especially at the beginning of the project. If not the primary trainer, he/she should coordinate content carefully with the Mentoree PVO’s project manager and/or trainer to assure thorough and accurate coverage of core content. Mentoring can take place through modeling training styles and providing technical content, and ownership of re-training sessions can then be transferred over time to the field NGOs and personnel. B. Points for reflection and further discussion In addition to lessons learned and recommendations made by project personnel, several issues emerged during the development of this report for which there seemed to be no ready answers. Perhaps further experience with other mentoring partnerships, in other contexts, will shed more light on these somewhat rhetorical issues. ß Should a Mentoring Partnership CS grant, by its very nature, presuppose that more time will be needed to accomplish project goals than in a traditional CS project, and allow such “extra” time to be built into the project timeframe? ß Assuming the Mentoree PVO is the primary grantee, and the Mentor PVO is the sub-grantee, how much accountability should the Mentor PVO have for any subsequent lack of achievement of project goals and objectives, or for areas of concern within the project? ß How does the Mentoree PVO manage office and field staff in order to assure that personnel are junior enough to acknowledge a need to be mentored, yet are senior enough to demonstrate the capacity and credibility to manage veteran health professionals in the host country? 104 ß How much should the Mentoree PVO rely on Mentor PVO input and TA versus bringing in other external consultants? ß A question raised in the Mid-Term Evaluation was whether the three-tiered mentoring structure of this project had too many layers to be as effective as a “flatter” mentoring structure might have been. Staff interviewed were mixed in their response to this issue. Some thought it was very workable, and in fact was working well. Others suggested that it has indeed involved too many layers, and have suggested that in future mentoring partnerships, the mentor PVO should try to work with a PVO who is the actual implementing partner in all aspects of the field project. The design of the project documented here is clearly “workable,” and they have had an impact in their target communities of Haiti. It remains to be seen whether alternate models might have fewer constraints and even greater successes. As noted previously, mentoring, by its very nature, adds an additional level which is not necessarily the most efficient in delivering services, but which should result in better quality and sustainability of services, and faster achievement of capacity building. 105 Appendix A: Table of Mentoring Activities Note: This table presents the primary mentoring tasks that have been planned and/or accomplished thus far in the project, and is not meant to be inclusive of all project activities. Program Year One Mentoring Level (see key below) Date Personnel Involved (see key below) Activity I – II – III Sep – Oct 97 ARHC PD & SPS, FOCAS ED & CSPM, MOH Develop plan for first 6 months of project II – III Beginning through LOP ARHC SPS, FOCAS HPD & CSPM Locate/develop CS tng guides, evaluation & supervisory mat’ls I Nov 97 ARHC PD FOCAS ED, HQ & BD Meet with FOCAS HQ staff & BD re: institutional strengthening II – III Oct-Nov 97 ARHC SPS & FOCAS CS PM Baseline KPC II – III Jan 98 ARHC SPS, FOCAS CSPM, NGO PMs, Health Workers Conduct rally posts, offer vaccines, vitamin A, GM/P I – II – III Jan – Mar 98 ARHC PD & SPS, FOCAS HQ & CSPM, NGO PMs DIP preparation and submission III Mar 98 ARHC SPS, FOCAS CSPM, NGOs Training in and conducting of first census II - III Mar 98 ARHC PD & SPS, FOCAS HPD & CSPM & NGOs Design HIS I April 98 FOCAS ED, ARHC ED & PD Formalizing cooperative mentoring agreement I April 98 FOCAS Sr. Staff, ARHC PD & BD ARHC PD introduces Sr. FOCAS staff to ARHC BD III April 98 – April 99 ARHC SPS, FOCAS CSPM Development of key educational messages I May 98 ARHC ED, FOCAS HQ & BD Visit of ARHC ED to FOCAS HQ and BD mtg re: fundraising strategies II – III May 98, then ongoing through LOP ARHC SPS, FOCAS HQ CSPM, NGOs, field staff Conduct manual analysis (field) and computer analysis (HQ) of project data II - III July 98 ARHC SPS, FOCAS CSPM, NGOs Training: micronutrients, factor analysis, growth monitoring, home visits II – III Several; see App D ARHC SPS, FOCAS HPD, CSPM, NGO PMs Conduct training with FOCAS/NGO field staff II – III Aug 98 ARHC SPS, FOCAS HPD & CSPM Develop evaluation mat’ls for pneumonia case mgt (translate and use pneumonia toolbox) II – III Ongoing through LOP ARHC PD & SPS Review progress in project implementation II – III July 98 FOCAS HPD & CSPM, NGO PMs Development of Annual Implementation Plan (AIP) 106 Program Year Two Mentoring Level (see key below) Date Personnel Involved (see key below) Activity II – III Oct 98, ongoing ARHC PD & SPS Development, procurement of training materials for CBIO approach III Several; see App D ARHC SPS, FOCAS HPD, CSPM, NGO PMs Conduct training with FOCAS/NGO field staff II Nov 98 FOCAS HPD & CSPM Budget development; field visit to Save the Children II – III Nov 98 ARHC SPS, FOCAS HPD & CSPM, NGOs Training: HIS, QI, Pneumonia Toolbox II – III Feb 99 ARHC PD & SPS, FOCAS HPD & CSPM & NGOs Training: technical capacity in CBIO methodology / Bolivia II – III Apr 99 FOCAS HPD & CSPM, NGOs, ARHC SPS Pre-Midterm Eval: Review action plan, HIS review II - III May 99 ARHC SPS, FOCAS HPD & CSPM Training: Begin HEARTH nutrition strategy III September 1999 ARHC SPS, FOCAS HPD & CSPM Develop/acquire tng mat’ls for diarrhea and Epi Info management Program Year Three Mentoring Level (see key below) Date Personnel Involved (see key below) Activity I – II – III Oct 99 ARHC SPS, FOCAS HPD, CSPM, NGOs Mid-Term Evaluation (MTE) I – II Dec 99 ARHC PD, FOCAS ED & HPD Review of MTE results III March 00 CSPM, NGOs Re-training in pneumonia mgt III March 00 FOCAS HPD, CSPM, NGOs visit HHF in Haiti Improved ARI case management, Revision of ARI QI Checklist I May 00 FOCAS ED, HPD, BD ARHC ED/PD ARHC attends FOCAS BD meeting I / II May 00 FOCAS ED & HPD, ARHC ED Review of FOCAS HQ adm & mgt issues: planning, finance, org. chart, personnel, communication I June 00 FOCAS HPD, ARHC ED & BD FOCAS attends ARHC BD meeting II July 00 ARHC / Bolivia, FOCAS HPD Training: technical capacity in CBIO methodology II – III Several FOCAS HPD, CSPM, NGOs Training, management sessions in Haiti I – II – III Jul-Aug 00 ARHC ED, FOCAS HPD, CSPM, NGOs TA in fundraising, CBIO review, pgm planning & reporting, nutrition pgm, mortality data analysis, HIS review 107 Projected Activities for 2000 / 2001 Mentoring Level (see key below) Date Personnel Involved (see key below) Activity II – III Early 01 FOCAS HPD, CSPM, NGOs Additional training in Hearth nutrition model II – III Early 01 ARHC SPS, FOCAS HPD, CSPM, NGOs Advanced CQI methods I – II Mar 01 ARHC HQ, FOCAS HPD Training: Work team mgt I – II Mar 01 ARHC HQ, FOCAS HPD Training: Medical supplies acquisition capacity I – II – III Mar 01 ARHC SPS, FOCAS HPD & CSPM & NGO PMs Researching / preparing grant applications, develop fundraising plan (field level) I May 01 ARHC ED, FOCAS HQ Review FOCAS HQ adm & mgt progress; strategic planning II – III Nov 01 ARHC SPS, FOCAS HPD & CSPM & NGOs SE? Strengthen KPC leadership II – III Nov 01 ARHC SPS & HQ FOCAS HPD & CSPM Strengthen project evaluation leadership I Once in 00 - 01 FOCAS ED, HPD & BD, ARHC ED & BD FOCAS board/staff attend ARHC Bd Mtg I Once in 00 - 01 ARHC ED, FOCAS ED & HPD & BD ARHC board/staff attend FOCAS Bd Mtg Key to Mentoring Level: I: Primarily HQ level involvement II: Primarily ARHC technical/managerial to FOCAS/USA & Haiti level III: Primarily FOCAS/USA & Haiti to NGO partner level Key to Personnel Abbreviations: FOCAS: HPD - Health Program Director CSPM - Child Survival Program Manager NGOs - Haiti Partner NGOs (MEI & OBDC) ARHC: PD – Program Director SPS – Senior Program Specialist BOTH: ED – Executive Director BD – Board of Directors HQ – Other HQ Personnel 108 Appendix B: Training Schedule for Field Staff, Years 1 – 2 Date to Implement Topic Who* Length of Training 3/98 Conducting a Census, Using QI Checklist HA’s announce census, HA’s and Superv’s conduct 2 days plus heavily supervised practicum 5/98 ARHC’s CBIO methodology (staff visit Bolivia), home visits, HIS, verbal autopsies, mortality review FOCAS Pgm Mgr, FOCAS Hlth Dir, NGO Pgm Mgrs Bolivia trip: 5 days Home visits: 3 days + field pract HIS: 3-5 days + field pract 7/98 GM/P, Micro-nutrient supplementation, deworming, Development of Educ Msgs Supervisors: Supervisory visits, use of QI checklists HA’s & Superv’s, NGO PMs GM/P, micronutrients, deworming: 4 days + field pract IEC messages: 3 days + field pract Supervision: 3-5 days 8/98 Immunizations (refresher) HA’s & Superv’s, NGO PMs 5 days 9/98 Family Planning / Child Spacing and Reproductive Health, IEC Msg Development HA’s & Superv’s, NGO PMs 10 days 11/98 Management of Diarrhea, IEC Msg Development HA’s & Superv’s, NGO PMs 5 days 2/99 Hearth Methodology Nutrition Monitors & Supervisors, Orientation in same for HA’s, Clinic Staff 2 weeks 4/99 Training of TBAs TBAs, Supervisors To Be Determined 6/99 Improved health ed methods (use of health ed cards, flipchart) HA’s, Supervisors, NGO PMs, Nutrition Monitors, Clinic Staff 5 days 8/99 Pneumonia Toolbox Tng ARI Standardized Case Mgt; Modification of ARI/Pneumonia IEC msg HA’s begin ARI mgt. Clinic staff begin improved ARI mgt Supervisors use QI checklist 15 days 10/99 Community Organization (As required by MOH) 5 days 12/99 Control of Epidemics, First Aid (As required by MOH) 10 days * ARHC Senior Program Specialist coordinated and facilitated most of these trainings, in conjunction with FOCAS CS Project Manager and Health Programs Director when possible. 109 ANNEX 2 – Mortality Report FOCAS Under-Five Mortality Rates, 1999 – 2002 In January 2003, 2002 birth and death data from the Vital Events Registries (VER) of OBDC and MEI were reviewed by MEI and OBDC Supervisors, Archivists, and one Program Manager, the FOCAS CS Program Director, Judy Gillens, Phil Moses of FOCAS, and Tom Davis of Curamericas. Data for the years 1999-2001 were then gathered by the Archivists in each organization over the next few days in the same manner. The under-five mortality rates were calculated for both organizations from 1999 to 2002 (see below). As Health Agents hear about vital events, the events are recorded in the VER. Also, data in the VER is updated every six to twelve months when the census data cards are updated with changes in the household composition and vital events not previously recorded during a repeat census. The Supervisors were confident that the information that they had on child deaths and births was complete and accurate. OBDC, 1999: Child Deaths: 10 Child Births: 148 U5MR: 68 per 1,000 2000, OBDC: Child Deaths: 50 Child Births: 494 U5MR: 101 per 1,000 2001, OBDC: Child Deaths: 55 Child Births: 456 U5MR: 121 per 1,000 2002, OBDC: Child Deaths: 28 Child Births: 595 U5MR: 47 per 1,000 This represents a decrease in the U5MR of 31% in four years for OBDC (from 1999 to 2002). OBDC added more territory each year than MEI, so some of the delay in decreasing deaths in the OBDC area may be related to this fact. MEI, 1999: Child Deaths: 25 Child Births: 133 U5MR: 188 per 1,000 2000, MEI: Child Deaths: 28 Child Births: 237 U5MR: 118 per 1,000 2001, MEI: Child Deaths: 27 Child Births: 260 U5MR: 104 per 1,000 110 2002, MEI: Child Deaths: 21 Child Births: 318 U5MR: 66 per 1,000 This represents a decrease in the U5MR of 65% in four years for MEI (from 1999 to 2002). Under-five Mortality Rate, FOCAS-Curamericas Mentoring Project 188 118 104 68 66 101 121 47 160 160 160 160 0 50 100 150 200 Year U5MR MEI OBDC DHS MEI 188 118 104 66 OBDC 68 101 121 47 DHS 160 160 160 160 1999 2000 2001 2002 111 Summary of Verbal Autopsy Data for January – November 2002 Child Deaths Forty-seven percent (23 of 49) of child deaths in the FOCAS program area were related to diarrhea and pneumonia, so we reviewed verbal autopsy data and other documentation on these two major causes of child deaths. Diarrheal Deaths in the Program Area: ÿ For OBDC, there were 7 diarrhea deaths from January – November, 2002. Two of those deaths were of children in the 1-11m old period, and five were of children in the 12-59m old period. ÿ For MEI, there were 6 deaths, all six in the at 1-11m old period. ÿ In general, the children who died were not malnourished. ÿ No relationship to gender. ÿ Diarrheal deaths do not seem to be related to transportation issues – diarrhea deaths were just as common in marginal as in rural areas. ÿ Many of the children had vomiting along with the diarrhea. ÿ Only about half of the children had been to a health facility during the illness. ÿ Most who did go to a health facility received medications. ÿ Most received counseling on use of ORS. ÿ Most saw a health agent during their illness. Recommendations on Diarrhea Activities: ÿ Begin doing one-day follow-up of children with diarrhea with signs of dehydration, severe disease and all children referred, and diarrhea with vomiting. ÿ Develop a form to document case management, referral, and follow-up of diarrheal cases in the same way that it is done for ARI cases. Make it a register-type form where more than one case can be put on each sheet of paper. ÿ Refer all children who have diarrhea and vomiting if the child is unable to keep any liquids down. ÿ Explain to all mothers how to give ORS if a child vomits it up (waiting 15 minutes and then giving the ORS more slowly with a spoon). ÿ If a child has had vomiting recently, stay and observe the mother rehydrating the child to see if the child can drink the ORS and keep it down. ÿ Improving the water sources and purification of water would probably help decrease diarrheal incidence. ÿ We considered zinc supplementation, but FOCAS felt that this was too expensive for now. ÿ Promote consumption of tomatoes (cooked and paste) in children on a regular basis (e.g., 3-5 times a week) since some data is showing a relationship between this and diarrheal deaths (and it can’t hurt). ÿ Make changes to the Supervisor Monthly Report form to report on the quality of the (new) Diarrhea Case Management forms (e.g., % of children referred properly). (Done) ÿ Add a section to report the quality of these forms on the Program Manager Monthly Report Form. (Done). 112 Pneumonia Deaths in the Program Area: ÿ In the OBDC area, there were six pneumonia child deaths, two of which were in the 0-7 day period, one in the 8-29 day period, one in the 1-11m period, and 2 in the 12-59m period. ÿ In the MEI area, there were four pneumonia child deaths, two in the 1-11m period, and 2 in the 12-59m period. ÿ In general, the children who died were not malnourished. ÿ No relationship to gender. ÿ Deaths do not seem to be related to CHAs who are unauthorized. (We did a 2x2 table to look at deaths by authorized and unauthorized CHAs, and the Odds Ratio was 1.0 – no relationship.) (N.B. CHAs authorized are those qualified to detect and begin treatment of suspected pneumonia with Clortrimoxazole). ÿ Pneumonia deaths may be related to transportation issues – pneumonia deaths were slightly more common in marginal than rural areas (6 vs. 4), and two of the mothers said that they did not take their child to the CHA or health facility during the illness preceding the child’s death. Only 60% went to a health facility. ÿ Community Health Agents only saw 6 of the 10 children who died during the illness preceding the child’s death. ÿ A review of 58 ARI Case Management forms showed that: · Only 19% of children with pneumonia, severe pneumonia, or severe disease were followed up within two days. · Over half (52%) either had follow-up at one month or later, or had not follow-up documented on the form. · 43% of children (3 of 7) who should have been referred during the initial contact were not referred. · 33% of children (2 of 6) who were the same or worse at follow-up (according to what the agent marked on the form) were not referred. · 40% of children whose respiration rate had increased were not referred. ÿ Last year, there were some stock-outs of cotrimoxazole due to decapitalization of the Health Agents’ supplies. Health Agents give away medicines to indigent families, and are not restocked for those since they have to turn in money for antibiotics. [These shortages have been regularly covered through private contributions; however, both NGOs have been provided with guidance on maintaining an inventory management and accountability system for greater long-term sustainability of such initiatives.] Recommendations on Pneumonia Activities: ÿ Make changes to the Pneumonia Case Management form. Remove elements that are not analyzed. Add to the form if the mother was educated on pneumonia prior to the child’s illness (using the Child Register to find that data). Make this form a register-type form where more than one case can be put on each sheet of paper. Specific changes that should be made include: · Add a spot for showing vitamin A treatment. · Add yes/no boxes for documentation of danger signs. (Currently, leaving a box blank means that the child does not have a danger sign. But when a box is left blank, you cannot tell if the child did not have a sign or if the HA simply forgot to fill out that part of the form.) 113 · Put the respiration rate cut-offs for rapid breathing under age category to remind HAs of that. · Change ‘last date of vitamin A’ to “Dose of Vitamin A in last 4 months?” · Add “or vomits everything” to the “cannot drink at all” danger sign. ÿ Add stock-outs to the Program Manager Monthly Report so that stock-outs on antibiotics can be dealt with in a timely manner. (Done) ÿ Develop a system for preventing stock-outs of cotrimoxazole. Consider giving HAs additional stocks of antibiotics each month so that they will have those available for indigent families. Have them keep track of families who receive free medicines so that Supervisors can do spot checks to assure that this system is not abused. ÿ Make changes to the Supervisor Monthly Report form to report on the quality of the Pneumonia Case Management forms (e.g., % of children referred properly). (Done) ÿ Add a section to report the quality of these Pneumonia forms on the Program Manager Monthly Report Form. (Done). ÿ Micheline to go over results with Agents and Supervisors and show examples from forms. Other General Recommendations for Prevention of Child Deaths: ÿ Develop a system for doing follow-up of children who go to clinics/hospitals outside of the OBDC/MEI system. ÿ Make changes to the child verbal autopsy form to capture more of the delays (delays in recognizing need for care, in making decision to seek care, in getting to a health facility, etc.) Include a narrative section near the beginning where the Supervisor can document the flow of events in detail (including times) of the case. Teach Supervisors to take good histories on this. ÿ Have HAs do quarterly external checks of the mortality data to see if local leaders know of any other child deaths not included in the Vital Events Register. ÿ Do verbal autopsies on all deaths where the cause of death is listed by the Health Agent has “not known.” Maternal Deaths There were two maternal deaths in the program area during 2002. One was an eclampsia death in the MEI area, and the other was of a woman who hemorrhaged and had a poor diet and signs of anemia. MMR for the program area for 2002 is 219 per 100,000 live births. (There were 913 live births in the program area in 2002.) ÿ One died in the hospital, and one appeared to have died at home. ÿ Both had received prenatal care. ÿ One died when the pregnancy was at term, the other one died during her second trimester. ÿ The woman who died of a hemorrhage had received advice from the HA, but did not accept it. She had four children previous to this pregnancy. ÿ Data on the current Maternal Verbal Autopsy form does not allow for analysis of many of the possible delays to receiving care. General Recommendations for Prevention of Maternal Deaths: ÿ There is too little data to make specific recommendations based on these two cases. However, other general changes should be made to improve data collection on these events. 114 ÿ Assure that all mothers know the EOC danger signs. (TBAs have now had training on these.) Develop flipcharts for this purpose. ÿ Make changes to the maternal verbal autopsy form to capture more of the delays (delays in recognizing need for care, in making decision to seek care, in getting to a health facility, etc.). Include a narrative section near the beginning where the Supervisor can document the flow of events in detail (including times) of the case. Teach Supervisors to take good histories on this. ÿ Have HAs do quarterly external checks of the mortality data to see if local leaders know of any other maternal deaths not included in the Vital Events Register. ÿ Do verbal autopsies on all deaths where the cause of death is listed by the Health Agent has “not known.” ÿ Make a change to the Vital Events Register: Add a column to easily identify women who have died from causes related to pregnancy and delivery (during or before 45 days following delivery). (Done) Thomas P. Davis Jr., MPH Senior Program Specialist Curamericas January, 2003 115 ANNEX 3 -- Malnutrition Report (excerpt from full tracking table) FOCAS Indicator Tracking Table (July 2002) NOTE: The baseline levels for this table are based on a KPC Survey conducted in November 1997. The objectives were included in the FOCAS/Curamericas DIP produced in February 1998. The final levels are based on two sources: (1) A KPC survey using lot quality assurance sampling in September 2001 (by Dr. Antoine Augustin) and (2) a review of children and women’s registers conducted in July 2002 (based on a census of all children and women in the project area, updated monthly by each health agent) using systematic sampling. Data from this study should not be compared to the targets set in the 2000 cost-extension proposal since only one-year of activities would have elapsed since that baseline – too little time to measure changes in K, P, and C. – Tom Davis, MPH Intervention OBJECTIVE & BASELINE LEVELS (1997) INDICATOR Baseline Levels (w/CIs) Final Levels (w/CIs) Achieved / Comments Nutritional Improvement Increase the average weight-for-age z-score of children under two years of age by one half of one standard deviation (WAZ change of +0.5) by end of the project period (4 yrs.) Average weight￾for-age z-score of children from 6-23 months in the project area. No baseline was established for this indicator No final level was established for this indicator. See graph of malnutrition for trend. In the MEI area, there was a 46% drop in moderate and severe malnutrition from October 98 to April 2002 based on a trend line of monitoring data for all preschool children for that period. In the OBDC area, there was a 64% decrease in the moderate and severe malnutrition during the same period based on the monthly monitoring data. Selection bias, of course, may have affected this estimate. (See attached.) 116 Intervention OBJECTIVE & BASELINE LEVELS (1997) INDICATOR Baseline Levels (w/CIs) Final Levels (w/CIs) Achieved / Comments Nutritional Improvement (cont.) Decrease the % of children under 2 who are less than one standard deviation below the median weight￾for-age by 20%. Percentage of children 6-23 months in the project area who are less than one standard deviation below the median weight-for-age. No baseline was established for this indicator No final level was established for this indicator. For children 0-59m weighed based on trendline from monitoring data: OBDC: 5% decrease in M1 MEI: 9% decrease in M1 There was not a large drop in the proportion of children who were in the M1category for either organization. (See attached.) Increase the avg. weight￾for-age z-scores of NDF participants by one standard deviation (WAZ change of +1.0), comparing WAZ at the beginning of the NDF with WAZ 12 months after the NDF. Change in weight￾for-age z-score of NDF participants, comparing beginning z-score with 12 months afterward. FOCAS suspended NDFs in 2000 after several months of operation. No final level was established for this indicator. NDFs suspended in 2000. N/A 117 Proportion of Children Weighed Who Are Malnourished or at Risk of Malnutrition: MEI Area For M2+M3 Line: y = -0.0012x + 1.5005 R 2 = 0.3468 For M1 Line: y = -0.0008x + 1.3668 R 2 = 0.1226 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 50.0% Oct-98 Apr-99 Oct-99 Apr-00 Oct-00 Apr-01 Oct-01 Apr-02 Month % of children % M2+M3 % M1 Linear (% M2+M3) Linear (% M1) Children in M2+M3: Number of Months Slope Decrease in Percentage Points Starting Value Final Value Percentage Decrease 42 -0.0012 -5.0% 10.9 5.86 -46% Children in M1: Number of Months Slope Decrease in Percentage Points Starting Value Final Value Percentage Decrease 42 -0.0008 -3.4% 39.5 36.14 -9% 118 OBDC: Proportion of Children Malnourished For M2/M3 Line: y = -0.0016x + 1.9963 R 2 = 0.5526 For M1 Line: y = -0.0004x + 0.8588 R 2 = 0.0473 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% Oct-98 Apr-99 Oct-99 Apr-00 Oct-00 Apr-01 Oct-01 Apr-02 Months % Malnourished % in M2+M3 % in M1 Linear (% in M2+M3) Linear (% in M1) For M2+M3: Number of Months Slope Decrease in Percentage Points Starting Value Final Value Percentage Decrease 119 42 -0.0016 -6.7% 10.5 3.78 -64% For M1 Line: Number of Months Slope Decrease in Percentage Points Starting Value Final Value Percentage Decrease 42 -0.0004 -1.7% 36 34.32 -5% ANNEX 3 (continued) -- Anthropometry Data FOCAS Haiti KPC, August 2003 TABLE 1. MALNUTRITION RATES OF CHILDREN 6-23 MONTHS OLD (WEIGHT/AGE) IN MEI AND OBDC AREAS. ZONES Z-SCORE BELOW –2SD (INCLUDES –3SD) Z-SCORE BELOW -3SD N MEI 6% (5/79) CI: 2.4 -- 14.8% 1% (1/79) CI: 0.1 -- 7.8% 79 OBDC 15% (16/107) CI: 9.0-23.4% 0% (0/107) 107 NATIONAL 21.6 (DHS 2000) 5.1 (DHS 2000) SOURCE FOR NATIONAL DATA: DHS 2000 (EMMUS III) TABLE 1.1. MALNUTRITION RATES OF CHILDREN 6-59 MONTHS OLD (WEIGHT/AGE) IN MEI AND OBDC AREAS. RESULTS FROM SIBLINGS OF INDEX CHILDREN SELECTED FOR KPC INTERVIEWS. ZONES Z-SCORE BELOW –2SD (INCLUDES Z-SCORE BELOW -3SD N 120 –3SD) MEI 2.7% (1/37) 2.7% (1/37) 37 OBDC 15.2% (7/46) 0% (0/46) 46 121 ANNEX 4 – Indicator Tracking Table FOCAS Indicator Tracking Table (Reflecting Indicators Measured During the Final KPC August 2003) INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) OBDC: 52% (CI: 44.3-60.3%) 80% OBDC 70% * (100/142) CI: 62- 78% NUTRITION Increase the % of children who were breastfed within one hour after birth from 52% to 65% in the OBDC project area and 57% to 70% in the MEI project area. % of infants/children ( < 24 m) who were breastfed within the first hour after birth. MEI: 57% (CI: 49.4-65.3%) 75% MEI 73% (78/107) CI: 63% -- 81% 34 Source: “Report on Baseline KPC Survey: October 28 to November 21, 1997, FOCAS / MEI / OBDC Child Survival Project”. November 1997. Arsene Ferrus, MD, MPH, FOCAS Project Manager, Thomas P. Davis Jr., MPH and Julie Mobley, MPH, Project Consultants ARHC 34 For each proportion the numerator and denominator are listed in parentheses followed by the 95% confidence interval. The confidence intervals were calculated using Epi-Info 2002’s FREQUENCY function in the Data Analysis module. 122 INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) OBDC, 0-3m: 5% (CI: 0-10%) NONE Targets and Indicator Modified in 2001 0-3M: 50%* (10/20) CI: 27 --73% OBDC, 0-5m: 4%(CI: 0-9%) 64% OBDC 0-5M: 44% * (16/36) CI: 28 -- 62% MEI, 0-3m: 3% (CI: 0-6.9%) None MEI: 0-3M: 69.2%* (9/13) (CI: 38.6% - 90.9%) Increase the percentage of children who were given only breastmilk during the first 6 months from: 56% to 64% in EDJ 61% to 69% in BLM % of infants less than 6 months, who are being given only breastmilk.35 MEI, 0-5m: 2% (CI: 0-5.5%) 69% MEI MEI: 0-5M: 50%* (11/22) (CI: 28.2% - 71.8%) 58% OBDC 42% OBDC Targets Modified in 2001 OBDC: 24% (13/54) CI:14 – 38% Decrease the % of children who were ever bottlefed from: · 58% to 29% in ODBC & MEI % of children 0-11m who are being fed with a bottle 58% MEI 42% MEI MEI: 30% (13/43) CI: 17% -- 46% 35 At the beginning of LOP, MOH Haiti used 0-3 months for its exclusive breastfeeding (EBF) indicator. International standards define EBF as 0-5 months. This report reflects both indicators for clarity. 123 INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) OBDC: 19% (CI: 4.6-32.7%) OBDC 70% Targets modified in 2001 78%* (31/40) CI: 62 – 90% Increase the % of children 6-72m who have received two doses of Vitamin A per year from 19% to 75% in the OBDC area and 21%36 to 75% in the MEI project area. % of all children 17- 23m who have received at least two doses of Vitamin A in the past 12 months. MEI: 45% CI: 26.2-63.1%) MEI 80% Targets modified in 2001 83%* (24/29) CI: 64 – 94% OBDC: 54% (CI: 43.6-64.3%) OBDC: 64% Target Modified in 2001 74%* (104/141) CI: 66 -- 81% Increase the % of children who had their growth monitored for the first time during the first month of life from 54% to 90% in the OBDC area and 37% to 85% in the MEI project area. % of children 2-23m of age who had their growth monitored for the first time at <1m of age (of children with a growth card). MEI: 37% (CI: 27.9-46.5%) MEI: 62% Target Modified in 2001 82%* (88/107) CI: 74% -- 89% 36 Please note that this was erroneously reported as 21% in the indicator table of the baseline KPC report. The actual level was 45% for MEI. 124 INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) OBDC: 11% (CI: 3.2 -19.7%) OBDC:70% Target Modified in 2001 81% * (58/72) CI: 70 – - 89% Increase the % of children under two who were weighed six times per year in the past 12m from 11% to 80% in the OBDC area and 32% to 90% in the MEI project area. % of children 12 -23m weighed at least six times in the past 12 months. MEI: 32% (CI: 19.5 -43.7%) MEI: 83% Target Modified in 2001 92%* (49/53) CI: 82 -- 98% OBDC: Same/More (including Breastmilk): 63% (CI: 55.1 -70.8%) NONE CI: OBDC: SAME/MORE (including Breastmilk): 100% (55/55)94 – 100% OBDC Same /More liquids(other than Breastmilk) 43% (CI: 31.2 -54.1%) OBDC: 72% Target Modified in 2001 OBDC SAME/MORE (Other than Breastmilk) 91%* (50/55) CI: 80 -- 97% DIARRHEA CASE MANAGEMENT Increase the % of children who were given the same amount or more liquids (other than breastmilk) during diarrheal episodes from 63% to 80% in the OBDC area and 69% to 80% in the MEI area. % of children 0 -23m who were given the same amount/more liquids (other than breastmilk) during a recent diarrheal episode. MEI: Same/More: 69% (CI: 58.8 -79.1%) NONE MEI:SAME/MORE (including Breastmilk) : 100% (27/27) CI: 87 – 100% 125 INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) More liquids 46% (CI: 33.6-58.9%) : MEI: 67% Target Modified in 2001 MEI: SAME/MORE (Other than Breastmilk) 85%* (23/27) CI: 66 -- 96% OBDC: 58%’ (CI: 49.0-67.7%) OBDC: 67% Target Modified in 2001: OBDC 64% (35/55) CI: 50 – 76% Increase the % of children who were given the same amount or more food during diarrheal episodes from 59% to 75% in the OBDC area and 55% to 75% in the MEI area. % of children 0-23m who were given the same amount/more food during a diarrheal episode. MEI: 54% (CI: 44.4-64.1%) MEI: 50% MEI: 82% (22/27) CI: 61.9 – 93.7% OBDC: 46% OBDC: 70% Targets Modified in 2001 OBDC: 91% (133/146) CI: 85 -- 95% Increase the % of mothers who say that they can easily find ORS in their community. (Indicator added in 2001) % of mothers who say they can easily find ORS in their community. MEI: 46% MEI 70% MEI: 90% (96/107) (CI: 82.3 - 94.8%) 126 INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) OBDC: 53% (CI: 41.2-64.6%) OBDC: 60% Targets Modified in 2001 OBDC: 53% (19/36) CI: 36 -- 70%) PNEUMONIA CASE MANAGEMENT Increase the % of mothers who seek assessment or medical tx for their child with cough and rapid, difficult breathing from 53% to 70% in the OBDC project area and from 44% to 65% in the MEI project area. % of mothers who seek medical assessment or tx for their child < 24m with cough and rapid, difficult breathing in the past 2 weeks. MEI: 44% (CI: 33.3-54.0%) MEI: 50 % MEI: 36 % (4/11) CI: 10.9 – 69.2% OBDC: 16.7% OBDC: 83% Targets Modified in 2001 OBDC: 70% * (37/53) CI: 56 -- 82% IMMUNIZATION Increase the % of infants who are fully immunized from 31% to 80% in the OBDC project area and from 67% to 85% in MEI. Percentage of all children 12-23 m who received BCG, OPV3, Polio3, and Measles before their first birthday..(Card confirmed) MEI: 50.9%% MEI: 92% MEI: 67% (29/43) CI: 51 -- 81% Child Spacing Objectiv es Increase the % of mothers who are using a modern contraceptive method from 13% to 25% in the OBDC project area and from 15% The percentage of non￾pregnant mothers of children 0-23m of age who desire no more children in the next two OBDC: 12.5% (CI: 7.5-17.4%) OBDC: 40% Targets Modified in 2001 OBDC: 57%* (83/145) CI: 49 -- 65% 127 INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) to 30% in MEI. years, or are not sure, who are using a modern method of child spacing MEI: 15% (CI: 9.3-20.7%) MEI: 40% 50%* (50/101) CI: 39 -- 60% OBDC: 77% OBDC: 89% Targets Modified in 2001 OBDC: 80% (118/148) CI: 72 -- 86% Maternal and Newborn Care Increase the proportion of mothers who had a qualified health professional attend their last birth from: · 77% to 89% in ODBC · 74% to 82% in MEI % of mothers of children 0-23 months who had a trained health professional attend to the birth of her child. MEI: 74% Baselines from 2001 KPC MEI: 82% MEI: 76% (81/107) CI: 66.5% - 83.5% 128 INTERVENTION OBJECTIVE & BASELINE LEVELS (1997) INDICATOR 1998 DIP Baseline Levels34 (w/CIs) 1998 Targets FINAL RESULTS FROM KPC 2003 (* indicates statistical significance) Confidence intervals are 95% unless otherwise noted) OBDC: 61% of those w/cards (CI: 40.9-80.8%); 9% of all mothers (CI: 4.7- 14.0%) OBDC: 55% Targets modified in 2001 OBDC: 26% *(Of all mothers) (38/148) CI:19% -- 34% Increase the % of pregnant women who received two doses of TT (or more) before the birth of her youngest child from 9% to 30% in the OBDC project area and from 11% to 30% in MEI. Percentage of mothers who received at least two tetanus toxioid injections (card confirmed) before the birth of the youngest child less than 24 months of age. MEI: 61% of those w/cards (CI: 42.6-78.8%); 11% of all mothers (CI: 6.3- 16.4%) MEI: 42% MEI: 31% *OF ALL MOTHERS (33/107) CI: 22 -- 40% 129 Indicators to be measured outside of the KPC (See Final Evaluation Report) Intervention Objective Indicator Nutrition Increase the avg. weight-for-age z-scores of Nutrition Workshop participants by 0.5, comparing WAZ at the beginning of the workshop program with WAZ 12 months after completion. Change in weight-for-age z-score of workshop participants, comparing beginning z-score with 12 months afterward. Pneumonia Case Mgmt. Significantly increase the proportion of children w/ARI assessed and treated properly by CHAs and clinic personnel from: ß 75% to 82% in EDJ ß 63% to 75% in BLM Number of children w/ARI assessed and treated properly by Health Agents and clinic personnel. Quality of SCM will be quantified through use of an observational checklist. Child Spacing /Family Planning Significantly improve the FP counseling done by health center staff (both clinics) through training in counseling skills. Percentage of appropriate health center staff trained in FP counseling; percentage score on supervisory checklist. 130 ANNEX 5 -- Water Quality Study Preliminary Report Water Quality of Selected Water Sources in Haiti Miami University and FOCAS July 14 to 26, 2002 From July 14 to July 21 (2002), a research team composed of personnel from Miami University (Oxford, Ohio) and from FOCAS (a health organization serving a portion of the Haitian population) analyzed 36 water samples from a variety of water sources that provide water for Haitian households. The purpose of the chemical and bacterial analyses was to provide an estimate of the safety of the water for drinking purposes. The personnel from Miami University were part of a field course – “Environment and Culture of Haiti.” The research team was divided into two groups, and each group was guided to sites selected by FOCAS personnel. Field tests were conducted by colorimeters (HACH) for pH (H+ ), dissolved oxygen (O2), nitrate (NO3 - ), phosphate (PO4 -4 ), sulfate (SO4 -2 ), dissolved iron (Fe+2 and Fe+3), and sulfide (H2S o , and HS- and S --). Tests conducted at night included alkalinity (by acid titration, pH meter, and Gran analysis), total hardness (Ca++ and Mg++) and calcium (Ca++). Bacterial populations were assessed using two membrane filtration/incubation methods (agar growth techniques) and a colorimetric technique. Each team also questioned end-users of the water to assess methods of in-situ treatment, time for collection and transport, typical uses of the water, attitudes about water quality, and other information. From July 22 to July 26 (2002), research continued by two students from Miami University (Oxford, Ohio) and from FOCAS (a health organization serving a portion of the Haitian population). These students analyzed 14 more water samples from a variety of water sources that provide water for Haitian households. The purpose of the chemical and bacterial analyses was to provide an estimate of the safety of the water for drinking purposes. The personnel from Miami University were part of a field course – “Environment and Culture of Haiti.” Additional sites were visited for 4 days to collect more field data in the area. Field tests were conducted by colorimeters (HACH) for pH (H+ ), dissolved oxygen (O2), nitrate (NO3 - ), phosphate (PO4 -4 ), sulfate (SO4 -2 ), dissolved iron (Fe+2 and Fe+3), and sulfide (H2S o , and HS- and S --). Tests conducted at night included alkalinity (by acid titration, pH meter, and Gran analysis), total hardness (Ca++ and Mg++), and calcium (Ca++). Bacterial populations were assessed using two membrane filtration/incubation methods (agar growth techniques) and a colorimetric technique. Each team also questioned end-users of the water to assess methods of in-situ treatment, time for collection and transport, typical uses of the water, attitudes about water quality, and other information. Overall Preliminary Conclusions: 131 Inorganic chemistry: Nitrate levels were high in the following samples: 7221200 (10.4 mg/l), 7221315 (9.47 mg/l), and 7231220 (17.03 mg/l). This is most likely due to agricultural run-off and/or wastewater contamination. Sulfate levels reached the maximum limit (80 mg/l) of the colorimeter in samples 7250925 and 7261130. Additionally, sulfide levels were high (.15 mg/l) in sample 7241030. This would account for the smell of sulfur that the locals commented on. Bacteria: Nearly all samples (97%) had measurable quantities of Enterococci or Escheriachia coli (two bacteria associated with poor water quality). A modified Manja test indicated that 78% of the samples had bacteria associated with poor water quality. Only one sample of the 36 samples tested contained no measured indicator of contamination. Survey results: Most users were aware and concerned about water quality. Water from cisterns and holding tanks (reservoirs) typically added variable amounts of an oxidant (Clorox or a swimming pool oxidant) to kill bacteria, but they were uncertain of its effectiveness. Most users distinguished end uses of water (drinking versus washing). Tables of data and graphs of these data include: Table 1. Water Quality Analysis Summary Table 2. Inorganic Chemistry – Data Figure 1. Oxygen concentration of samples. All samples are aerobic. Figure 2. Bacterial Contamination: Contamination Index combines results of the MEI and mTEC cultures and the Manja tube analyses. Figure 3. Comparison of Agar / culture methods. Multiple methods of analysis are needed to evaluate water quality. A more detailed report with full descriptions of methods, photographs, references, and discussion of results will follow. Submitted by: Dr. Mark R. Boardman, Geology Department, Miami University, Oxford, OH Dr. Avram Primack, Geography Department, Miami University, Oxford, OH Dr. Sandra Woy-Hazleton, Institute of Environmental Sciences, Miami Univ., Oxford, OH Dr. Judy Gillens, FOCAS, Director of Haitian Development Programs, Haiti Table 1. Water Quality Analysis Summary July 14-21, 2002 Sector Locality Spring Name Bacterial Index * WATER_TYPE 132 1 Jalousie Mena 0 Cistern 2 Planchet Marron Dian Marron Dian 0 Spring 3 Santo Tete de L'Eau, Pelerin 1 CAMEP reservoir 4 Santo De Semite De Semite 1 Cistern 5 Fessard Des Rivieries Grabe 1 Stream 6 La Boule Boutiviller Haute La Boule 1 Cistern 7 La Plaine 1 Water post 8 Grenier Redoute,Mason, Bwa Mason Ti Source 2 Spring 9 La Boule La Boule 12 Desire 2 Well 10 Santo Jalousie 4 Cistern 11 Santo Loe 4 Secondary 12 Santo La Chaume Pelerin 4 4 Spring 13 Planchet Ti Source Ti Source 4 Stream 14 Grenier Chevalier, Corail Pierre Louis 5 Spring 15 Grenier Obery Obery 5 Spring 16 Pyron Aime Bastien 5 Cistern 17 Grenier Gimpa Gimpa 5 Spring 18 Santo Jalousie 5 Cistern 19 Grenier House near clinic that draws from Grenier Spring 6 Cistern 20 Pyron Source Careau 6 Stream 21 Pyron Aime Bastien 6 Cistern 22 Grenier Reserve 6 Cistern 23 Grenier Private house near Grenier 6 Secondary 24 Santo Monovile 6 Cistern 25 Santo Bas-Calvaire Kajou 1 6 Cistern 26 Fessard Borgella 6 Spring 27 Santo Santo 6 Cistern 28 Grenier Galat St. Laurant 6 Cistern 29 Grenier Grenier spring 7 Spring 30 Fessard Sanyon Source Tises 7 Spring 31 Francois Francois Source Francois 8 Stream 32 Santo Abaco Tedlo 9 Secondary 33 Pyron Sans Souci 10 Cistern 34 Santo Flippo Flippo 10 Cistern 35 Santo Ceresier 10 Supply pipe 36 Santo Bas-Calvaire Kajou 2 10 Cistern * Bacterial Index 0 No Contamination 1 - 4 Contaminated 5 - 8 Highly Contaminated 9 - 12 Very Highly Contaminated Prepared by: Department of Geology, Miami University, Oxford, OH 45056, USA 133 Table 2. Inorganic Chemistry – Data Sample Site Number x y Time (mddhhmm ) PO4 (ppm ) Fe (ppm ) SO4 (ppm ) NO3 (ppm ) pH O2 (ppm ) S-- (ppm ) Alkalinit y (meq/kg) Ca (pp m) Hardness (ppm CaCO3) Bacterial Index log mTEC target cells (red) (per 100 ml) log MEI target cells (blue) (per 100 ml) 1 -72.292 18.50927 7151320 0.95 0.03 0.3 0.97 7.8 6.8 0.01 4.71 3.8 2.23 0.60 2 -72.3401 18.48975 7151640 0.19 0.00 0.0 4.50 7.4 5.2 0.00 4.49 6.5 1.57 1.94 3 -72.3407 18.49102 7160915 0.09 0.00 0.0 1.90 9.0 5.7 0.00 0.75 24 60 5.7 1.72 0.00 4 -72.3226 18.48928 7160930 0.49 0.06 0.0 1.90 8.0 6.7 0.02 1.12 32 80 9.8 1.76 4.00 5 -72.3498 18.48894 7161015 0.16 0.00 0.7 5.63 7.9 6.5 0.00 3.41 96 220 4.6 0.78 1.85 6 -72.3045 18.48641 7161045 0.03 0.00 0.0 0.00 8.2 7.7 0.02 3.31 120 320 5.7 2.03 2.69 7 -72.3519 18.48911 7161100 0.17 0.03 0.0 4.23 7.6 5.2 0.02 3.62 104 260 5.0 0.30 1.71 8 -72.3693 18.48544 7161230 0.35 0.00 0.3 4.70 7.6 7.6 0.01 5.88 96 240 2.3 0.30 0.00 9 -72.3256 18.48688 7161230 0.11 0.00 0.0 0.83 7.7 4.3 0.01 0.84 24 80 6.0 0.00 4.00 10 -72.3253 18.48695 7161305 0.23 0.01 0.0 1.27 7.8 5.1 0.00 0.81 24 80 4.9 1.76 2.14 11 -72.369 18.4878 7161315 0.29 0.01 6.3 4.53 7.6 6.4 0.00 3.79 96 240 5.2 0.00 2.20 12 -72.3289 18.48304 7161400 0.09 0.04 0.0 0.77 8.1 2.4 0.01 0.89 32 60 6.3 0.48 1.79 13 -72.3333 18.48364 7161436 0.21 0.00 0.0 3.23 7.8 7.3 0.01 2.03 64 160 5.8 2.81 0.00 14 -72.2867 18.50223 7171025 0.29 0.02 0.0 3.67 7.7 8.1 0.01 4.57 120 280 1.1 0.00 1.11 15 -72.2912 18.51082 7171035 0.29 0.00 0.0 1.27 8.2 4.7 0.00 3.92 128 280 8.5 4.00 2.51 16 -72.2905 18.50944 7171105 2.70 0.03 80.0 0.87 8.3 7.1 0.01 4.27 104 260 5.5 1.23 2.25 17 -72.294 18.51355 7171130 0.35 0.00 12.0 1.70 8.5 6.7 0.02 5.44 112 340 1.3 0.00 1.34 18 -72.2934 18.51368 7171200 0.28 0.02 0.0 1.27 8.7 6.8 0.00 3.98 128 240 9.6 3.74 2.87 19 -72.2955 18.51851 7171240 2.75 0.05 10.0 7.00 7.5 6.2 0.00 7.20 176 420 9.7 4.00 3.70 20 -72.3004 18.51559 7171300 0.59 0.01 1.0 0.47 8.4 6.8 0.00 4.03 120 260 4.3 1.40 1.89 21 -72.3001 18.51602 7171330 0.00 0.00 2.3 0.90 8.7 6.8 0.04 0.83 24 80 0.5 0.00 0.48 22 -72.2995 18.51611 7171405 0.25 0.01 49.0 0.70 9.0 6.7 0.01 5.02 112 420 5.7 0.30 2.36 23 -72.2941 18.50933 7171430 0.48 0.04 12.0 4.33 8.1 7.5 0.04 6.02 112 260 5.7 1.91 1.76 24 -72.2941 18.50933 7171500 0.34 0.02 26.0 2.67 8.4 7.1 0.00 6.99 104 400 9.6 4.00 2.64 25 -72.3378 18.48258 7181030 0.26 0.02 0.0 2.63 7.9 7.0 0.01 3.73 72 180 6.3 0.88 2.40 26 -72.3544 18.47268 7181100 0.20 0.02 3.0 1.83 8.1 6.9 0.02 3.22 64 180 8.5 3.74 1.72 27 -72.34 18.476 7181210 0.38 0.00 0.0 0.53 7.9 7.8 0.01 3.09 64 140 1.2 0.00 1.19 28 -72.3546 18.48313 7181225 0.28 0.02 2.7 1.17 7.9 7.0 0.00 5.86 96 300 6.8 1.90 1.89 29 -72.3097 18.50848 7190945 0.16 0.00 2.7 1.20 8.0 5.4 0.01 0.84 24 80 5.6 1.52 1.08 30 -72.293 18.50065 7191030 0.34 0.00 3.0 3.73 7.6 5.7 0.00 4.33 104 260 4.2 1.72 0.48 31 -72.3374 18.47683 7191030 1.07 0.04 0.7 4.70 7.7 7.7 0.02 1.90 64 140 3.7 1.69 0.00 32 -72.3365 18.47722 7191105 0.26 0.01 1.0 2.87 8.1 7.6 0.01 2.04 72 160 0.0 0.00 0.00 33 -72.3467 18.49699 7191130 0.33 0.02 7.7 1.30 8.1 4.2 0.00 1.31 32 80 5.9 4.00 0.90 34 -72.3299 18.4954 7191300 0.15 0.06 4.0 1.00 8.6 6.7 0.02 5.72 128 260 2.0 1.28 0.70 35 -72.3302 18.50628 7191345 0.29 0.01 9.0 0.87 8.5 5.6 0.01 1.71 16 40 1.1 0.48 0.60 36 -72.2905 18.59539 7191400 0.98 0.02 17.7 3.23 7.6 6.0 0.00 6.75 96 300 0.6 0.60 0.00 134 Figure 1. Oxygen Content Figure 2. Bacterial Contamination. The contamination index includes equal possible weight from the three methods to evaluate contamination used in this study. Dissolved Oxygen 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 Sample Dissolved Oxygen (ppm or mg/l) saturated Bacterial Contamination 0 4 8 12 1 4 7 10 13 16 19 22 25 28 31 34 Sample Contamination Index very highly contaminated highly contaminated contaminated 135 Figure 3. Agar Culture Techniques: No single method of analysis is sufficient to determine water contamination. Multiple methods of evaluation are needed. Agar Culture Techniques 0 1 2 3 4 5 0 1 2 3 4 5 Modified mTEC (log bacteria / 100 ml) MEI (log bacteria / 100 ml) 136 ANNEX 6 – Training Schedules Training Schedule for Field Staff, Years 1 – 2 (from A Report on the First Three Years of a Mentoring Partnership, Julie Mobley, MSPH, October 2000) Date to Implement Topic Lead Trainer Who* Length of Training 3/98 Conducting a Census, Using QI Checklist Tom Davis, MPH Curamericas Senior Program Specialist HA’s announce census, HA’s and Superv’s conduct 2 days plus heavily supervised practicum 5/98 ARHC’s CBIO methodology (staff visit Bolivia), home visits, HIS, verbal autopsies, mortality review Tom Davis, MPH Curamericas SPS FOCAS Pgm Mgr, FOCAS Hlth Dir, NGO Pgm Mgrs Bolivia trip: 5 days Home visits: 3 days + field pract HIS: 3-5 days + field pract 7/98 GM/P, Micronutrient supplementation, deworming, Development of Educational Messages Supervisors: Supervisory visits, use of QI checklists, Barrier Analysis Tom Davis, MPH Curamericas SPS HA’s & Superv’s, NGO PMs GM/P, micronutrients, deworming: 4 days + field pract IEC messages: 3 days + field pract Supervision: 3-5 days 8/98 Immunizations (refresher) Dr. Arsene Ferrus, CS Program Manager HA’s & Superv’s, NGO PMs 5 days 8/98 Pneumonia Toolbox Training ARI Standardized Case Mgt; Modification of ARI/Pneumonia IEC messages Tom Davis, MPH Curamericas SPS & Local Consultant HA’s begin ARI mgt. Clinic staff begin improved ARI mgt Supervisors use QI checklist 15 days 9/98 Family Planning / Child Spacing and Reproductive Health, IEC Msg Development Dr. Arsene Ferrus, CS Program Manager HA’s & Superv’s, NGO PMs 10 days 11/98 Management of Diarrhea, IEC Msg Development Dr. Arsene Ferrus, CS Program Manager HA’s & Superv’s, NGO PMs 5 days 2/99 Hearth Methodology Drs. Gretchen & Warren Berggren w/staff follow-up Nutrition Monitors & Supervisors, Orientation in same for HA’s, Clinic Staff 2 weeks 4/99 Training of TBAs Local Consultant TBAs, Supervisors To Be Determined 6/99 Improved health ed methods (use of health ed cards, flipchart) Tom Davis, MPH Curamericas SPS HA’s, Supervisors, NGO PMs, Nutrition Monitors, Clinic Staff 5 days 10/99 Community Organization (?) (As required by MOH) 5 days 12/99 Control of Epidemics, First Aid (?) (As required by MOH) 10 days 137 Training Schedule for Field Staff, Years 5-6 TRAINING SCHEDULE 2001 - 2003 TRAINING TOPICS YEAR 1 YEAR 2 1 2 3 4 1 2 3 4 Mortality Rate / Verbal Autopsies X X ARI X Health Information System X Diarrhea Mngt. / Environmental Hygiene X Health Education and Animation Technique X Community Organization X Growth Monitoring / Promotion X Micronutrients X Quality Improvement Techniques X TBA ( Matwons) X X Monitoring & Evaluation (Target Coverage Charts) X RH/FP X Refresher Days ARI X Diarrhea X TCC X