1 Catholic Relief Services Honduras Program Final Evaluation Community-Based Child Survival Intibuca, Honduras Cooperative Agreement Number: FAO-A-00-99-00012-00 Grant Period: 9/30/1999 to 1/29/2004 Date of Submission: February 2004 Prepared by: Frank Anderson MD, MPH Alfonso Rosales MD, MPH&TM Alexander Flores, MD Judith Galindo, RN, MPH Sara Russell, BSN, RN 2 A. Summary Honduras, with a per-capita GNP of US $730, is the third poorest country in Latin America. Although the health indicators have improved over the last two decades, they remain among the weakest in the region. The Region 2, where the Intibuca department is located, is among the poorest regions in Honduras.1 During the past four years CRS and COCEPRADII in close collaboration with the Secretary of Health, have been implementing a program financed by USAID in 95 communities spread over 3 municipalities of the department of Intibuca. The four-year project (1999-2003) is designed to reduce mortality in 11,324 children under five years old and 11,310 women of childbearing age in 95 communities in six municipalities of Intibucá through the following interventions: q Safe Motherhood and Newborn Care Emphasis is placed on obstetric first aid at the community and health center level and the creation of community emergency transportation plans to first and second level SOH facilities q Lacto-Amenorrheic Method Natural family planning education initiatives to increase knowledge on this particular method, aimed at birth spacing. q Pneumonia Case Management Project efforts are focused on improving case detection and care-seeking behavior at the community level and increasing quality of attention through Standard Case Management (SCM) training and links with Secretary of Health first level facilities q Diarrhea Case Management Interventions were targeted at improving the ability of families and mothers to detect, prevent and treat dehydration and increasing quality of attention through training and coordination with SOH first level facilities The main strategies to reach or achieve the Project’s objectives were inter-sector coordination, community participation and applying updated methodologies with international support in maternal – infant health. The main accomplishment of the program was in the area of maternal health. The program’s work with parteras, or traditional birth attendants (TBAs) was innovative and begins an important process of integrating and expanding the role of the TBA in community based health projects where there are no other viable alternatives for delivery. (TBA will be used in place of partera as TBA is a more commonly understood description of this level of provider). TBAs were invited into the hospitals to receive their training, which was expanded to provide information about “obstetric first aid”. Besides being trained in hygienic practices and sterilization of equipment, they were instructed in emergency post partum hemorrhage treatment, and neonatal resusication. Training of trainers was conducted by the American College of Nurse Midwives, further 1 Secrataria de Salud Honduras, Ashonplafa, USAID, CDC, & MSH. (2001). Honduras: Encuesta Nacional de Epidemiologia y Salud Familiar (ENESF-2001). Encuesta Nacional de Salud Masculina (ENSM-2001). 3 training of local health workers and TBAs was then conducted by the Secretary of Health personnel and project staff. This exposure of the TBAs to the larger health system broke down barriers that had existed between the TBAs and the workers of the secretary of health. Additionally, the program was highly successful in its other components. Thus, it has met and even exceeded the indicator goals set by the Detailed Implementation Plan (1999). The training of health educators and traditional birth attendants has been successful in reaching the community level as evidenced by the increased knowledge level and better disease management skills of the mothers in all areas surveyed. Mothers demonstrated in the survey that they had increased their knowledge of danger signs during pregnancy and many more women had received postpartum visits than at baseline. More women were exclusively breastfeeding and there were a greater number of women who understood the benefits of the Lacto-Amenorrheic Method (LAM) for birth spacing. There were significant improvements in mothers being able to identify signs and symptoms of pneumonia and less children were reported as having fast and agitated breathing in the last two weeks. More mothers were able to identify signs and symptoms of dehydration of children with diarrhea and could identify the more serious symptoms, which require medical attention. Mothers reported that they increased fluids before and increased fluids and food after the diarrhea illness. However, mothers did not reach the goal of 50% set to increase food during the illness. Most mothers surveyed administered ORS during the diarrhea illness and most mothers could correctly state how to prepare ORS. The project was able to document many changes. Most notably: 1. 17% of all pregnancies in the project area were identified with some complication or danger sign that required and resulted in a referral 2. Institutional births, a proxy for the global indicator of increased births with a birth attendant increased from 16 to 23% 3. Exclusive breastfeeding in children under 6 months of age increased from 32 to 63% 4. Caretakers knowledge on pneumonia signs identification increased from 38 to 89% 5. Care-seeking behavior for pneumonia increased from 53 to 86% 6. Oral rehydration treatment for children under two years of age with diarrhea in the past two weeks increased from 27 to 78% The main programmatic priorities evidenced from this evaluation were related to issues of sustainability and expansion of interventions, mainly the maternal related activities, given its urgency in rural and poor communities. 4 1. Results: Summary Chart Intibuca KPC 2003 Summary of Results Safe Motherhood and Newborn Care Indicator Baseline 1999 Goal Final 2003 Change from Baseline Increased percentage of women receiving a postpartum visit 1% 75.8% ? 74.8 % points Increased % of timely referrals No data 100% Undetected Increased percentage of TBA accompanied referrals No data 100% Undetected Increased percentage of communities having a ready action plan for transportation 0% 98% ? 98 % points Lacto-Amenorrheic Method (LAM) Baseline 1999 Goal Final 2003 Change from Baseline 50% of children under 6 months of age will be exclusively breastfed 32.2% 50% 63.2% ­ 31 % points 30% of women in their reproductive age will have knowledge regarding LAM No data 30% 81.8% Undetected Pneumonia Case Management Baseline 1999 Goal Final 2003 Change from Baseline Prevalence of Acute Respiratory Infection 56.3% 41.1% Ø 15.2 % points Proportion of children with ARI with increased and agitated breathing (Pneumonia Indicator) 77.5% 54.7% Ø 22.8 % points 60% of mothers of children less than 24 months will be able to identify signs of pneumonia 38.3% 60% 89.1% ? 50.8 % points 60% of mothers of children less than 24 months presenting with ARI will have sought medical care 53.4% 60% 86.3% ? 32.9 % points 60% of children less than 24 months with ARI in the past two weeks will receive treatment and/or referral by community health worker No data 60% 64.7% Undetected Diarrhea Case Management Baseline 1999 Goal Final 2003 Change from Baseline Prevalence of diarrhea 32.3% 34.0% ? 1.7 % points 45% of children will receive an increased amount of home-based fluids (breastfeeding or other liquids) 7% 45% 67.9% ? 60.9 % points 50% of children under 24 months with diarrhea in the past two weeks will have received increased ration of food during 26.8% 50% 29.9% ? 3.1 % points 5 illness 50% of children under 24 months with diarrhea in the past two weeks will receive an increased ration of food after illness 7.2% 50% 48.5% ? 41.3 % points 40% of mothers of children under 24 months will be able to identify at least one sign of dehydration 10% 40% 87.7% ? 77.7 % points 50% of mothers with children under 24 months with diarrhea in the past two weeks will treat their child with ORS 26.8% 50% 78.4% ? 51.6 % points 90% of mothers with children under 24 months will know how to correctly prepare ORS No data 90% 89.8% Undetected 2. Technical Approach A. Overview of the project: The Intibuca Community Based Child Survival program is a four-year USAID grant (Oct. 99-Sept.03), which has been implemented over the past four years in 95 communities of the most underserved communities in Intibuca, Honduras by Catholic Relief Services and a local non-governmental organization (COCEPRADII) in collaboration with the Secretary of Health. The program benefits directly 11,310 women of reproductive age and 11,324 children less than five years of age. Four health interventions are included in this program: Safe-motherhood and Newborn Care, Lacto￾Amenorrheic Method, Pneumonia Case Management, and Diarrhea Case Management. The Safe-motherhood and Newborn care has two main objectives: to improve the ability to address obstetric emergencies at the community level and to improve access to health care delivery for obstetric emergencies at health facilities. The main strategies in this intervention are to improve the ability of women, families and traditional birth attendants (TBAs) to recognize/prevent and respond to obstetric complications and to improve the ability of women, families and TBAs to access first level referral facilities in the event of an obstetric complication. The program contacted the American College of Nurse Midwives to train physicians and nurses from the Ministry of Health on Home Based Life Saving Skills (HBLSS). These individuals modified the program to train community level midwives and other health professionals in the program region. Health Committees in the region were trained on community mobilization approaches and the methodology to create an emergency transportation plan. The Lacto-Amenorrheic Methodology encourages exclusive breastfeeding for children under six months and educates on the value of using LAM as a natural family planning method for increased birth spacing. Breastfeeding was integrated into the activities of two main interventions, diarrhea control and maternal health. The approach will focused upon: - Immediate breastfeeding after delivery 6 - Exclusive breastfeeding as a nutritional approach and diarrhea prevention for the first six months after delivery - Exclusive breastfeeding as part of LAM methodology for birth spacing TBAs, community volunteers, and health facility personnel were trained in the LAM methodology. These personnel in turn promoted the LAM method among women in their reproductive age through home visits, mother’s groups, community meetings, and prenatal care. Pneumonia and Diarrhea Case Management were integrated within the national approach (AIN-C) to address health care of children under two years of age. The main objectives of these interventions were to improve case detection and management at the community level as well as appropriate referral to a health facility. Secretary of Health staff and Community Health Monitors received training in “institutional Integrated Management of Childhood Illnesses (IMCI)”. They were trained on the AIN-C module one and two. These modules educated on appropriate management of childhood illnesses such as diarrhea and pneumonia. In order to monitor the impact of the interventions, a KPC survey was implemented at baseline, midterm and final using LQAS sampling methodology. As well as the survey, information on maternal/child health indicators was gathered through the program’s health information system (HIS). The HIS monitored the status of maternal and neonatal health in the community through detection of pregnant women, identification of pregnancy related health problems and outcomes for both mother and child. The data was gathered by trained TBAs and submitted on a monthly basis to the local health educators. The health educators consolidated the information given and submitted the data to the field supervisors who in turn submit the information to the program manager. During these four years the program succeeded with one program manager, one health coordinator, three field supervisors, and 15 health educators. All program requirements, such as baseline survey, DIP development, annual reports, and midterm evaluation have been fulfilled on time. The program has developed the following protocols for the implementation of the interventions: Home Birth Life Saving Skills (HBLSS) manual for training of Traditional Birth Attendants (TBAs), and a health information system for monitoring and evaluation2 . B. Progress report by intervention area: B.1. Safe Motherhood and Newborn Care: This community based Child Survival Program sought to improve maternal health through three main community based interventions: 2 Rosales, A., Arita, M. & Nunez, M. (n.d.). Evaluation Report of a Community Based Health Information System in Rural Honduras: CRS/COCEPRADII Experience. 7 1. Train family members in the recognition of risk factors and danger signs in pregnancy 2. Train and supervise traditional birth attendants in danger sign recognition and new emergency obstetrics skills 3. Establish and maintain emergency committees in the community to respond with transportation resources when family members or traditional birth attendants recognize danger signs and complications. These maternal care activities were integrated into the other child survival activities, including nutrition, diarrhea control, and the integrated management of childhood illnesses. Community health workers who were involved with growth monitoring and sick child recognition, were also involved in increasing mother’s awareness of the presence of emergency committee and danger signs. These community-based interventions were then linked by educational programs and data collection/dissemination systems with activities and capacity building at the health center and to a lesser extent at the hospital level. The program’s work with parteras, or traditional birth attendants (TBAs) was innovative and begins an important process of integrating and expanding the role of the TBA in community based health projects where there are no other viable alternatives for delivery. (TBA will be used in place of partera as TBA is a more commonly understood description of this level of provider). TBAs were invited into the hospitals to receive their training, which was expanded to provide information about “obstetric first aid”. Besides being trained in hygienic practices and sterilization of equipment, they were instructed in emergency post partum hemorrhage treatment, and neonatal resuscitation. Training of trainers was conducted by the American College of Nurse Midwives, further training of local health workers and TBAs was then conducted by the Secretary of Health personnel and project staff. This exposure of the TBAs to the larger health system broke down barriers that had existed between the TBAs and the workers of the secretary of health. Because of improved communication and trust, the project was able to document many changes. Most notably: 1. 17% of all pregnancies in the project area were identified with some complication or danger sign that required and resulted in a referral 2. Institutional births, a proxy for the global indicator of increased births with a birth attendant increased from 16 to 23%. A monitoring and evaluation system was developed that identified each pregnancy in each of the 95 intervention communities. Mothers’ compliance with prenatal care activities and compliance with referrals could be monitored on a month-to-month basis, and if individual women needed counseling, team members could identify and provide enhanced education and counseling. A group of approximately 16 educators were employed by the project to provide supervision to the growth monitors, emergency committees and TBAs. Educators provided a link between these community groups and the health center and hospital staff. 8 These educators were also the key players in collecting community-based data and transferring that to the level of the health center. The monitoring and evaluation system proved to be a highly useful system for project monitoring. Catholic Relief Services (CRS) worked closely with a local partner, COCEPRADII, to implement the project. COCEPRADII, whose origins are in water projects in the area, had been transformed from a local water organization, to a water and health organization. COCEPRADII’s experience in establishing and maintaining water committees in communities facilitated the establishment of emergency committees. Community members were trained to activate and sustain the emergency committees. With action plans for both fund raising and transport of sick community members, these emergency committees have been active in transporting women, children and other adults. Evaluation Findings and Recommendations Overall Goals and Objectives Finding: The program’s technical objectives as defined in the DIP were not measurable as defined, but many monitoring indicators demonstrated impact. The main program objectives for maternal care as defined in the DIP were: 1. Increase the percent of families correctly performing obstetric first aid 2. Increase the percent of TBA’s correctly performing obstetric first aid 3. Increase the percent of women who receive follow-up care at defined intervals As overall goals, these points were addressed by the project activities. As objectives, the numerators and denominators are not defined. Data from the project’s monitoring system provided numerator data for the number of TBAs trained and supervised and the number of families who received educational messages (see tables and charts in Attachment A). The objectives, as written above, were not directly measured by the project. Numerators and denominators for the indicators to measure these objectives were not defined at the project onset. For example, the first objective would have, as the denominator, all families, and a numerator, “families” correctly performing first aid. This would have required training of families and demonstration of learned skills, with a follow up visit to the families to see if they were performing it correctly. The monitoring and evaluation system employed by the project did not create data points that would provide the percentage of people correctly performing first aid. In this project, many families were given instruction about complication recognition, and what steps to take if complications and emergencies occur. The project did measure the number of people who attended the educational sessions and trainings, which is more of a process indicator. Clearly, this objective as written isn’t feasible to measure. The second objective, increase the percent of TBA’s correctly performing first aid is also difficult to measure. Obstetric first aid comprises many different actions that would be 9 performed in response to any of a number of clinical situations. The major obstetric first aid intervention conducted by the project was the management of post partum hemorrhage. In this case, interviews with the TBAs demonstrated a familiarity and interest in post partum hemorrhage and other emergency management actions. The third objective, again is more of an overall goal and the specific content and time measures was not defined in the objective. Finding: The project developed and utilized monitoring objectives to demonstrate project impact. The project defined several monitoring indicators in safe motherhood and newborn care that provided a longitudinal picture of program progress and impact. The first monitoring indicator is a percent of women who receive at least two prenatal consultations and tetanus toxoid by trained personnel. Other monitoring indicators include: percent of pregnant women who deliver with a trained attendant, the percent of women who begin breast-feeding within the first hour after delivery the percent of women who receive a post-partum follow up within 48 hours of delivery the percent of TBAs trained in home based life saving skills the percent of women in need of referral/referred and the percent of communities with a functioning referral and emergency transport system or plan the percent of TBA’s who received at least one supervisory visit during the previous four months the percent of health facility personnel trained in life saving skills. The other major project goal was to improve the ability of women, families and TBAs to access first level referral facilities in the event of obstetric complication with the following monitor indicators: 1. Increased percent of timely referrals 2. Increased percent of TBA accompanied referrals 3. Increased percent of families having a ready action plan for transportation In the case of these indicators, project data was collected and demonstrated an increase in the percentage of referrals that occurred from the community to the next level of care. These referrals can not be judged as timely or not because the information about completion of referrals outcomes was not collected by the project data system. The second indicator, increased the percent of TBA’s accompanied referrals, was measured by the monitoring system however at this point that data is not available. The third indicator, increase percent of families having a ready action plan for transportation. In this case the percent of families was not measured however 100% of 10 the communities developed and an action plan for the handling of emergency obstetrics conditions and the plans that were available at the time of evaluation can be found in Attachment B. Recommendation: For future projects, goal and objective setting should be accomplished in such a way that the objectives of the program have clearly stated numerators and denominators and a stated method by which both numerators and denominators can be measured. Finding: Program success was demonstrated through measures of births by a trained attendant and rate of referral for complications. The monitoring and evaluation system developed by the project generated a rich source of data that allowed for the calculation of indicators that are, in retrospect, more appropriate and reflect project success. Finding: Institutional births increased in the project area. The project was able to measure, by proxy of institutional birth, the global indicator Increase the percent of deliveries by a skilled attendant. Interestingly, the project has noted an increase in the percent of women who deliver in the hospital. Finding: Referral for complications attained the expected percentage. The project was also able to measure the percentage of pregnancies referred for danger signs and complications and documented an overall transfer rate of 17%. Attachment A. This increase in institutional birth and referral is likely due to the intensive educational efforts that were conducted by the project staff to mothers’, families, emergency committees and traditional birth attendants and the increased access made possible by the emergency committees. Focus groups conducted during this evaluation with the traditional birth attendants reveal that the training offered by Catholic Relief Services and COCEPRADII introduced them to new information in the management of emergencies. Increased awareness of danger signs and complications was linked to a community-based intervention to increase access, the emergency committee, and these emergency committee’s provided resources and logistics to assist women in emergency transport. See Attachment A for graphic representations of the number of referral compared to the number of deliveries that occur each month during the project. As mentioned in the detailed implementation plan and recognized globally, approximately 15 % of all pregnancies are complicated in some way. One of the overall results of the project is that approximately 17% of all pregnancies are recognized in the community and were referred for complications. Emergency committees assisted with over half of these referrals. The ability of the project to measure this referral number and their ability to refer 17% of pregnancies, signify a major accomplishment in both the technical and measurement aspects. 11 The reasons for referral are many. The severity of the complications can not be determined from the current data collection system but overall the project referred approximately 17% of all pregnancies identified in some of the most rural areas to the next level of care. This outcome exceeds the expected 15% but certainly suggests that the project has been able to achieve a significant level of access to care for women with complications from pregnancy. Recommendations: Track and follow-up all referrals. The program was able to document the rate of referral and the indication for the referral from the community to the next level of care. The extent to which these were severe or mild complications cannot be determined. The maternal and neonatal outcomes were not systematically reviewed, and data was not collected describing the outcomes for mothers and neonates, such as c-section and vaginal birth rate, APGAR scores, and neonatal mortality. In many cases, referral outcomes were discussed informally between secretary of health physicians and project physicians. These informal discussions although helpful for the monitoring of individual cases, did not contribute to the ability of the program to define successes and challenges. B.2. Lacto-Amenorrheic Method: In order to increase the health of both mothers and their children appropriate child spacing has been indicated in improved health outcomes. The mothers were assessed according to the percentage that were exclusively breastfeeding at six months and the percentage of mothers that are aware of the lacto￾amenorrheic method (LAM) as a tool for birth spacing. Exclusive Breastfeeding At final evaluation, 63.2% of mothers reported exclusively breastfeeding their children under six months of age. At baseline, only 32.2% of mothers reported exclusively breastfeeding of children under six months of age. This is an increase of 31% points from baseline to final evaluation. Knowledge of LAM 81.8% of the women surveyed reported understanding the value of LAM in birth spacing. 12 Lacto-Amenorrheic Method Indicators 0% 20% 40% 60% 80% 100% Breastfed exclusively LAM knowledge Baseline Goal Final B.3. Pneumonia Case Management: Children with pneumonia must be appropriately managed to decrease the incidence of mortality from this common childhood illness. Mothers and community members must be trained to recognize the signs and symptoms of pneumonia and know the route for referral for treatment at a health clinic or hospital. Prevalence of Acute Respiratory Infection (ARI) At baseline, 56.3% of mothers reported that their child under 24 months of age had an acute respiratory infection (ARI) in the past two weeks. At final evaluation, 41.1% of mothers reported that their child had a respiratory infection within the last two weeks. This is a decrease of 15.2% points from baseline to final evaluation. The time-period in which baseline (December-January) and final (August-September) KPC were implemented is different, which might explain differences in prevalence. Of these children, 54.7% of mothers reported that their child had difficulty breathing with the ARI. This indicator was used as a rough estimate of the prevalence of pneumonia in this population. This indicates that approximately 22% of children surveyed had symptoms of pneumonia in the last two weeks. Case Detection At the final assessment 89.1% of mothers could appropriately identify at least one symptom of pneumonia. The mothers identified the following symptoms: fast and agitated breathing (76%), noisy breathing (35%), and retractions of the chest during 13 respiration (49%). This is an increase of 50.8 percentage points from baseline to final (38.3% ? 89.1%). Referral System In the final evaluation, 86.3% of mothers sought medical advice for their child’s respiratory infection. This is an increase of 32.9 percentage points from baseline. (53.4% ? 86.3%) Most sought medical advice from a nurse (54.7%), physician (18.8%) or a community health monitor (17.1%). Of the mothers that sought advice 14.5% were referred for treatment to a community health center or hospital. Of those that received a referral, 64.7% received a referral from a Community Health Monitor while the rest received a referral from either a nurse (17.6%) or physician (17.6%). Pneumonia Case Management Indicators 0% 20% 40% 60% 80% 100% Identify signs of pneumonia sought medical care referral Baseline Goal Final B.4. Diarrhea Case Management: Diarrhea is a common childhood illness that requires mothers to be educated on the necessity of increasing fluid and food during and after the illness. Mothers and community health workers need to be aware of the signs of symptoms during a diarrheal episode that requires referral to an appropriate health facility. Mothers also need to be educated on the correct preparation of Oral Rehydration Solution (ORS). Prevalence of Diarrhea Of the mothers surveyed 34% reported that their child had a diarrheal illness in the past two weeks. This is essentially unchanged from baseline where 32.3% of mothers reported a diarrheal illness in the past two weeks. Management of Diarrhea At baseline, 7% of mothers increased the amount of home-based fluids during a diarrheal illness and 26.8% increased the amount of food that the child received. 7.2% increased 14 the amount of food that the child received after the illness. By final evaluation, mothers increased the amount of fluids their child received (67.9%). This is an increase of 60.9% points from baseline. 39.2% of mothers increased the amount of food during the episode of diarrhea. This is an increase of 12.4% points from baseline. 48% of mothers increased the amount of food that the child received after the illness. This is an increase of 40.8% points from baseline. Diarrhea Case Management Indicators 1 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Increased fluids during illness Increased food during illness Increased food after illness Baseline Goal Final Case Detection At final evaluation, 87.7% of mothers could identify at least one sign of dehydration. This is an increase of 77.7% points from baseline (10%? 87.7%). 93% of mothers could correctly identify at least one symptom of the illness that would be an impetus for the mothers to ask for help from a healthcare provider. The symptoms included: Irritable crying (33%), unconsciousness/listlessness (31%), increased thirst (16%), sunken fontanel (8%), sunken eyes (41%), crying without tears (11%), tenting of the skin (29%), decreased urine output (4%), diarrhea with blood (10%) and diarrhea that lasts more than fourteen days (12%). Oral Rehydration Solution Of the children who had an episode of diarrhea within the last two weeks, the mothers were questioned on what they used for a home-based intervention for their child. 78.4% of the children received Litrosol (ORS) during the episode. This is an increase of 51.6% points from baseline (26.8% ? 78.4%). The other interventions mentioned included homemade serum, rice or maize flour gruel, and medicine. Of the mothers surveyed 89.8% knew how to correctly prepare ORS. 15 Referral System Of the children that presented with diarrhea, 89.1% of mothers requested assistance from a healthcare provider. Of those that asked for assistance, most mothers contacted nurses (67.3%). Others contacted included physicians (18.9%) and Community Health Monitors (13.0%). Diarrhea Case Management 2 0% 20% 40% 60% 80% 100% Identify signs of dehydration Treat with ORS Correctly prepare ORS Baseline Goal Final B.5. Vaccination Coverage: Vaccination against the immunizable diseases reduces morbidity and mortality in mothers, improving their health status. Of the mothers surveyed, 94.9% reported having received an immunization against Tetanus within the last year. Of those women that reported receiving a Tetanus immunization, 59.9% of them had a record of the immunization. The women cited the following reasons for why there was no record of the Tetanus: they forgot the immunization record (47.6%), they were not given a record (49.2%) and that the person that keeps the records was not present at the interview (3.2%). B.6. Growth Monitoring and Nutrition Breast and Complementary Feeding At final evaluation, 85.6% of mothers reported that they were breastfeeding their child at the time of the interview. Of those that weren’t breastfeeding, 90.2% reported that they had breastfed their child at least once. 72.6% of mothers reported not giving their child liquids before six months of age and 91.6% did not give their child food before six months of age. 16 75.8% of the mothers reported that they had breastfed their child for the first time within one hour after birth. 14.6% breastfed within one to eight hours after birth while 8.9% breastfed for the first time more than eight hours after birth. Growth monitoring At final evaluation, 89.1% of the mothers reported having a card that documented their child’s weight. Of those with documentation, 96.5% of the children had been weighed in the last four months. 3. Results: Crosscutting approaches Crosscutting approaches: Community mobilization Finding: The project utilized community mobilization as one of its major interventions that cut across the technical areas. The effectiveness of the approach for community mobilization was measured in two ways for this evaluation. Objective measures were obtained by documenting changes in indicators as measured by the Knowledge Practice and Coverage survey. Qualitative outcomes were obtained through focus group discussions with traditional birth attendants, emergency committee members and growth monitors. Comments made during these focus group discussions can be found in Attachment C (Sections 1, 2 & 3 notes from the focus group interviews with traditional birth attendants). The project engaged 95 communities during the project period and mobilized the community to: (1.) Develop emergency committees. See Attachment B for examples of work plans for the emergency committees in approximately 65 of the 95 villages. These were the plans that were available at this time of the evaluation. These emergency committees were developed as a result of activities of COCEPRADII before the Child Survival Project started. COCEPRADII was involved in water projects and had helped communities develop water committees. Because COCEPRADII had this history with the communities, the establishment of the emergency committees followed from their previous work. In the focus group discussions with emergency committee members, these committees have been active in many communities, and will continue to function even after the project ends. The emergency committees members interviewed are committed to the continuation and sustainability of the emergency committees. There was concern raised as to how members of the emergency committee might continue to maintain these committees, and as a result of this final evaluation no recommendation has been made to COCEPRADII to develop plans to continuously train new members to serve on the emergency committee. (2.) A second plan for community mobilization was the training of families and the recognition of danger signs and complications of pregnancy as can be seen 17 in the graphical results in Attachment A. Numerator data only demonstrates that a large number of educational sessions had been held for families. These educational sessions will not be able to be maintained in the intensity and level at which they were preformed during the project. (3.) The educators who were employed by the project will no longer be employed by the project. The educators have expressed an interest in continuing to work with their communities and facilitating the data transfer between communities and the health center. The health centers and the secretary of health along with CRS and COCEPRADII had some sustainability plans to maintain the educator stipends but because of restrained resources it is unlikely that educators will remain in their current position. Crosscutting approaches: Capacity Building The capacity building objectives of the project were to enable CRS headquarters and Honduras staff to implement innovative child survival interventions. The project hoped to do this through protocol development, training, monitoring and supervision. In this case the project did develop a training program and supervision system for the provision of some emergency obstetric skills provided by TBA to community members at home. The project was able to document an increase in the percent of women who were referred and an increase in the percentage of institutional births. The extent to which these successful components of intervention have enabled CRS headquarters and Honduras staff to implement these programs globally will be determined after the project’s completion. Another important capacity building objective was to increase the number of villages with an organized structure addressing health. The project hoped to have 50% of villages within the area of influence to have an organized structure. The definition of organized structure was not completely clear. However, 100% of the villages who work with COCEPRADII did develop emergency obstetric evacuation plans. Finding: Capacity was built in this project at the PVO level, but was not measured. As an organization, CRS had not previously implemented an emergency obstetric care project. The project developed a program for the training of TBAs in emergency obstetric care and combined this with a monitoring and evaluation system that could detect the changes in referral patterns and measure the number institutional births. This combination of program components is the main factor that lead to the impressive results of this project. CRS is in a position now to implement similar projects in their other program countries throughout the world. At the country level in Honduras CRS and COCEPRADII had been working in water and environmental projects and had been working in only small health projects. The current activity increased the capacity of CRS Honduras to implement and evaluate larger health programs and creates the possibility to expand health capacity in the country. 18 Positions were created that included not only water but also health and these positions will be maintained at the country level organization. Finding: Local partner capacity improved was not measured. COCEPRADII, CRS’s local partner, had been involved in implementing this child survival project. Through strengthening, the local partner COCEPRADII now considers itself a water and health organization. The project had defined a capacity building objective for partner strengthening as listed above. To measure this objective, CRS had proposed an institutional capacity assessment of COCEPRADII . This evaluation study was not performed due to finical constraints. There are no further plans to conduct that study. Recommendation: Continue to plan, implement and measure maternal care focused projects, building on the current model. Recommendation: Participate in CORE’s safe motherhood working group to build capacity and leadership in maternal care projects. Recommendation: Consider performing the capacity building evaluation that was originally proposed. Crosscutting Approach: Health Facilities Finding: The project worked from the community level to the health center level. Finding: Work at the maternity center level and at the hospital level, and communication/planning/partnering with organizations working at this level was limited. This project was community based and did not focus on the improvement of the health facilities in the area. However, the training of trainers performed by the American College of Nurse-Midwives had an impact on facilities. According to interviews with hospital staff, the training increased awareness of comprehensive and compassionate care and emphasized the need for the procurement and maintenance of materials, drugs, and equipment necessary for the provision of emergency obstetric care. There were no health facility assessments performed before or after the project and assessments were not proposed as part of the project activities. Interviews with physicians and personnel at the health facilities recognized that this project strengthened their relationship between the health facility and the community. Based on findings from focus groups with all participants this project, TBAs and secretary of health personnel were able to develop and extend a relationship based on mutual understanding. Partnering with other organizations was limited. The CRS project was aware of the Quality Assurance Project (QAP) but no common work plans were developed. This CRS 19 project was unaware of the JHPIEGO Maternal and Neonatal Health (MNH) program being conducted at the district hospital. As mentioned in other parts of this report, awareness and cooperation with the MNH program would have created partnerships resulting in interventions that extended from the most peripheral level all the way through the community. Working with MNH could have improved follow up on referrals, project activities could have been experienced by both groups, and health facilities may have been more involved in the project if this synergy had been developed. Recommendations: CRS Honduras should work with the maternity center and the district hospital in future projects to look at base line facility assessments in maternal care, identify areas that are weak, or might require resource or human capacity development to improve the quality of care. Using the expertise developed through this project, CRS could tap the resources of the Averting Maternal and Death and Disability program from Columbia University, the MNH program, the World Health Organization Making Pregnancies Safer program, the World Bank, PAHO, and other organizations who have resources to improve the facility structure in maternal and neonatal health. Improving health facilities and repairing and having the capacity to treat obstetric emergencies is the main intervention currently being pursued by the international community to decrease maternal mortality. Community based interventions such as the one that CRS has been engaged in do not have high priority on the international stage. It is important that CRS, through the activities of this project, increase awareness about the needs at the community level and build bridges between the community level and the institutional level. Through these bridges improvements at health facilities will have the greatest impact on the greatest number of people. Strengthening Health Worker Performance: Training Finding: The project trained TBAs to recognize one obstetric complication, provide some first aid measures, and refer to the next level of care utilizing community mobilized resources to ensure access. The training for TBAs was innovative. The American College of Nurse Midwives conducted life saving skills training for project staff and secretary of health physicians and nurses. A curriculum was adapted and developed for 6 skills taught to TBAs. The content from skill set was assembled into a manual for the training of TBAs. This manual is now available in Spanish and can be translated and used in other countries. Finding: The TBA training manual is not a comprehensive guide for handling all emergencies or problems that could be encountered. The manual addresses 5 major skill areas for TBA training: 1. An overview of Obstetric first aid at the community level 2. Post partum infection prevention through clean equipment and hand washing 3. New born care and resuscitation 4. Clean childbirth 5. Hemorrhage 20 The only clinical emergency for mothers covered is the management of post partum hemorrhage. The TBA training manual developed by Catholic Relief Services for their Honduras Child Survival Project goes beyond what the usual training for TBAs. In this case, the manual includes information on first aid for the acute recognition and management of obstetric hemorrhage (due to uterine atony) and in newborn care and resuscitation as real skill sets that need to be acquired and evaluated. The other themes of the TBA training manual requires further development, and other areas of Obstetric first aid that are not included in the manual. For example, the curriculum for Obstetric first aid in the community is fairly limited. Community level recognition of post partum infection and preeclampsia/eclampsia should also be included. Finding: TBAs discuss the practical skills of hand washing and equipment sterilization most often, discuss post partum hemorrhage second most often, and do not discuss infection recognition often. Three focus group discussions were held with three groups of TBAs, approximately 10 from each of the three main project areas. Attachment C contains a listing of comments. TBAs and “assistant” TBAs were interviewed in groups. TBAs were consistently appreciative of the equipment and supplies maintained by the Secretary of Health. TBAs actively demonstrated an understanding of hand washing and equipment cleaning. TBAs were very aware of post partum hemorrhage and the actions to take when post partum hemorrhage occurs. Clinically post-partum hemorrhage occurs frequently in pregnancy and in approximately 3-5 % and severely in about 1% of pregnancies. The training manual provided information for trainers to teach TBAs about bimanual uterine compression when excessive bleeding occurs due to uterine atony, and to activate the referral system through the emergency committees. The manual though does not cover the other first aid or community based management of other obstetric emergencies. TBAs actively demonstrated understanding of hand washing and equipment cleaning and sterilization techniques. Some TBAs demonstrated knowledge of the surface anatomy needed to identify the uterus and apply bimanual compression during a hemorrhage. Most had never learned of the life saving skills prior to the training provided by this project. Recognition and first aid management of severe pre-eclampsia, and especially eclampsia was not part of that training. TBAs are not allowed to measure blood pressure, although many of the TBAs interviewed in focus groups discussed this as a skill they would like to be taught. The recognition in management infections during labor and especially post￾partum uterine infections was also not included. These two areas, pre eclampsia and infection recognition that require further exploration for recognition and treatment at the community level that community based child survival projects could train Traditional Birth Attendants in addressing. Clearly there are other emergencies of pregnancies as well that may not be amenable to training TBAs in the 21 first aid treatment. But in this case the TBAs were given an overall orientation to obstetric emergencies and have a sense of the need to referral, Finding: TBAs are comfortable making referral to the health facility. It was clear from focus group discussions with traditional birth attendants, Attachment C, which this particular training, of traditional birth attendants conducted with the health facility personnel and the secretary of health personnel, has improved the relationship between the traditional birth attendants and the health facility staff. Although the direct cause and effect relationship cannot be made based on the data presented in this evaluation, and increase in the referral rate was noted by the project monitoring system. This rise in referral rate, and the rise institutional birth rate suggests that the program activities lead to an increase in the number of women referred by traditional birth attendants and suggests increase level of comfort among the traditional birth attendants in suggesting referral. Sustainability strategy Finding: Reviews of maternal deaths do not occur in a systematic way. One of the most important components of a sustainable maternal/neonatal mortality prevention program is the anonymous investigation of each maternal death that occurs in the community. Although the program did not specifically name this as an intervention, CRS has been involved in some investigations of maternal deaths that have occurred in the area. CRS has worked with the ministry of health in their process of investigating maternal deaths. Finding: Maternal mortality reviews are not specifically geared towards identifying the factors associated with maternal deaths but are part of a larger process of the monitoring and review of all deaths that occur in the area. When maternal deaths occur, the department is notified and within the specific offices, notice is taken of the details of the death. In an informal way this information is used at subsequent meetings when health policies are discussed. CRS is aware of this process and participates superficially. Recommendation: The project should assist ministry of health in developing specific and individual yet anonymous maternal mortality review committee. Recommendation: Conduct a formal conference initiating a new maternal mortality review process. This review process would be a separate entity or committee made up of representatives from different organizations involved in health in the area, and would serve to collect and organize all the information pertaining to maternal deaths. The records should be maintained longitudinally so that trends and maternal death can be determined. This 22 maternal mortality review committee should not be linked to assessing the quality of care provided by a specific person, or clinic. All references to time, place and person should be removed. Specific provider or clinic issues should be addressed in other committees. As a result of the maternal mortality review committee’s deliberations over one year, program implementation activities or policy interventions may be suggested to address identified trends. Recommendation: As this formal maternal mortality committee has not yet been established, it would be a good idea to collect the information on maternal deaths that have occurred in the last two years and present this at a kick-off meeting so that the process of anonymous maternal mortality review can be initiated. The CDC has publications that assist with maternal mortality reviews, (http://www.cdc.gov/reproductivehealth/pdf/Strategies.pdf) and the WHO has a monograph for verbal autopsy for maternal death. Finding: Work with other maternal care organizations was limited or absent. Part of the sustainability strategy for the CRS project in Honduras could have included partnering with other organizations in the implementation of the Maternal Health Care Program. The JHPIEGO MNH Program is working in Honduras to improve the quality of care at the health facility level. The quality assurance project is also working at the hospital level to improve the quality of care delivered by the hospital as a whole. Project staff was aware of the QAP Project, but were not aware of the MNH Program and the activities it was conducting at the hospital level in the project area. The nurses and physicians interviewed at the district hospital were actively engaged with the MNH Program in implementing the WHO protocols presented in the IMPACT manual, managing the complications of pregnancy and childbirth. The MNH Program worked from the national level to the level of this district hospital in the implementation of their program. The CRS Child Survival Project worked from the community level up to the level of the maternity center. The gap between the maternity and the hospital level was never bridged. The activities of the two projects never coincided so the benefits that CRS could have derived by partnering with the MNH program were never realized. There could have been significant impact on the sustainability of the overall child survival program in the region had more synergy been cultivated. Recommendation: Work with the MNH program and other maternal care organizations. The CRS Project in Honduras should contact the MNH in Honduras and during this last part of program, these two organizations should at least share lessons learned. MNH staff should be invited to participate in some of the close out conferences that will be conducted by the CRS. A partnership between CRS and MNH can be explored. Program Management: Planning Findings: The CRS Project in Honduras worked with its local partner COCEPRADII in the planning of projects, especially in the development of emergency committees. 23 COCEPRADII was mainly involved in water management issues in the project areas. COCEPRADII has established water committees in most of the remote villages in this Intibuca area. During the planning phase of the program CRS and COCEPRADII worked closely together to determine expectation and outputs that each organization was expected to contribute to the overall program success. The planning process allowed for the experience of the water committees to be extended to the development of the emergency committees. This evaluation has determined that COCEPRADII was a key element in the ability of the program to establish emergency committees, and COCEPRADII is a key component of the sustainability of these committees. Staff Training and Supervision Finding: Staff at each level were well trained. Finding: The supervision of program staff in this project followed a very clear and effective outline. The educators supervised the TBAs visiting each once a month and reviewing a skill with them using check lists. Each educator was able to visit their TBA on a monthly basis, and each educator was able to observe the six skills taught to TBAs on at least two occasions. Each traditional birth attendant had each of her six skills observed by an educator at least twice. Finding: Traditional birth attendants during their focus group discussions were able as a group to discuss their supervision, and in all cases were quite happy and satisfied with their level of supervision. Please see the results of the focus groups discussions in Attachment C. Finding: TBAs seemed more comfortable discussing the skills of washing hands and sterilizing their equipment and were less likely to discuss the steps needed for post￾partum hemorrhage control. This issue was discussed with the educators and it was felt as though: (1.) Hygienic practices and equipment sterilization are much more concrete and are able to be understood and repeated by the TBA. (2.) Post-partum hemorrhage is not seen on a regular basis and therefore the utilization of these skills is rare. Recommendation: It is recommended that educators increase their supervision and support the strengthening of TBA skills with the abstract and theoretical skills they have been taught such as post-partum hemorrhage control infection recognition. Finding: sub-coordinators supervised Educators. Each of the three sub-coordinators was able to visit each of the educators as least once a week. Sub coordinators utilized tools to supervise their educators and in all cases educators were quite satisfied with their level of 24 supervision by the sub-coordinators. The coordinator then supervised sub-coordinators and again no issues were identified as problematic in that area. Recommendation: The supervisory system employed by this CRS project was excellent and no further recommendations are made. Finding: The CRS project staff and secretary of health staff relied heavily on the reduction of the maternal mortality ratio as evidence of progress. It was quite common during the evaluation process to members of the community, members of the project staff, members of the ministry of health and members of the partner COCEPRADII, to discuss “a decrease in the maternal mortality ratio” as evidence that the program has been successful. It is known that the maternal mortality ratios are unstable from year to year as vital statistics. The number of maternal deaths varies considerably from year-to-year, and small changes in the number of deaths can result in very big changes in the maternal mortality ratio because of the large denominator. A yearly change in the maternal mortality ratio does not objectively mean that the program has been successful. Recommendation: For a correct understanding of how to measure progress of maternal health, the project should provide education for all the partners involved about the value of other indicators that are more measurable and reflect more accurately project success than the maternal mortality ratio. This project measured the percent of referrals that were made to the health facility, which is a much better indicator of the progress of maternal health. This project also measured institutional births, which is also a significant indicator of progress. By engaging in discussions with the partners COCEPRADII and the ministry of health, and by examining the data systems that are currently available, the project could come up with other indicators of project success that would be acceptable to all partners and easily understood. Using maternal mortality ratio as an indicator is not accepted globally, and quoting such figures may decrease the chances of the project being funded as it is widely accepted that maternal mortality ratio can not be used as an indicator of project success. Recommendation: Develop a measure of access. Another issue identified by the team, improvements in maternal care, occur through increasing access to emergency obstetric services. Finding: The project uses a combination of the risk factor approach for triage and the danger sign complication recognition approach. The program policies as well as the secretary of health policies have focused on the risk factor approach in addition to the danger sign approach. Overemphasis on the risk factor approach is concerning because we know that most of the pregnancies who have risk factors go on to deliver normally. Most of the problems occur in women without risk factors. The project’s emphasis on risk factors has not taken away from emphasis on 25 danger signs. Among the people working in the project, risk factors, danger signs and complications are terms that seem to be used interchangeably. All risk factors are not created equally, and some risk factors have more predictive value than others. For example, previous cesarean section is a very strong indicator for subsequent poor pregnancy outcome. Height on the other hand is not. Age is not, especially if a woman has had a previous normal birth. Most multiparous women also do well, even though they as a group are at increased risk. The actions taken based on risk factor and danger sign findings will also have an effect on their value. The program has adopted a policy of having all women deliver their first pregnancy at the health facility, I think that this is a very reasonable risk factor to use and encourages all women in their first pregnancy to have access to medical care. The value of this practice is not based on evidence, however. Grand multiparty may or may not be a good risk factor to use when referring a patient for a long stay in a maternity waiting home, but certainly is a good indication for a referral. Separation from family for a long period of time may negatively impact the family in other ways that make the risks of such a policy outweigh the unknown and unproven benefits from such a policy. Overall the project has developed a good mix of risk factors and danger signs as ways to identify women who need more then the usual care during pregnancy. 26 Summary of Conclusions and Recommendations Safe Motherhood and Newborn Care Finding: The program’s technical objectives as defined in the DIP were not measurable as defined, but many monitoring indicators demonstrated impact. Finding: The project developed and utilized monitoring objectives to demonstrate project impact. Finding: Program success was demonstrated through measures of births by a trained attendant and rate of referral for complications. Finding: Institutional births increased in the project area. Finding: Referral for complications attained the expected percentage. Recommendation: Objectives of the program have clearly stated numerators and denominators and a stated method by which both numerators and denominators can be measured. Recommendations: Track and follow-up all referrals. Lacto-Amenorrheic Method Natural family planning education initiatives to increase knowledge on this particular method, aimed at birth spacing. Pneumonia Case Management Project efforts were focused on improving case detection and care-seeking behavior at the community level and increasing quality of attention through Standard Case Management (SCM) training and links with Secretary of Health first level facilities Diarrhea Case Management Interventions were targeted at improving the ability of families and mothers to detect, prevent and treat dehydration and increasing quality of attention through training and coordination with SOH first level facilities Cross Cutting Finding: The project utilized community mobilization as one of its major interventions that cut across the technical areas. Finding: Capacity was built in this project at the PVO level, but was not measured. Finding: Local partner capacity improved was not measured. Recommendation: Continue to plan, implement and measure maternal care focused projects, building on the current model. Recommendation: Participate in CORE’s safe motherhood working group to build capacity and leadership in maternal care projects. Recommendation: Consider performing the capacity building evaluation that was originally proposed. Crosscutting Approach: Health Facilities Finding: The project worked from the community level to the health center level. 27 Finding: Work at the maternity center level and at the hospital level, and communication/planning/partnering with organizations working at this level was limited. Recommendations: CRS Honduras should work with the maternity center and the district hospital in future projects Finding: The project utilized community mobilization as one of its major interventions that cut across the technical areas. Strengthening Health Worker Performance: Training Finding: The project trained TBAs to recognize one obstetric complication, provide some first aid measures, and refer to the next level of care utilizing community mobilized resources to ensure access. Finding: The TBA training manual is not a comprehensive guide for handling all emergencies or problems that could be encountered. Recommendation: Continue to explore and develop resources to improve the skills of TBA’s to manage complications at the community level. Finding: TBAs discuss the practical skills of hand washing and equipment sterilization most often, discuss post partum hemorrhage second most often, and do not discuss infection recognition often. Finding: TBAs are comfortable making referral to the health facility and the program activities lead to an increase in the number of women referred by traditional birth attendants Sustainability strategy Finding: Reviews of maternal deaths do not occur in a systematic way. Finding: Maternal mortality reviews are not specifically geared towards identifying the factors associated with maternal deaths but are part of a larger process of the monitoring and review of all deaths that occur in the area. Recommendation: The project should assist ministry of health in developing specific and individual yet anonymous maternal mortality review committee. Recommendation: Conduct a formal conference initiating a new maternal mortality review process. Recommendation: Collect the information on maternal deaths that have occurred in the last two years and present this at a Maternal Mortality Review Committee Development Conference. Finding: Explore ways to work with other maternal care organizations was limited or absent. Recommendation: Work with the MNH program and other maternal care organizations. Program Management: Planning Findings: The CRS Project in Honduras worked with its local partner COCEPRADII in the planning of projects, especially in the development of emergency committees. Staff Training and Supervision 28 Finding: Staff at each level considered themselves well trained. Finding: The supervision of program staff in this project followed a very clear and effective outline. Finding: Traditional birth attendants during their focus group discussions were able as a group to discuss their supervision, and in all cases were quite happy and satisfied with their level of supervision. Please see the results of the focus groups discussions in Attachment C. Recommendation: It is recommended that educators increase their supervision and support the strengthening of TBA skills with the abstract and theoretical skills they have been taught such as post-partum hemorrhage control infection recognition. Recommendation: The supervisory system employed by this CRS project was excellent and no further recommendations are made. Finding: The CRS project staff and secretary of health staff relied heavily on the reduction of the maternal mortality ratio as evidence of progress. Recommendation: Provide education for all the partners involved about the value of other indicators that are more measurable and reflect more accurately project success then the maternal mortality ratio. Recommendation: Develop a measure of access. Finding: The project uses a combination of the risk factor approach for triage and the danger sign complication recognition approach. 29 RESULTS HIGHLIGHTS CRS Community-Based Child Survival Intibuca Addressing the “Three Delays” at the Community Level The Safe-Motherhood and Newborn care component of the program improved the ability of women, families and traditional birth attendants (TBAs) to recognize/prevent and respond to obstetric complications and to improve the ability of women, families and TBAs to access first level referral facilities in the event of an obstetric complication. Health Committees in the region were trained on community mobilization approaches and the methodology to create and sustain emergency transportation plan. Through this combination, the “the delays” were effectively addressed in a remote community. This program focused in 4 major areas: q Safe Motherhood and Newborn Care Emphasis placed on obstetric first aid at the community and health center level and the creation of community emergency transportation plans to first and second level SOH facilities q Lacto-Amenorrheic Method Natural family planning education initiatives to increase knowledge on this particular method, aimed at birth spacing. q Pneumonia Case Management Project efforts focused on improving case detection and care-seeking behavior at the community level and increasing quality of attention through Standard Case Management (SCM) training and links with Secretary of Health first level facilities q Diarrhea Case Management Interventions targeted at improving the ability of families and mothers to detect, prevent and treat dehydration and increasing quality of attention through training and coordination with SOH first level facilities The project was also able to demonstrate results and changes, most notably: 1. 17% of all pregnancies in the project area were identified with some complication or danger sign that required and resulted in a referral 2. Institutional births, a proxy for the global indicator of increased births with a birth attendant increased from 16 to 23%. 3. Emergency evacuation plans were present in 100% of communities where non has been in place before. 4. Exclusive breastfeeding in children under 6 months of age increased from 32 to 63% 5. Caretakers knowledge on pneumonia signs identification increased from 38 to 89% 6. Care-seeking behavior for pneumonia increased from 53 to 86% 7. Oral rehydration treatment for children under two years of age with diarrhea in the past two weeks increased from 27 to 78%