CATHOLIC RELIEF SERVICES Maguindanao Child Survival Project Maguindanao, Philippines. USAID/BHR/PVC/CSH CA No. FAO–A–00-00-0037-00 Final Evaluation Report: March 2005 ________________________________________________________________________ Project Location: Maguindanao, Philippines Project Duration: September 30, 2000 to January 31, 2005 Team Leader: Dr. Marcelo Castrillo, Consultant Field Contact: Milagros Lasquety, MPH Headquarters Contact: Alfonso Rosales, M.D., M.P.H.-T.M 2 TABLE OF CONTENTS A. SUMMARY………………………………………………………………………………. B. ASSESSMENT OF PROJECT OUTCOMES AND IMPACT 1. Outcomes: Summary Tables……………………………………………………………… 2. Outcomes: Technical Approach……………………………………………………………. 3. Outcomes: Crosscutting Approach…………………………………………………….. C. PROJECT MANAGEMENT……………………………………………………………. D. OTHER ISSUES IDENTIFIED BY THE TEAM………………………………………… E. CONCLUSIONS AND RECOMMENDATIONS………………………………………… F. ONE PAGE SUMMARY OF OUTCOMES ……………………………………………... 5 7 8 34 37 37 39 ATTACHMENTS A. Evaluation Team Members B. Evaluation Assessment Methodology C. List of people interviewed and contacted D. Diskette or CD with electronic copy of the report in MS WORD 2000 E. Special Reports F. Project Data Sheet Form – Updated version G. Focus Group Discussion Protocol: Mothers of Children under 5 Years of Age Participating in the MCS Project H. Focus Group Discussion Protocol: Health Providers in MCSP Municipalities 3 LIST OF ACRONYMS ADB Asian Development Bank ARI Acute Respiratory Infection ARMM Autonomous Region of Muslim Mindanao BCC Behavioral Change Communications BDC Barangay Development Council BHC Barangay Health Committee BHS Barangay Health Station BHW Barangay Health Worker CBHO Community-Based Health Organization CDD Control of Diarrheal Disease CO/CD Community Organizing/Community Development COP Culture of Peace CORE Child Survival Collaborations and Resources Group CRS Catholic Relief Services DHS Demographic Health Survey DIP Detailed Implementation Pla n DOH Department of Health FACS Food Assisted Child Survival FE Final Evaluation FGD Focus Group Discussion FHSIS Field Health Service Information System FIC Fully Immunized Children GEM Global Excellence in Management GOP Government of the Philippines HKI Helen Keller International HNP Health and Nutrition Post HIS Health Information System HWG Health Working Group IEC Information, Education, and Communication IGP Income Generating Project IMCI Integrated Management of Childhood Illnesses IPHO Integrated Provincial Health Office (Maguindanao) KFI Kadtuntaya Foundation, Incorporated KII Key Informant Interview KPC Knowledge, Practice, and Coverage Survey LGU Local Government Unit LQAS Lot Quality Assurance Survey MCSP Maguindanao Child Survival Project MHO Municipal Health Officer MIS Management Information System MLGU Municipal Local Government Unit MST Municipal Support Team MTE Mid -term Evaluation MTWG Municipal Technical Working Group 4 NDS National Demographic and Health Survey NGO Non-Governmental Organization ORS Oral Rehydration Solution ORT Oral Rehydration Therapy PCM Pneumonia Case Management PHN Public Health Nurse PRA Participatory Rapid Appraisal PVC Private Voluntary Cooperation RHM Rural Health Midwife RHU Rural Health Unit SCM Standard Case Management TBA Traditional Birth Attendant TMCHP Targeted Maternal and Child Health Program TQM Total Quality Management UNFPA United Nations Population Fund UNICEF United Nations International Children’s Educational Fund 5 A. SUMMARY The project was implemented in Maguindanao province, one of the five provinces that make up the Autonomous Region of Muslim Mindanao (ARMM). The regional Child Mortality Rate of 97.6 per 100,000 live births is twice the national average, while health facility utilization rates are three times lower than the national average. Decades of violent conflict between rebel groups and the national government have disrupted the provincial health services network and contributed to some of the highest morbidity rates in the nation. The MCSP complemented other CRS projects in the area, particularly peace and reconciliation efforts between the majority Muslim population and the Christian population. The total population covered by the project was 123,117 inhabitants (including 13,948 children under five years of age and 24,049 women of reproductive age). The four-year MCSP (October 1, 2000 – September 30, 2004, plus a no-cost extension until January 2005) sought to improve the health status of children under the age of five and women of reproductive age in 76 communities (barangays) in five municipalities in Maguindanao province. The project addressed the principal causes of child mortality through child survival (CS) interventions in four areas: (1) Breastfeeding and Nutrition, by promoting exclusive breastfeeding, appropriate complementary feeding practices, and micronutrient supplementation through community health education and social mobilization activities; (2) Pneumonia Case Management, by improving case detection and care-seeking behaviors at the community level and enhancing the quality of care through the training of midwives (RHM), Barangay Health Workers (BHW), and Rural Health Unit (RHU); (3) Diarrhea Case Management, by strengthening the ability of caregivers to prevent, recognize, and treat or seek care for chronic diarrhea and dehydration through training at the community, midwife, BHW and RHU levels; and (4) Expanded Program in Immunization (EPI), with an emphasis on strengthening the capacity of the Integrated Provincial Health Office (IPHO) to deliver quality EPI services and mobilizing communities to participate in immunization campaigns. CRS/Philippines collaborated with two partners in project implementation: the Integrated Provincial Health Office (IPHO) and the Kadtuntaya Foundation, Inc. (KFI). The MCSP used a decentralization approach to community health care delivery. The aim was to shift delivery of essential health services from health facilities (physicians and nurses) to community health structures (barangay health workers and midwives) and to link these two points in the system through a strong referral system. This approach featured the adaptation and implementation of the Community Integrated Management of Childhood Illness (C-IMCI) strategy to increase resolution capacity at the community level. This was reinforced by the implementation of a health facility-IMCI strategy to improve the quality of services at that level of the health care system. The MCSP strategy was based on the CORE Group’s C-IMCI model and, in addition to disease identification and management tools, included a locally-designed Behavioral Change Communications (BCC) program, a standardized approach to community mobilization, and the facilitation of private-public partnerships between the IPHO, KFI, CRS, and local and municipal governments. 6 The Final Evaluation Team consisted of CRS Maguindanao staff, Manila and Baltimore headquarters representatives, IPHO staff, other partners (See Attachment A), and the external evaluator. The final evaluation (FE), which was conducted from October 4 - 28, 2004, included quantitative and qualitative surveys. Quantitative data was collected through a KPC survey using LQAS sampling methodology, while qualitative data was collected through focus group discussions and key informant interviews of project staff, mothers of children under 5 years of age, community leaders, and DOH personnel. The evaluation’s main findings showed quantitative improvements in all but one of the project indicators. The nutrition and breastfeeding intervention met and exceeded all eight programmatic targets as demonstrated by comparing baseline, midterm, and final data (see Tables 1 and 2). The overall malnutrition rate in children under two years of age declined by 34% (target 10%). Initiation of breastfeeding within one hour of birth increased by 42 percentage points, the rate of exclusive breastfeeding of children under 6 months of age increased by 28 percentage points, the complementary feeding rate for children aged 6 to 9 months rose by 13 percentage points, and growth monitoring coverage increased by 41 percentage points. In the second program component, pneumonia case management, all key indicators exceeded the targets established at the outset of the project (see Tables 3 and 4). Moreover, the project led to improvements in case detection skills at the household and community levels, access to prompt treatment, and quality of care at the institutional and community levels. The ability of mothers of children under 2 years of age to recognize danger signs in their children increased 31 percentage points over the baseline figures (Table 3), while 83% of children under two had received ARI treatment by the end of the project, 13 percentage points over the established target. The CDD intervention led to a decline of 12 percentage points in the overall prevalence of diarrheal disease in children under two years of age (see Table 8). All four indicators to measure the impact of the CDD intervention had been met and exceeded by the end of project implementation. For example, there was a 44 percentage point increase in the number of children under two who had experienced diarrhea in the preceding two weeks and had received either the same or more fluids and food, and caregivers of children under age two with diarrhea during the preceding two weeks had increased their care-seeking behaviors by 39 percentage points. Similarly, the availability of functioning ORT facilities increased by 92 percentage points, while the availability of trained community health workers increased by 100 percentage points (see Tables 5 and 6). Finally, the availability of immunization services increased during the life of the project. The final data showed that EPI access had increased to 99%, possession of a vaccination card rose by 51 percentage points, and quality of services also improved, as measured by the percentage of trained health personnel and a proxy indicator for proper cold chain management. Nonetheless, utilization of vaccination services did not follow service availability. The overall impact indicator for the EPI intervention “percentage of fully immunized children” showed no quantitative change between baseline (58%) and final data (57.3%). Several key factors contributed to the impressive outcomes achieved in a relatively short time period. The first was the decentralized approach to community health care delivery. The data collected during the baseline assessment and final evaluation facilitated a thorough analysis of the impact of program interventions on improving access to health care (service availability and 7 service utilization) and to preventive services. As Donabedian1 observed in his 1972 study on health care access, “the proof of access is use of the service, not simply the presence of a health facility.” CRS Philippines used CORE’s C-IMCI model2 to address issues of health services supply and demand. The second factor was the locally-adapted Behavioral Change Communications (BCC) strategy with a focus on danger signs, through which culturally appropriate, targeted health messages were designed. A total of 20 health messages were created for the four interventions and a multi￾channeled message distribution system was used. Third, all project components were built on the foundation of a solid community structure. The project used a participatory, community-based outreach and development process to ensure local participation and to optimize learning. This process was divided into six phases: the preparatory phase, barangay entry process, barangay mobilization-planning phase, community health mobilization and capacity-building, networking/linkage-building, and exit phase. 1 Donabedian A. (1972) ‘Models for Organizing the Delivery of Personal Health Services and Criteria for Evaluating Them’, Milbank Memorial Fund Quarterly, 50: 103-54. 2 Winch P, Leban K, Kusha B, 2001. Reaching Communities for Child Health and Nutrition: a Framework for Household and Community IMCI. Calverton MD: Child Survival Technical Support Project 8 B. ASSESSMENT OF PROJECT OUTCOMES AND IMPACT 1. Outcomes: Summary Tables by MCSP Indicator The following tables show the MCSP indicators included in the Detailed Implementation Plan (DIP), including established targets for the duration of the project. Breastfeeding and Child Nutrition Indicators Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Breastfeeding Initiation Percentage of children aged 0-23 months who were breastfed within the first hour of birth 50% 34% 60% 76.4% (71.2% – 81 6%) Percentage of infants aged 0-5 months who were fed breast milk only (no water) in the preceding 24 hours 72.7% (62.2% – 83.2%) Exclusive Breastfeeding Rate Percentage of infants aged 0-5 months who were fed breast milk only (including water) in the preceding 24 hours 50% 45% 58% 81.3% (72.1% – 90.5%) Complementary Feeding Rate Percentage of infants aged 6-9 months who received breast milk and solid foods in the preceding 24 hours 50% 77% 87% 90.2% (82.4 %– 92.0%) Growth Monitoring and Anthropometric Indicators Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Percent Underweight Percentage of children aged 0-23 months who are less than 2 standard deviations below the median weight-for-age of the reference population 22% 25% - 16.6% (13.3% - 19.9%) Childhood Immunization Indicators Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Vitamin A Coverage Percentage of children aged 6-23 months who received a dose of Vitamin A in the preceding six months 50% 36% 96% 69.2% (63.6% – 74.8%) Fully Immunized Children Percentage of children aged 12-23 months who received OPV3, DPT3, and measles vaccines before their first birthday [out of the population of children 12-23 mos. with cards] 70% 57.7% 76% 57.3% (50.70% - 63.9%) Sick Child Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Maternal Knowledge of Child Danger Signs Percentage of mothers of children aged 0- 23 months who knew at least two signs of childhood illness indicating the need for treatment 50% 64% 86% 95.3% (92.7% - 97.9% 9 ARI Indicators Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) ARI Care-seeking Percentage of children aged 0-23 months with cough and rapid/difficult breathing during the preceding two weeks who were taken to a health facility or received antibiotics from an alternative source 70% - - 83.1% (75.4% – 90.8%) Percentage of mothers with children <2 with rapid/difficult breathing who sought immediate care from a health care provider 60% 70% 57% 78.7 Quality of health care Percentage of BHWs trained in pneumonia identification 60% 0% 100% Percentage of trained BHWs able to correctly assess and refer pneumonia cases 60% 0% 50% Diarrhea Control Indicators Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Diarrhea treatment Percentage of children <2 with diarrhea in the preceding two weeks who received increased fluids and food during their illness 40% 31% - 75.4% Care-seeking for Diarrhea Percentage of children aged 0-23 months with diarrhea in the preceding two weeks whose mothers sought outside advice or treatment for the illness 70% 40% 71.2% 78.7% Health care access Percentage of health facilities with a functioning ORT corner based on WHO standards 100% 8% - 100% Percentage of BHWs trained in CDD home-based case management 80% 0% 100% Prenatal Care Indicators Indicator Description/Definition Target Baseline Midterm F. Eval. Tetanus Toxoid Coverage Percentage of mothers who received at least two tetanus toxoid injections (mother’s recall) before the birth of their youngest child under 24 months of age 70% 47% 72% 84.4% (81.1% – 87.7%) Iron Supplement ation Coverage Percentage of mothers who received/bought iron supplements while pregnant with their youngest child, under 24 months of age 60% Pregnant 70% Lactating 20% - 31% 41.6% 10 2. Outcomes: Technical Approach a. Overview of the Maguindanao Child Survival Project Maguindanao province is located on the southern Philippine island of Mindanao, one of five provinces that make up the Autonomous Region of Muslim Mindanao (ARMM). The population of Maguindanao is 90% Muslim, with the remaining 10% made up of Christians and tribal groups. The predominant language in the province is Maguindanaoan, and the various tribes speak their own languages. Although members of all three groups are distributed throughout the province, Christian populations tend to live closer to major population centers and along the main roads. In contrast, tribal groups have been able to preserve their unique culture by withdrawing to the highland areas where they are geographically isolated. The social and economic indicators in this poverty-stricken region are consistently the lowest in the country. This is attributable, in part, to central government neglect, as well as to the prolonged conflict. The project covered 76 barangays (villages) in five municipalities of Maguindanao (Sultan Sa Barongis, Kabuntalan, South Upi, Mamasapano, and Talitay). The total population in the project area was 123,117 inhabitants, including13,948 children under the age of five and 24,049 women of reproductive age. The project consisted of four interventions: Breastfeeding and Nutrition (35% of project effort), Pneumonia Case Management (25%), Diarrhea Case Management (25%), and Expanded Program in Immunization (EPI) (15%). The MCSP collaborated with two partners in project implementation: the Integrated Provincial Health Office (IPHO) and the Kadtuntaya Foundation Inc. (KFI): ß The IPHO was responsible for implementing child survival activities through health care provision, health education, and social mobilization; ß KFI, a Muslim NGO with 11 years of community experience in Maguindanao, reinforced the IPHO’s child survival activities by providing intensive support for community health mobilization activities in 45 of the 76 target barangays. KFI worked closely with communities and IPHO staff to organize the Barangay Health Committees ß CRS/Philippines served as a catalyst for the development of a synergistic private/public partnership between the IPHO and KFI. The project was designed to complement the IPHO plan to improve access to quality health services by implementing the Integrated Management of Childhood Illnesses (IMCI) strategy at the health facility level and introducing a community component—based on CORE’s C-IMCI framework and using CRS’ C-IMCI tools—that would forge tangible links between the formal health system and the communities. The MCSP used the following strategies in implementing its health-related activities: community mobilization for health, community-based health education, promotion of private/public partnerships, and capacity-building in health facilities and in the communities. Total funding for the four-year program was US$ 1,618,252, including a USAID investment of US$ 999,592. The annual project cost per beneficiary was US $6.67. 11 CRS/Philippines has worked in the project area for over two decades. It worked with the USAID Title II-sponsored Maternal-Child Health Program until its closure in 1996 and later was involved in promoting peace and reconciliation activities with the Kadtuntaya Foundation, Incorporated (KFI) and other NGOs, prior to the current Child Survival Project. b. Outcomes by intervention area. In order to assess project outcomes by intervention area and project strategy, the evaluation team organized a three-day technical meeting for staff from CRS/Mindanao and its local counterparts, the IPHO and KFI, (see Annex A for the complete list). The meeting was divided into two parts, the first to review and interpret the findings of the KPC survey carried out by CRS during the previous month, and the second to examine technical and crosscutting strategies and approaches. For the first part of the meeting, the evaluation team prepared a background information packet that contained: (1) part of the DIP including the MCSP objectives and core strategies, (2) USAID’s final evaluation (FE) guidelines,3 and (3) the list of FE team members and working￾groups. The plenary was then divided into four subgroups to examine each intervention area: Subgroup 1 - pneumonia case management indicators; Subgroup 2 - diarrhea control indicators; Subgroup 3 - EPI indicators; and, Sub-group 4 - nutrition and breastfeeding indicators. During the second half of the meeting, the team reviewed and discussed technical and crosscutting approaches and strategies. The same four subgroups then reconvened to examine the following areas: Subgroup 1 - community mobilization strategies; Subgroup 2 - behavioral change communication strategies; Subgroup 3 - capacity-building and training strategies; and, Subgroup 4 - program management and sustainability strategies. The subgroups discussed the indicators set forth in the DIP and how the strategies had been implemented, determined what additional information was needed, and prepared semi-structured interview guides (for Focus Group Discussions and Key Informant Interviews) to obtain the necessary additional information. Then the Subgroups developed a three-day field visit plan. Finally, after conducting the semi-structured interviews for three consecutive days, the team met again to triangulate quantitative and qualitative data and to develop some overall conclusions and recommendations for the project. The main findings, conclusions, lessons learned, recommendations, and commitments for ongoing MCSP activities are presented below, first by technical intervention and second by strategies and approaches. I. Nutrition and Breastfeeding: Percentage of total project effort: 35% (DIP) Objectives The objectives of this intervention were to increase early initiation of breastfeeding and exclusive breastfeeding, increase the use of micronutrient supplements among children and mothers, and prevent malnutrition associated with poor feeding practices and illness. 3 Guidelines for Final Evaluation. PVO Child Survival and Health Grants Program, USAID/GH/HIDN/NUT. July 2004 12 Indicators: · 50 % of postpartum mothers began breastfeeding within one hour of delivery · 50 % of infants aged 4-6 months had been breastfed exclusively in the preceding 24 hours · 50 % of children aged 6-9 months received breast milk and complementary foods in the preceding 24 hours · 50 % of children between the ages of 12 and 59 months had been given a Vitamin A supplement · 60 % of pregnant women and 70% of lactating women had received iron supplements · 50 % of lactating mothers were given a Vitamin A supplement within 4 weeks postpartum · 10% decrease in the percentage of children below –2SD from the median weight for age (according to the WHO/NCHS reference population) · 40 % of children 0-23 months old received the same or increased levels of fluids, food, and breast milk during their most recent illness. The MCSP used a community-centered approach and has worked to create a sustainable impact through the following strategies: Strategy 1: Community Mobilization for Health Strategy 2: Community Health Education Strategy 3: Promoting Public/Private Partnerships Strategy 4: Capacity-building Table 1: Comparison of Breastfeeding and Child Nutrition Indicators at baseline, midterm, and final evaluation. Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Breastfeeding Initiation Percentage of children aged 0-23 months who were breastfed within the first hour of birth 50% 34% 60% 76.4% (71.2% – 81 6%) Percentage of infants aged 0-5 months who were fed breast milk only (no water) in the last 24 hours 72.7% (62.2% – 83.2%) Exclusive Breastfeeding Rate Percentage of infants aged 0-5 months who were fed breast milk only (including water) in the last 24 hours 50% 45% 58% 81.3% (72.1% – 90.5%) Complementar y Feeding Rate Percentage of infants aged 6-9 months who received breast milk and solid foods in the last 24 hours 50% 77% 87% 90.2% (82.4 %– 92.0%) Continued Breastfeeding Percentage of children aged 20-23 months who are still breastfeeding - - - 62.5% (45.9% – 79.1%) Ever Breast fed Percentage of children under 24 months of age who were breastfed 97% 96% 97.5% (95.6% – 99.4%) Percentage of pregnant women who received iron supplements 60% 20% 31% Micronutrient Coverage Percentage of lactating women who received iron supplements 70% 20% 41.6% Percentage of children aged 6–23 months who had received a Vitamin A dose in the preceding six months 50% 36% 96% 69.2% (63.6%-74.8%) 13 Table 2: Comparison of Growth Monitoring and Anthropometric Indicators at baseline, midterm and final evaluation. Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Growth Monitoring Card Possession Percentage of children aged 0-23 months who have a growth monitoring card 47% 73% 88.1% (84.1 – 92.1%) Percent Underweight Percentage of children aged 0-23 months who are less than 2 standard deviations below the median weight￾for-age of the reference population 22% 25% - 16.6% (13.3% - 19.9%) Weighed at Birth - 17% - 67.8% (62.1% – 73.5%) The nutrition and breastfeeding intervention met and exceeded seven out of eight programmatic targets as demonstrated by comparing baseline, midterm, and final data (see Tables 1 and 2). The overall malnutrition rate in children under two years of age decreased by 34% (target 10%). Initiation of breastfeeding within one hour of birth increased by 42 percentage points and exclusive breastfeeding of children under 6 months increased by 28 percentage points. The rate of complementary feeding of children 6 - 9 months of age rose 13 percentage points, and there was a 41 percentage point increase in growth monitoring coverage. The target for micronutrient coverage in children was met and exceeded by almost 20 percentage points. However, the target for iron supplement coverage4 for pregnant and lactating women, although improved relative to baseline data, was not met. Summary of the main points discussed by the Evaluation Team in reviewing the project outcomes as measured by comparison of the Baseline and Final Evaluation Surveys, including factors that contributed to whether established targets were met or not: Strategy 1: Community Mobilization for Health. Key elements included the development of strategic plans to address the causes of poor health; implementation of activities to increase community utilization of health services provided mainly at the health station and by BHWs and midwives; and, advocacy efforts directed toward local authorities for improved services. Community participation contributed to improved health services. The quality of services was enhanced by the provision of equipment and/or supplies and, in some areas, previously nonexistent services were provided. Local financial support was leveraged through local government units, either for equipment and supplies or, in some cases, for staff salaries. Finally, active case detection and follow-up of malnourished children played a significant role in bringing down malnutrition rates. 4 Data was obtained from IPHO-Modified Field Health Service Information System (FHSIS – Annual 2004 except December). 14 Strategy 2: Health education. Key factors in the health education strategy included the design of an information, education and communications (IEC) program based on qualitative research conducted at the beginning of the project. Likewise, the classification of target audiences by type of message, coupled with a multi-channel dissemination mechanism, contributed significantly to the behavioral change impact. Health messages were deliberately kept simple, tailored to each target audience, and they were behavior-centered and action-oriented. The project used a combination of health awareness activities through the mass media (radio spots) and group discussions, in addition to intervention-specific messages conveyed through household visits and individual counseling. For example, the project produced and distributed IEC materials, comics, flipcharts, posters, and prenatal care cards, and broadcast radio spots containing specific messages and announcements. This was complemented by community education delivered by BHWs, distribution of IEC materials, home-based education, community lectures and discussions, demonstrations of breast feeding, weaning, and childhood nutritional practices, and individual counseling. According to project staff, radio spots and education through the mass media were more effective for raising awareness about health, but less significant in terms of changing or improving health-related behaviors. These informants reported that face-to-face education was the method of choice for bringing about behavioral changes. Strategy 3: Promoting public/private partnerships. The project was successful in involving Local Government Units (LGU), which established and/or constructed Health and Nutrition Posts in conjunction with the local people’s committees. In fact, 45 HNPs were constructed during the life of the project with funds and manual labor provided by the communities and LGUs. KFI played an important role in strengthening the capacity of the LGUs and municipal governments in the areas of strategic planning and implementation. Community leaders included health activities on their broader agenda, which in most cases included supporting barangay HNPs and community-based personnel. Strategy 4: Capacity-building. Local partners (LGUs) and municipal level partners have developed skills in health program planning and management. Not all LGUs and municipal governments have advanced to the same degree, however; as is to be expected, some have made more progress than others. In some cases, local authorities have carried out regular management and follow-up of BHWs and field supervisors. The implementation of the IMCI strategy at the health facility and community levels also has contributed to the improved quality and growing utilization of health services. Limiting programmatic factors: Volunteer staff attrition: The FE team observed that many BHWs left for other jobs after being trained by the project. Future programmatic interventions would benefit from a systematic review of incentives for volunteer workers, as well as community involvement in retaining such workers. The project needs BHWs to remain for a certain minimum period of time in order to ensure the viability and cost-efficiency of training expenses and equipment provision. Project records showed a volunteer staff attrition rate of 3.8% (5 out of 130 BHWs). 15 Overall conclusions, recommendations, and lessons learned as identified by the Evaluation Team: The project’s nutrition and breastfeeding component consisted essentially of two activities: behavioral changes associated with nutrition, primarily during the first year of life, and health services delivery in the community, including nutritional counseling related to illness and growth monitoring. Since the overall impact of this component was associated more with community￾based than institutional-level activities, there was less improvement in indicators more closely linked to health facility activities, such as iron and vitamin A distribution. The health education strategy was successful in changing behaviors in the community relating to breastfeeding and early childhood nutrition. Radio was found to be an effective channel for generating health and nutrition awareness, while printed materials were helpful as visual aids in face-to-face educational activities. The use of various communications channels or mechanisms is highly effective in influencing behavioral change, as each one has different advantages and contributes to a complementary approach to behavioral change. The IPHO-KFI partnership proved to be mutually beneficial. KFI spearheaded activities to strengthen community organization and support health and nutrition activities. The IPHO, for its part, successfully implemented health and nutrition interventions using community-based workers, thereby optimizing public health resources. Health Committees were organized. The final evaluation, however, was not fine-tuned enough to detect variations between the 45 communities initially selected and other local communities. After piloting the Hearth model (described below), field teams concluded that it constituted an important contribution to the overall quality of health education and appropriately involved families and communities in improving the nutritional status of children. Nevertheless, since the Hearth model was not introduced until after the first half of the project, the team felt compelled to pursue the strategies proposed in the DIP and endorsed during the Mid-term Evaluation (MTE); as a result, there was not enough time to launch a full-blown Hearth model. II Pneumonia Case Management: Percentage of project effort: 25% (DIP) Objective: The objective of the PCM intervention was to reduce mortality and morbidity from pneumonia in children under five years of age (with special emphasis on the very young infant) using Standard Case Management (SCM). Indicators · 70 % of children with pneumonia were treated with appropriate antibiotics. · 60 % of mothers with children under 2 yrs old with rapid and difficult breathing sought immediate care from a health care provider. · 60 BHWs have been trained in pneumonia identification · 60 % of trained BHWs could correctly assess and refer pneumonia cases. 16 This project intervention included three specific objectives: improve the quality of pneumonia standard case management, improve access by leveraging community resources, and positively influence care-seeking behavior. To achieve these objectives, the project used the IMCI approach at the institutional and community levels, applying the following strategies: Strategy 1: Community Mobilization for Health. The same approach described in the previous section was used to mobilize the community around each project intervention. Key components included the development of strategic plans to address the causes of poor health, implementation of activities to increase community utilization of health services (usually provided at the health station and by BHWs and midwives), and advocacy efforts directed toward local authorities to obtain better services. Strategy 2: Health Education at the Community Level. Based on the BCC plan, messages promoting prompt care-seeking behaviors were disseminated to caregivers through various channels. At the community level, for example the project engaged in community education through face-to-face counseling, social communication, and home visits. Health messages were kept simple and stressed the importance of “giving additional food and plenty of liquids to a child with a cough and cold” and “taking the child to the health center if he/she has a convulsion, will not eat or drink, is breathing rapidly, has stridor, and/or the measles.” Strategy 3: Promoting Public/Private Partnerships. Improved quality of care delivery at the health facility and community levels was complemented by KFI’s efforts to strengthen community structures for mobilizing people around care-seeking behaviors and health services utilization. The IPHO reported that the MCSP area had a sufficient supply of cotrimoxazole for a complete course of treatment. LGUs and municipal governments supported the establishment of HNPs and provided additional funds for health workers to conduct community visits. Strategy 4: Capacity-building. The MCSP adopted a general IMCI (health facility and community-based) strategy for pneumonia management. The project also aimed to strengthen the clinical capacity of IPHO rural health centers through the provision of basic medicines and case management protocols. Health facility personnel received IMCI training following the national training protocol. CRS Philippines, in collaboration with headquarters and DOH, adapted CRS’ C-IMCI handbook to train Barangay Health Workers (BHW). Pneumonia case management was conducted exclusively at the health facility, while identification and referral occurred at the community level. Training was enhanced by immediate follow-up and on-site performance evaluations. In addition, some communities built health and nutrition posts, thereby facilitating access to health services for rural families. The project trained BHWs in case management, health education, and interpersonal communication skills and provided them with an information kit. BHWs maintained direct, regular contact with the mothers and optimized their health education activities. 17 Table 3: Comparison of maternal knowledge of danger signs indicators at baseline, midterm, and final evaluation. Outcomes: Sick Child Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Maternal Knowled ge of Danger Signs in Children Percentage of mothers of children aged 0- 23 months who recognized at least two signs of childhood illness indicating the need for treatment 50% 64% 86% 95.3% (92.7% - 97.9% Table 4: Comparison of Acute Respiratory Infection care-seeking behaviors and quality of care indicators at baseline, midterm and final evaluation. Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Percentage of children aged 0-23 months with a cough and rapid/difficult breathing in the preceding two weeks who were taken to a health facility or received antibiotics from an alternative source 70% - - 83.1% (75.4% – 90.8%) ARI Care￾seeking Health facility: Alternative source: 78.9% 12.4% Percentage of mothers with children under 2 years of age with rapid/difficult breathing who seek immediate care from health care providers 60% 70% 57% 78.7% (70.3% – 87.1%) Quality of health care Percentage of BHWs trained in pneumonia identification 60% 0% 100% Percentage of trained BHWs who correctly assess and refer pneumonia cases 60% 0% 50% Where help was sought: Hospital 29% 1.9% Health Center 47% 78.9% Barangay Health Worker 9% 12.4% Other 2% 2.5% Traditional Practitioner 13% 2.5% Who decided where to take the child for care: Respondent 81.0% Husband 37.3% R.’s Mother 6.3% Mother-in-law 3.8% Friends 0.6% Other 1.3% The data collected during the final KPC showed that all key indicators exceeded the targets set at the start of project implementation (See table 3 and 4). The project improved case identification skills at the household and community levels, access to prompt treatment, and quality of care in health care facilities and in the community. Mothers of children under 2 years of age improved their ability to recognize danger signs in their children by 31 percentage points (see Table 3). 18 ARI treatment was received by 83% of children under two by the end of the project, a figure which exceeded the target by 13 percentage points. Care-seeking behavior, although not statistically significant, improved almost 9 percentage points over baseline data. The data also show that the majority of the mothers (78.9%) took their sick child to the health center for care, while 12% turned to a BHW. This is a good indication that mothers have increasing confidence in the ability of rural health service-providers to provide care. Specifically, all BHWs in the target municipalities were trained in pneumonia identification and, based on observation, 50% of them could correctly assess and refer pneumonia cases. It is interesting to note that mothers are the main decision-makers regarding their children’s health care, contrary to the popular assumption that the father is responsible for most family decisions in this setting. In addition, the IPHO health information system showed that the number of pneumonia cases treated with antibiotics exceeded the 70% targeted. The same source also showed that 91 BHWs were trained in pneumonia identification, compared to the target number of 60. There were two main outcomes concerning utilization of health services: first, utilization increased as evidenced by the high rate of ARI treatment (almost 90% of cases) and second, the point of health care delivery changed significantly. Specifically, service delivery by a physician decreased 27 percentage points. At the same time, health care delivery by a midwife (health center) rose 32 percentage points, while delivery by a BHW increased by 3.4 percentage points. These figures point to a decentralization of health care delivery as a result of implementation of the IMCI approach at the health center and community levels, with the attendant improvement in quality of care. From this we can infer that the project contributed to improved health care quality and coverage as well as to decreasing health care costs in this region of the Philippines. Summary of the Evaluation Team’s main points and recommendations based on its review of project outcomes, as measured by comparing the Baseline and Final Evaluation Surveys, including factors that contributed to positive outcomes and those that did not: The project led to improvements in the following areas: caregiver awareness of danger signs for pneumonia; health services utilization at the community and health facility levels; quality of health services (including staff technical skills and medication availability); and, efficiency in health services delivery. This was accomplished through a systematic approach to community mobilization, multi-channel dissemination of simple, incisive health messages, well-defined roles in the public-private partnership, and implementation of the IMCI strategy. The team made several recommendations to scale up and expand the impact of interventions: data analysis and management should occur at the RHU level before being forwarded to the IPHO; trained BHWs can transfer their skills to other barangays by assisting with training; and, health plans should be integrated into barangay and municipal development plans. It further recommended that community authorities participate in monitoring activities. Limiting factors identified by the team included: (1) limitations on physical access; (2) precarious situation of peace and public order resulting in erratic visits by the midwife to the area and the evacuation of families; (3) economic barriers: mothers are unable to take their children to 19 the RHU even with a BHW referral, due to lack of money; and (4) negative attitudes and distrust of health workers on the part of mothers and caregivers. III Control of Diarrheal Diseases: Percentage of project effort: 25% (DIP) Objectives: To prevent cases of child diarrhea and dehydration through prompt treatment of diarrhea in the home, prompt care-seeking for complicated cases, and prevention of malnutrition through appropriate feeding of the child during and after illness. Indicators · 40 % of children 0-23 months old with diarrhea in the preceding two weeks received the same or higher amounts of fluids, food, and breast milk than usual. · 70 % of caregivers of children under two years of age with diarrhea sought immediate advice or treatment from a trained health provider · 100 % of health facilities (health posts and RHU) have functioning ORT corners · 80 % of BHWs trained in CDD Home Case Management Strategy 1: Community Mobilization for Health. The project employed a systematic method of organizing and empowering community structures to foster local support for ORT corners and water and sanitation activities. The latter received financial support from local government units. Strategy 2: Health Education at the Community Level. Using a general BCC approach, the project developed health messages on dehydration prevention, improvement of community sanitation, and hand-washing in key moments (after defecating, and before and after food preparation). Strategy 3: Promoting Public/Private Partnerships. The project sought to strengthen collaboration between communities and local governments around improving access to water systems and sanitation infrastructure. Using community social funds, and in collaboration with the barangay people’s organization, CRS implemented latrine construction. Strategy 4: Capacity-building. Diarrheal diseases were targeted through implementation of a basic IMCI strategy at the health facility and community levels. The MCSP focused on strengthening the skills of health facility staff in IMCI, including diarrhea-case management. At the community/barangay level, the project aimed to train BHWs as educators/trainers for home-based diarrhea management. Another project goal was to set up oral rehydration therapy (ORT) corners in health facilities. 20 KPC RESULTS: Table 5: Comparison of access to and availability of appropriate treatment for CDD indicators at baseline and final evaluation. Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) ORT use during a diarrheal episode Percentage of children aged 0-23 months with diarrhea in the preceding two weeks who received oral rehydration solution (ORS) and/or recommended home fluids (RHF) - - - 72.1% (56.2% – 88.0%) Quality of care Percentage of BHWs trained in CDD home case management 80% 0% - 100% Health care availabili ty Percentage of health facilities with functioning ORT corners based on WHO standards 100% 8% - 100% Health Educatio n Percentage of women who had heard about and could describe ORS preparation (out of the total # of women with a child experiencing diarrhea) - 46% - 90.7% Table 6: Comparison of access to and use of appropriate treatment for CDD indicators at baseline and final evaluation. Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Increased fluid intake during a diarrheal episode Percentage of children aged 0-23 months with diarrhea in the preceding two weeks who were offered more fluids during their illness 40% 31% - 80.3% (65.7% - 94.9%) Increased food intake during a diarrheal episode Percentage of children aged 0-23 months with diarrhea in the preceding two weeks who were offered the same amount or more food during the illness 40% 31% - 78.7% (63.4% - 94.0%) Increased fluids and feeding during an illness 40% 31% - 75.4% Care￾seeking for diarrhea Percentage of children aged 0-23 months with diarrhea in the preceding two weeks whose mothers sought outside advice or treatment for the illness 70% 40% 71.2% 78.7% Where help/advice was sought Hospital 2.1 Health Center 56.3% Private Clinic 2.1% Barangay Health Worker 33.3% Traditional Practitioners 6.3% 21 Table 7: Comparison of water and sanitation indicators at baseline and final evaluation Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Safe Drinking Water Percentage of households with deep-well drinking water - 10.5% - 24.8% (19.5% - 30.1%) Sanitary Excreta Disposal Percentage of households with access to a water-seal toilet - 31.2% - 49.7% (43.6% - 55.8%) Hand￾washing Facility Percentage of households reporting a specially designated place for hand washing - - - 97.9% (96.1% – 99.7%) Table 8: Comparison of diarrheal disease in children under two at baseline and final evaluation Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Diarrhea prevalence Percentage of children <2 with diarrheal disease within the preceding two weeks - 22% - 10% Conclusions, recommendations and lessons learned identified by the Evaluation Team (ET): The CDD intervention decreased by 12 percentage points the overall prevalence of diarrheal disease in children under two years of age (see Table 8). All four target indicators used to measure the impact of the CDD intervention were met and exceeded by the end of project implementation. Specifically, children under two with diarrhea in the preceding two weeks who received the same amount or more fluids and food increased by 44.4 percentage points, while caregivers of children under two with diarrhea during the preceding two weeks increased their care-seeking behavior by 39 percentage points. At the same time, the availability of functioning ORT facilities increased by 92 percentage points, and the availability of trained community health workers rose by 100 percentage points (see Tables 5 and 6). The data collected during the baseline assessment and final evaluation facilitated a thorough analysis of the impact of program interventions on improving access to health care (service availability and service utilization) and to preventive services. As Donabedian5 observed in his 1972 study on health care access, “the proof of access is use of service, not simply the presence of a health facility.” CRS Philippines used CORE’s C-IMCI model6 to address issues of health services supply and demand. On the supply side, ORS was available to 72% of the total population of children under two years of age, the availability of trained community personnel increased by 100 percentage points, and the availability of ORT services at health facilities rose by 92 percentage points. At the same time, knowledge about diarrhea home case management grew by 45 percentage points, 5 Donabedian A. (1972) ‘Models for Organizing the Delivery of Personal Health Services and Criteria for Evaluating Them’, Milbank Memorial Fund Quarterly, 50: 103-54. 6 Winch P, Leban K, Kusha B, 2001. Reaching Communities for Child Health and Nutrition: a Framework for Household and Community IMCI. Calverton MD: Child Survival Technical Support Project 22 while access to water sources increased by 14 percentage points and to latrines by 19 percentage points. (The project constructed 329 water pumps in the five target municipalities, with 5 -10 households per water pump, and funded two freshwater spring development projects and one water rehabilitation infrastructure project). On the demand side, the project contributed to a 49 percentage point increase in the use of fluids during diarrheal episodes, while proper feeding practices during diarrheal disease rose 48 percentage points, and behaviors seeking outside treatment by 39 percentage points. In addition, 98% of households reported having a designated hand-washing area. Health service utilization patterns were decentralized by the process of supply and demand. Specifically, utilization of centralized services (hospital/physician) declined by 2.1%, accompanied by an increase in the use of peripheral services (health center/midwife - 56% and community/BHW - 33%) (see Table 6). This decentralization process had the additional effect of improving the cost-effectiveness of health service delivery in the provincial health care system. Summary of the main points and recommendations of the Evaluation Team based on its review of project outcomes as measured by comparing the Baseline and Final Evaluation Surveys, including factors that contributed to positive outcomes and those that did not: The final evaluation team agreed that CORE’s operational model, C -IMCI, with its multi￾sectorial approach, played a key role in achieving this substantial increase in access to appropriate CDD services in the project area. At the same time, the Ministry of Health’s process of adopting and adapting CRS’ C-IMCI management tool7 promoted local ownership and sustainability at the level of local health structures. The project’s activities had an impressive and somewhat unanticipated impact on the diarrhea situation in the area. The team stressed the importance of local government and inter-sectorial support for complementary activities such as infrastructure provision (health post construction, water and sanitation infrastructure) in achieving sustainable health gains in communities. The Local Government Units (LGUs) actively supported the effort by providing stipends for BHWs and salaries for some municipal￾level midwives. Regardless of whether the project continues, the ET recommended sustaining the partnerships, strengthening CBHO networking capacity, involving religious groups -particularly in advocacy efforts-, and continuing the use of ORT corners. Water and sanitation education also included cleaning up the immediate surroundings, especially water sources and food handling areas. The strategy used was called “TKO,” a Philippine acronym that stands for the synergistic effect of toilet facilities and ORT. The IPHO nutritionist who supervises the RHUs on a monthly basis reported that all (100%) of the health facilities (5 RHUs, 15 BHSs, 46 HNPs) had a functioning ORT corner based on WHO standards. IV. Expanded Program of Immunizations (EPI): Percentage of project effort: 15% (DIP) Objectives: The objective of the EPI intervention was to increase vaccine coverage of children under the age of 24 months and pregnant women. 7 Rosales A, Weinhauer K, 2003. C-IMCI Handbook. Catholic Relief Services 23 Indicators: · 70 % of 9-11 month-olds are fully immunized children (FIC) · 70 % of pregnant mothers received at least two doses of Tetanus Toxoid · 70 % of health facilities maintain vaccines at the appropriate temperature · 80 % of health staff has been trained in Basic EPI, Cold Chain Management, Disease Surveillance, and TQM Strategy 1: Community Mobilization for Health. Community/barangay health workers, including TBAs, provided information and carried out community mobilization activities on a well-defined immunization schedule Strategy 2: Health Education at the Community Level. The project used radio broadcasts and other IEC methods to disseminate messages informing communities about the benefits of the EPI program and when to immunize children. Strategy 3: Promoting Public/Private Partnerships. LGUs provided transportation funds and materials for midwives, especially during national immunization days. Strategy 4: Capacity-building. The MCSP helped the IPHO staff upgrade their skills in EPI, particularly in the areas of disease surveillance, cold chain management and total quality management. The project also reinforced the “cold chain” by providing equipment and other materials and through its efforts to improve cold chain monitoring and maintenance. Trainings also included community/barangay health workers and TBAs. Immunizations were part of the IMCI protocols at both the health facility and community levels, so health workers could check a child’s immunization status. Table 9: Comparison of Childhood Immunization Indicators at baseline and final evaluation. Indicator Description/Definition Target Baseline Midterm F. Eval. (CI) Vitamin A coverage Percentage of children aged 6-23 months who received a Vitamin A dose in the preceding six months 50% 36% 96% 69.2% (63.6% – 74.8%) Possession of Vaccination Card Percentage of children aged 0-23 months who had a vaccination card - 36.4% - 87.7% (83.7% – 91.7%) EPI access Percentage of children aged 12-23 months who received DPT1 (out of the total # of children aged 12-23 months with cards) - - - 99.4% Measles vaccination coverage Percentage of children aged 12-23 months who received measles vaccine (out of the total population of children 12-23 months old) - 27% 79% 77.9% (70.0% – 85.8%) Fully Immunized Children Percentage of children aged 12-23 months who received OPV3, DPT3, and measles vaccines before their first birthday (out of total # of children aged 12-23 months with cards) 70% 58% 76% 57.3% (50.70% - 63.9%) TT coverage Percentage of pregnant women who received at least two TT doses 70% 47% 84.4% Health Facility Percentage of health facilities storing vaccines at appropriate temperature 70% 30% - 80% Health Staff Percentage of health staff trained in EPI 80% 14% 100% 24 The overall conclusion, based on the data collected, is that the project was unsuccessful in improving utilization of immunization services in children under two years of age. This conclusion is based on the fact that the figure for FIC did not reflect any quantitative change from baseline (58%) to final (57.3%) data. Conversely, the availability of immunization services increased during the life of the project. Access to EPI reached 99.4% (DPT1 coverage) by the end of the project, possession of a vaccination card increased 51.3 percentage points, and the quality of services improved as measured by the percentage of trained health staff and a proxy indicator for the proper cold chain. Nonetheless, utilization of vaccination services did not follow service availability. To date, the project has been unable to explain this finding and further qualitative research is required. We believe the most plausible explanation is the “missed opportunities” factor at the health facility level, since there is a discrepancy between measles vaccination coverage and FIC. When both indicators are compared using the same denominator (total # of children with vaccination cards), measles coverage rises to 95.3%, which correlates with DPT1 coverage. An alternative explanation is that the program relied mostly on two factors: improved service availability and health awareness promotion. We believe that the BCC component for this intervention did not use individual counseling as frequently as in the other three interventions and therefore its impact in creating demand for this type of services was insufficient to reach the targets. Moreover, the public-private partnership strategy, so successfully applied in the other three intervention areas, was barely mentioned during interviews for this particular component, leading us to believe that implementation was concentrated on immunization campaigns. We should note that the 2004 IPHO-FHSIS (Field Health Service Information System) reported 81% FIC coverage in the 5 municipalities covered by the project. Another finding worth noting is that when FIC information is analyzed by municipality, every municipality shows an increase in FIC, except for the municipality of Sultan Sa Boringis, which showed a decline from baseline to final data (76.5% to 50.8%) and Mamasapam, where the change is not statistically significant. (See table 10) Table 10: Comparison of FIC indicator at baseline and final by municipality. Municipalities Talitay Kabntalan SSB Mamasapam South Upi Baseline 0.0% 50% 77% 50% 67% Final 30% 68% 51% 54% 86% Tetanus Toxoid coverage for pregnant women increased 37.4 percentage points from baseline to final, meeting and exceeding the target of 70% coverage. Likewise objectives concerning improved quality of EPI service objectives were reached. (See table 9). Finally, Vitamin A supplementation increased from 36% to 69% when comparing baseline with final evaluation survey results. 25 Summary of the main points and recommendations of the Evaluation Team in its review of project outcomes, as measured by comparing the Baseline and Final Evaluation Surveys, including factors that contributed to positive outcomes or those that did not: The barangay health and nutrition posts (HNP) played a key role in mobilizing communities and educating families about EPI. BHWs carried out home visits before scheduled immunizations to remind families, followed up after the campaigns, and kept rosters of newborn children. Moreover, TBA and BHW training included home visits, and the C-IMCI chart booklet included assessing immunization status and providing the vaccine. It appears that the IEC strategy in the immunization intervention focused more on information than education; hence its limited impact on behavior changes in the area of immunization. The Provincial Health Officer was deeply committed to improving EPI access and coverage throughout the province. He therefore involved all external organizations and local governments in immunizations campaigns and regular activities at the community and health facility levels. CRS provided 10 motorbikes, a horse in one municipality (South Upi), and boats (Kabuntalan) to improve access to remote villages. The ET identified the following obstacles to achieving the desired outcomes: (1) a shortage of health personnel at the beginning of the project, which was partially solved by the project itself and by LGU involvement in which the provincial government and municipal LGUs of Sultan sa Barongis and Mamasapano hired eight additional RHMs to provide regular health services to the Barangays; (2) while slight improvement in the situation of peace and public order was observed, this continued to be a problem in some barangays; (3) the transportation allowances for health staff were insufficient to reach some of the most remote barangays. The team’s main recommendations to improve utilization of immunization services are: first, to reexamine the BCC approach in this project component and incorporate activities other than information dissemination including IEC initiatives that address community attitudes and behaviors towards immunization. Second, the program needs to promote the public-private partnership in implementation of this component; activities may include participatory health plan design, the establishment of clear targets, goals, and objectives, and capacity-building needs. Also, the evaluation team recommended strengthening the people’s organizations so that they can be duly registered and request external funding in support of immunization activities. c. New Tools or Approaches During the course of the project, some instruments were developed to improve the quality of counseling and service delivery: · Quality Improvement Checklist: During the course of the project, a health education observation checklist was developed. It was mostly used as a guide for health workers to improve the health education and face-to-face counseling provided to mothers. · Community-Based IMCI tools: The project developed a simplified version of the C￾IMCI protocol for management of the main childhood illnesses. The protocol was used for training community-based health workers and used by the latter to assess, classify, counsel, and refer. The tool was developed and tested in conjunction with the central and 26 provincial health office. Once the experience had been documented, the Ministry of Health adopted the approach and tools for implementation at the national level. Additionally, the project developed and implemented the following on-the-job aids and counseling materials: C-IMCI chart booklet; caretaker interviews for determining quality gaps; a health facility checklist; IEC materials (flipcharts, health information kits, posters, flyers); and radio spots. · Crossover experiences. One of the project’s more innovative approaches was the transfer, or crossover, of experiences between municipalities. For instance, some LGU members traveled to other municipalities in the country to learn more about LGU involvement in health programs and, specifically, the establishment of health and nutrition posts. · Municipal Health Working Group (MHWG). The project assisted the people’s organization to develop a MHWG. Its main functions are to design a local development investment plan for children’s health, and to create an Executive and Legislative Agenda to obtain support for BHW salaries. · PD-Hearth Model. In October 2002, the project conducted a monitoring visit with technical assistance from Pearl S. Buck International, Inc. One purpose of the visit was to assess the pilot implementation of the Hearth Nutrition Program in Maguindanao (see the complete report by Pearl S. Buck International Inc., a consultant firm). The MCSP had selected one barangay per municipality to pilot the PD-Hearth model. The results of the monitoring showed that children enrolled in the program had gained weight, and mothers reported that their children “looked healthier and more playful.” The same report mentioned that the anthropometrical indicators showed remarkable improvement within four months of program implementation. Improvements were also observed among caregivers, who adopted better nutrition and hygiene practices. The Hearth Model also promoted team cohesion and strengthened community activities. The monitoring team offered several recommendations to scale up and expand the model to other barangays, to improve the quality of the model and monitoring systems, and to document this pilot experience. During the final evaluation, CRS staff reported that they could not expand the Hearth model to other barangays because the field teams were already overloaded with activities. Therefore, implementation of the Hearth Model remained as a pilot project. · Social Health Insurance model. An inter-project visit to the CRS Health Program in another region of the country afforded KFI chief officers and some community leaders a basic orientation on the implementation of a social health insurance model. As a result, 15 CBHOs launched their own health savings program to provide financial assistance for transportation and medicines for sick members, using the accumulated contributions. · Decentralization Approach to Community Health: By empowering community structures (a BCC focused on danger signs, increased community resolution capacity through C-IMCI, and improvement of community access to local health services) the project promoted a decentralization process whereby health services delivery was shifted from health facilities to the community. As a result, service utilization declined by 54 percentage points at the central health facility/physician level and increased by an average of 49 percentage points at the peripheral level (41 percentage points for midwives and 56 percentage points for BHWs) (see Table 11). This decentralization 27 process improved the cost effectiveness of health services delivery in the provincial health system. There is a clear trend when comparing baseline and final data as to who constitute the main sources of information and health messages, and whom mothers consulted in the month prior to the Final Evaluation survey. Table 11. Comparison of health personnel utilization at baseline and final evaluation. Indicators, Description/Definition Target Baseline Midterm F. Eval. Less than 60 minutes to reach the health facility - 75% - 89.0% Consulted a doctor during the preceding month - 76% - 22.3 Consulted a midwife during the preceding month - 47% - 87.9% Consulted a BHW during the preceding month - 9% - 65.3% Sources of Information/General Information on Health: Doctor 31.2% Nurse/Midwife 66% 98.2% BHW 16% 83.4% TBA 48.4% “Quack” Doctor 3.0% Husband 55.7% Mother 53.4% Sister 31.8% Grandparent 26.5% Aunt 24.9% Friend 40.7% Traditional Healer 20.8% Village Elder 18.6% Sources of Health Messages in the Past Month: Radio - 71% - 87.5 Newspaper 7.6 TV - 15% - 33.7 Comics, Posters, Leaflets, Flyers 57.6 BHW - 13% - 77.7 Midwife 85.7 Nurse/Doctor 37.6 3. Outcomes: Cross-Cutting Approach To examine project outcomes from a crosscutting standpoint, the evaluation team used the second day of the technical meeting to discuss the five areas (see Annex A for the complete list). During the first part of the meeting, the evaluation team discussed conclusions and recommendations and identified information gaps, or data that had to be collected or confirmed through in-depth interviews with key personnel from Local Government Units and partner organizations. The main findings, conclusions, lessons learned, and recommendations for continuing the activities initiated by the MCSP are presented below. 28 a. Community Mobilization General Approach Improving community capacity to manage its own health care was one of the main anticipated outcomes of the Community Mobilization strategy. At the family level, the project hoped to contribute to behavioral changes and better practices that would benefit the overall health of the household, and particularly that of children under five. The project used participatory, community-centered outreach and development approaches to optimize participation and learning. The phases and steps of the MCSP model are presented below: Stage 1: preparatory or community pre-entry phase. This phase involved a series of consultations and discussions among the partners to define the criteria for barangay selection. Stage 2: barangay entry process. The purpose, in this phase, was to ensure that the community (barangay-assembly) had a general understanding of the nature of the project, its objectives, development activities and strategies, and local participation. Stage 3: barangay mobilization - planning phase. The objective during this phase was to ensure active participation of the community, through their representatives, in all stages of the MCSP development process; this was accomplished through a series of workshops and planning. Stage 4: community health mobilization and capacity building. A series of community mobilization activities were launched during this phase. Stage 5: networking/linkage-building. This phase featured the emergence of functional community-based organizations and communities established and managed at least one income-generating activity to finance their own health needs. Stage 6: exit phase. This provided a formal exit process which consisted of: assessing the communities’ experience with the project, highlighting lessons learned and improvements in community development processes and methodologies; a re￾planning process; and the formal turnover of projects to the CBHO, BDC, and MTWG. This phase was still underway in some barangays when the FE took place. Final Evaluation Results Of the 76 barangays, 45 barangays were chosen to follow the six-phase strategy implemented by KFI. The Mid-term Evaluation showed that 12 out of the 45 barangays were duly registered with either the Security and Exchange Commission (SEC) or the Department of Labor (DOLE), while the remaining 33 were organized but not formally registered. In any event, 32 CBHOs constructed and installed Health and Nutrition Posts and obtained support from LGUs in the form of materials, food, funds, and labor. Moreover, 41 CBHOs installed water projects in their respective barangays by organizing families around water system maintenance. All of the CBHOs mobilized their communities around health activities and classes for mothers. 29 Finally, 20 CBHOs designed some type of health savings plan, mainly to provide funds for the emergency health needs of local residents, such as hospitalization and purchase of medicines. Other health related activities conducted by the CBHOs were home gardening, communal farming, and construction of sanitary toilets. The FE team arrived at the following outcomes and recommendations regarding community mobilization: ß The community mobilization process was not clear in some communities at the beginning of the project. However, not all communities needed to follow all the steps, and some would operate based on the initial situation of that particular community; ß Community-based health organizations need to be cognizant of the different indicators used in monitoring their activities so that they can rate their own progress. The project could develop self-assessment tools. ß The ideal community organizer ratio is 1 CO per 2 barangays, but this is also contingent upon the skills and experience of the CO. The project ratio of 1 CO per 5 barangays was insufficient. Other factors to consider are access, LGU commitment, and the presence of a traditional leader who could assist the CO. The CO can use coaching or mentoring methods to train local leadership to continue community mobilization. ß The municipal government and barangay LGUs play a critical role in ensuring that programs are incorporated at the institutional level, and therefore, they should be involved in all activities. Their support and commitment should be reflected in a legal instrument such as a memorandum of agreement, resolutions, or executive order. ß After creating their community health plans, the CBHOs had difficulty obtaining support from the LGU because the health plans were not part of the barangay and municipal development plans. Plans should be integrated into the broader framework to sustain health activities. It is important to work with the development councils (officially mandated local specialized bodies). ß The BHWs are usually members of the CBHO and often chair or hold an elected post (auditor, treasurer or secretary) in the health committee; this proved advantageous for mobilizing the membership and ensuring feedback at monthly meetings. ß Finally, most CBHO members were male. While they were largely unaware of the importance of having women members, when asked about women’s participation, they all agree that women could bring several skills to the process: “women are good in planning and budgeting,” “ they should also receive the health benefits,” “ they should know more about the health program because they are the ones taking care of the children,” and so forth. Therefore, women’s participation must be endorsed from the outset of the project. Sustainability Plans CBHOs will continue to meet regularly based on their established schedule. A financial report will be made to the members at every scheduled meeting to practice transparency. Proper recording and accounting procedures will be used for all funds collected by the 30 treasurer and all funds are to be deposited in the bank. The savings passbook and all records will be open for scrutiny by any member. Health and membership fee collection will be ongoing so that the organization will have sufficient resources to fund its own programs. Finally, partnerships will be forged with the Barangay and Municipal LGUs for health-related and financial activities. The FE team recommended inviting more women to become members of the organization, because women were very active in the project. Improve fundraising activities (buying and selling, communal farming system) to earn more income for health and membership dues. The sustainability plans are realistic inasmuch as they do not rely wholly on external assistance. Local municipal health and/or agricultural offices will provide any technical assistance needed by the CBHOs. General Conclusions Community mobilization is an effective strategy for strengthening the capacity of community-based groups to manage their own health care. It is truly empowering since health care is placed in the hands of the people. The workload of the barangay health worker is eased because she/he can tap community-based organization members to assist in some roles such as dissemination of information and case referral. With a pro-active CBHO, the barangay LGU health investment will be focused on preventive health measures. Horizontal visits to learning sites are an effective learning tool for CBHOs. Mentoring and coaching can serve as a capacity-building strategy for local leaders, especially in transferring CO-CD skills. Efforts should be stepped up in the area of documenting best practices and community mobilization for health initiatives. b. Behavioral Change Communications General Approach The BCC plan operated at three target levels: primary audience: family and caregivers; secondary audience: health care providers; and tertiary audience: LGUs and NGOs. Messages developed for the primary audience were: prompt care-seeking behaviors and early detection of signs of illness; promotion of breastfeeding; proper preparation of locally grown nutritional complementary foods; importance of hygiene; use of the latrine; hand washing; importance of immunization; preparation and use of ORS; and, case management of diarrhea and ARI. These messages were delivered through printed IEC materials and media broadcasts (radio spots), health education classes, individual classes/counseling, and community talks. IEC materials included flipcharts, community IMCI cards, community IMCI child growth basic learning packets, and comic books. The BCC plan for the secondary audience (MHOs, PHNs, RHMs, BHWs, TBAs) sought to increase health workers’ knowledge about the assessment, classification, prevention, control, and management of malnutrition and other childhood illnesses. These messages were disseminated through training sessions, orientations, and interpersonal counseling. 31 IEC and work-related materials included the MCSP information kit, an IMCI Manual, a child growth guide, and counseling cards. The messages designed for the tertiary audience (LGUs, NGOs, the media) aimed to raise awareness among local authorities and leaders regarding the health needs in their areas. These motivational messages encouraged them to take action on pressing situations. Desired outcomes included hiring additional health staff, better policy-making, increasing medical supplies, and improving health infrastructure. Strategies also included multi￾sectorial advocacy meetings, flyers, and seminars. Final Evaluation Findings The project developed and implemented a comprehensive social and community-based communications program. The health messages were developed during social communications workshops for the partners. In 2001, CRS contracted Helen Keller International to conduct a formative study on caregivers’ attitudes and behaviors vis-à-vis child survival interventions. The study’s findings formed part of the basis for the project’s BCC strategy. HKI also pre-tested the materials among health personnel, while CRS conducted an additional round of field testing at two levels of the target audiences: mothers/caregivers (for the comics) and health workers. The multi-channel approach drew on a variety of methods and materials and focused on the target audience by considering the needs, culture, and realities of Maguindanao. The project also used multi-media techniques such as community presentations, individual counseling, caregivers’ classes, printed comics, posters, flyers, charts, and radio messages. A total of 20 health messages were created for the primary audience, and they were translated into cartoon formats and local dialects. Fifteen targeted messages were developed for the secondary audience (health workers), and messages for the tertiary audience were disseminated through flyers and seminars in each municipality, among other methods. According to the ET, project monitoring has shown that health care providers have improved the quality of the care they provide. BHWs were able to assess, classify, manage, and refer sick children using the C-IMCI protocol, while a growing number of mothers sought help from the BHWs. Some LGUs have hired RHMs and have provided stipends for the BHWs. Others have equipped the HNPs with supplies and medicines, as well as electricity and septic containers for used syringes. LGUs also provided transportation funds and materials for midwives, especially during national immunization days. Printed materials were not as effective given the low literacy rate in Maguindanao. The project effectively collaborated with other agencies in the area, using an “inter￾sectorial approach.” For example, the program distributed seeds from the Department of Agriculture to plant home gardens. The health staff worked in collaboration with 32 agencies from other sectors as part of their regular activities. Similar partnerships and collaboration were observed with the Department of Education for “Nutrition Month.” Advocacy and LGU involvement brought additional funding and support to the project. Lessons Learned Mass media broadcasts and IEC materials were adapted and translated in keeping with local norms and dialects. However, the project failed to include religious figures, who are highly influential community leaders, in the tertiary audience. Other programs, such as the UNFPA, have worked successfully with religious leaders, for example, by announcing activities through the mosque after prayers. In that case, religious leaders supported family planning interventions. In South Upi, the health team coordinated with the local church for additional funding. Sustainability Plans BCC strategies already are an integral component of the municipal and IPHO annual work plans. The IPHO has stated that it will continue to broadcast radio messages. According to the FE team, the face-to-face education approach is the most sustainable because it has been integrated into the IPHO’s existing activities, in contrast to producing materials, which requires additional financial resources. Conclusions and Recommendations The project used an effective BCC strategy involving a variety of methodologies and media outlets to convey messages to specific target audiences. Messages and materials were adapted and tailored to local traditions and realities. Projects and programs should make more effort to include influential religious leaders in health education initiatives and to foster active community participation. A clearer monitoring and evaluation system should be set up and analysis and decision￾making should occur at the local level. The radio was the most effective channel for announcements, while printed materials proved less effective because of the low literacy levels of the beneficiaries. Since no single IEC method is sufficient in and of itself, a combination of methodologies can ensure that the message will be received. c. Capacity-building Approach To assess the outcomes of the capacity-building approach, the evaluation team discussed the FE guidelines, facilitated Focus Group Discussions (FGD) and conducted Key Informant Interviews (KII) with municipal health authorities, health personnel, and mothers of children under five years of age (see Annexes G and H). After collecting this qualitative data, the team discussed the findings with the external evaluator and CRS staff. 33 The main findings, conclusions, lessons learned and recommendations for continuing capacity￾building activities initiated by the MCSP are outlined below. Strengthening PVOs At the headquarters level, health technical personnel acquired and/or improved skills in social health insurance models and community decentralization processes. Similarly, at the regional level, the technical capacity of personnel was improved in the areas of program planning and evaluation. National CRS staff acquired technical skills in applying the IMCI approach at the health facility and community levels, and acquired new skills in PD-HEARTH and KPC/LQAS. The MCSP shared successful project approaches and tools with other regional health programs at annual Health Technical Commission meetings. CRS headquarters is planning to implement a series of activities to share the lessons learned through this program with other CRS country programs. As a first step, the Philippines program manager will hold a series of presentations in Baltimore in April 2005. The final evaluation report will have been shared with all CRS country health programs by May 2005. The Senior Technical Advisor will participate in a panel at the CORE Group’s annual spring meeting in New York, where the C-IMCI approach used in the Mindanao program will be discussed. The social health insurance model will also be presented during the annual Global Council meeting in Washington, D.C. in May 2005. Strengthening Local Partners At the beginning of the project, CRS conducted an Organizational Capacity Assessment Test (OCAT) of the IPHO and KFI. The purpose of the OCAT was to identify strengths and weaknesses in the following areas: governance, management, human resources development, financial management, sustainability, service delivery, and external resources and partnerships. The findings of the OCAT served to develop a training plan to upgrade the technical and organizational capacity of these two local partners. Through this project, the IPHO and KFI improved their management capacity in program planning and evaluation. KFI also improved its financial systems and logistical capacity, while the IPHO enhanced its technical ability in facility-based IMCI and EPI programming skills. Likewise, both the IPHO and KFI improved their technical skills in C-IMCI. CRS has worked in close collaboration with the IPHO, and its staff was well integrated into planning processes. To date, the CRS project has responded to the vast majority of requests and needs through a Memorandum of Understanding between CRS and the IPHO which generally focused on trainings, provision of materials and/or medications, and use of vehicles. CRS-MCSP reviewed proposals and expenditures to determine whether they were consistent with the requests made and whether the proposed activities were in accordance with the DIP and quarterly and annual plans. Activities that enable the IPHO to function with its own limited resources should contribute to greater sustainability and should take priority over less sustainable activities. Clarifying and reassessing the partnership agreement would be a first step in shifting the focus from a 34 general training approach to a more strategic approach centered on sustainable institutional capacity-building. Strengthening Health Facilities A Health Facility Assessment was conducted at the beginning of the project to document the availability of supplies and equipment needed for key interventions, staff performance, and client satisfaction. Based on the findings of the HFA, the project supplied materials and equipment to the municipal health services and to the barangay health and nutrition posts. Local government units also contributed infrastructure upgrades, medicines, and manpower. Almost all activities have been geared toward improving services at the health facility level, and have included training, especially in IMCI and C-IMCI. IMCI standards are specially designed to monitor improvements in health facilities. The IPHO is applying these standards in almost all municipal and barangay facilities, which is an indication of significant progress towards their incorporation at the institutional level. Further monitoring of training quality could be done and, ideally, a final HFA survey is recommended. Strengthening Health Worker Performance MCSP training efforts have focused on improving health worker performance at the municipal and barangay levels. While it is recommended that the evaluation component be improved, the training programs have been completed as planned and have laid important groundwork for their institutional assimilation. The main initiative for improving health worker performance was the implementation of the IMCI strategy in the health area covered by the project. Pursuant to this strategy, the program would improve health worker skills through IMCI training and strengthen health infrastructure for applying the IMCI approach. To this end, the project trained 100% (baseline 0%) of health facility workers in the project area. By project end, 50% of these health workers were able to follow the IMCI protocol, 100% of health facilities were equipped with adequate IMCI supplies and equipment, and 100% of health facilities had received at least one supervisory visit, including case management, during the preceding six months. An important mechanism for providing stipends for BHWs has been implemented, but more work is needed to maintain LGU funding support once the project ends. During the extension period, the development of appropriate mechanisms should continue to be thoroughly evaluated in light of the current system as well as the IPHO’s long-term needs. Training The MCSP has successfully transferred technologies and techniques to the IPHO and targeted municipal and barangay health teams. The project’s training program has helped develop skills in areas such as community mobilization, specific CS interventions through the IMCI, data analysis and its uses, and management. Despite the amount of training that has been provided to health staff and BHWs, the project has not been 35 sufficiently assertive in the area of monitoring and supervision. Although, it appears that training is winding down due to the project’s termination, supervision should be a priority during the no-cost extension. Municipal level supervisors should continue providing on-the-job feedback to the BHWs. During the evaluation, some health staff pointed out that on some occasions, BHWs were not reporting information from the community. This apparently minor problem should be investigated further to determine the reasons why BHWs do not deliver data, i.e., availability of forms, ability to read and interpret the forms, etc. d. Sustainability Strategy Project strategies were designed to enhance sustainability in the following areas: · Capacity of health workers to provide quality health care · Capacity of health committees and community-based health organizations (CBHOs) to operate effectively · Promotion and adoption of desirable health practices · Community capacity to develop and manage health activities through community-based mechanisms such as the social health insurance model or other financial systems, and · Complementary partnership between the KFI and the IPHO All sustainability target indicators were met. (See Table 12) Table 12: Comparison of sustainability indicators at baseline and final evaluation. Indicators Results Baseline (%) MTE Final KPC (%) Variance (+/-%) Capacity Building Outcome Indicators: · % of communities actively involved in community health care and project management (45 communities targeted for community mobilization) 0 45 or 100% +100 · % of health staff demonstrating skills in community mobilization and organizing (Target: 5 MHOs/PHNs, 20 RHMs and 91 BHWs) 0 60 +60 · % of target barangays with functioning community-based health 0 15 or 33% +33 36 organizations (Target: 45 barangays) Sustainability Indicators: · % of target barangays with LGU cost-sharing 0 10 or 22% 22% According to key informant interviews (see attachment C), increasing membership by inviting women to participate would enhance sustainability efforts. Those interviewed also agreed on the need to strengthen links between the barangays and municipal LGUs. In terms of financial sustainability, the government launched several initiatives to develop health insurance systems; one such program is PhilHealth, which has begun to operate in Maguindanao and has nation-wide coverage. The project has assisted municipal and provincial authorities to expand and consolidate PhilHealth. However, “key informant” recommendations pointed out that only transparency in financial transactions would earn members’ trust, thereby boosting participation. The project also encouraged LGUs and CBOs to register with the Department of Labor (DOLE), so as to gain access to government resources and international donor agencies. The final evaluation team recommended that local organizations establish institutional working relationships with, and formal representation on, the Local Development Councils (municipal and provincial), to document the nature of their working relationships and to establish mutually satisfactory terms and conditions for pursuing previously established health objectives. Another recommendation was to organize and strengthen the BHW federation and its operational role in health services provision, and to conduct ongoing monitoring of progress in health care delivery. C. PROJECT MANAGEMENT 1. Planning The heads of CRS, IPHO, KFI collaborated effectively in project design, conducting the baseline study, and developing the DIP. However, the planning process lacked sufficient participation by community-based stakeholders, particularly at the municipal level. As a result, roles and responsibilities were not clear at the barangay level at the outset of the project. The project was unable to implement all of the proposed community mobilization steps in all of the target barangays. In some barangays, the ratio of 1 community mobilizer per 10 barangays was insufficient to complete the full cycle of community organization and development processes. In early 2003, the MTE recommended targeting a more realistic number of communities able to complete the six-stage development process set forth in the DIP. As a result, only 50% of the 45 targeted communities completed the six-stage process. The DIP was a practical tool in that it was integrated into the DOH’s implementation and evaluation plan and involved other potential partners. Indicators were realistic and took into account national priorities. Finally, the DIP was used to monitor community mobilization efforts. It was also used by the Health Working Group to assess project advances and planning, lending a sense of direction and accomplishment to the process. 37 2. Staff Training Monitoring activities have shown that health staff upgraded their knowledge and skills in implementing CS interventions. The teams developed monthly, quarterly, and annual action plans. In-service training and follow-up after formal training courses was found to be crucial for implementing the IMCI strategy. 3. Supervision of Program Staff The project’s supervision system was adequate in terms of motivating staff and assessing project advances. Feedback sessions were incorporated following supervision. Provincial trainers conducted in-service training and follow-up after IMCI training (at the facility and community levels) as a shared responsibility. IMCI supervision tools included exit interviews, health facility checklists, health workers performance assessments through direct observation, and HIS assessments, endorsed and formally incorporated by the IPHO. The MCSP helped expand the supervision system at the community level through midwives and nurses associated with the project. The DOH had designed the system, and the MCSP contributed to its implementation. The gaps identified by the FE team included insufficient feedback to barangay/community health workers and the community. This represented an important missed opportunity to involve stakeholders in problem identification and solution. 4. Human Resources and Staff Management The three MCSP implementing partners have personnel policies in place. There was a good working relationship between project and IPHO staff. Although the final evaluation team did not have much information to identify any issues concerning the partnership, regular feedback meetings and communication flow during the project was reported. Significantly, there was no project staff turnover, with the exception of the first project manager, who served only 6 months and was replaced without delay. 5. Financial Management Both partners, the IPHO and KFI, have established financial systems. The IPHO, as a government office, uses its own accounting system but complied with CRS’ financial reporting requirements. KFI, on the other hand, has fully adopted the CRS financial system as it is a CRS partner for various programs. KFI has installed and uses accounting software as part of its financial management system. Generally, CRS’ financial management practices were adequate and properly introduced to the partners through regular financial orientation sessions and monitoring. 38 Budget re-alignments were performed at the end of every CRS fiscal year. Unspent funds from previous fiscal years were examined and justified. Positive variances were due to dollar exchange rate fluctuation and activities that were not carried out on time. CRS provided the budget for each partner’s annual work plan implementation. The MCSP engaged in advocacy efforts to encourage the Municipal LGUs to integrate children’s health and development plan into their Executive Legislative Agenda (ELA). Moreover, the IPHO integrated the C-IMCI protocol and training packet into its general health delivery system and consistently lobbied for LGU financial support for basic medicines and supplies. 6. Logistics The availability of vehicles, motorcycles, pump boats, and horses enhanced the mobility of field staff, although physical access in the province will remain a challenge given the vast geographical extension of the project area. Logistical support systems were adequate throughout the life of the project. Procurement of needed supplies and equipment boosted morale and the desire to accomplish more. Nonetheless, there were gaps in inventory updating to prevent shortages and delays were reported in deliveries from central distribution points to the province and the barangays. Project staff and municipal and provincial government officials are still discussing how logistical systems and supplies will be maintained. 7. Information Management The project’s HIS design was based on the IPHO health information system. Data flowed from the community, where BHWs collected health information, to the rural health units (municipality), and from there were forwarded to the provincial health office. The provincial level health-working group examined CS indicators as an agenda item at its quarterly meetings. HIS analysis at the municipal level was only included during the last year of the project. During the course of the MCSP, the team conducted special assessments, mini-surveys, and focus group discussions, mainly when implementing new approaches such as the Hearth Nutrition Model. Overall, the project improved data collection systems at the municipal and provincial levels. 8. Technical and Administrative Support CRS headquarters and country offices provided regular and substantial technical assistance throughout the project. A summary of technical assistance activities is provided below: Dr. Anwar Aquil from CRS-HQ provided technical assistance in conducting the baseline KPC. 39 Dr. Alfonso Rosales, Senior Technical Advisor, CRS-HQ, provided ongoing technical assistance during the initial development of the DIP and visited the country regularly thereafter, once or twice a year. Dr. Rosales supported the adaptation and introduction of the C-IMCI strategy, and provided assistance during the mid-term and final evaluations. Dr. Rosales also has been instrumental in adapting the C-IMCI to local needs through a series of workshops with partners. Lorie Dostal, CRS-SEAPRO health technical advisor, visited the country on two occasions and provided technical advice on community mobilization and the mid-term and final evaluations. The CRS-Manila Office, through the Health Program Manager, supervised overall project implementation through regular visits, meetings, and facilitation of important partnership activities. Finally, the MCSP staff participated in CRS-sponsored trainings and workshops on subjects such as strategic planning, project framework, and peace building. 9. Management Lessons Learned The team concluded that strengthening the capacity of project partners to attain its objectives was the project’s most important contribution. A well-designed Detailed Implementation Plan (DIP) was crucial for effective coordination and support from all three-project partners. The participatory approach enhanced the knowledge and skills of provincial and municipal stakeholders and reporting of accomplishments contributed to a sense of project ownership. Adequate and timely provision of logistical support would enable health workers to properly perform their functions. D. OTHER ISSUES IDENTIFIED BY THE TEAM The FE team did not identify other issues. E. CONCLUSIONS AND RECOMMENDATIONS The MCSP project is the result of several years of work in close collaboration with the IPHO and local institutions. The main findings of the final evaluation showed quantitative improvements in all but one project indicator. Key factors in achieving these outcomes were: 1) the decentralization approach to community health care delivery, which was based on CORE’s C￾IMCI and used CRS’ C-IMCI tools to operationalize health care delivery at the community level; 2) a locally adapted BCC program focused largely on danger sign identification; and 3) a strong and systematic approach to community mobilization. The health sector reform underway in the Philippines includes a decentralization process to strengthen municipal and local governments and community participation. Traditionally, 40 decentralization approaches in the health sector shift health planning and service delivery financing from the central government to the local level. In these approaches, the most peripheral point of health services delivery is the rural health facility. The MCSP decentralization approach facilitated the traditional decentralization model, supporting the shift of health services delivery to the most peripheral point in the formal health system. At the same time, however, the MCSP went one step beyond the rural health facility into the community setting. Without fracturing the health system and while maintaining proper quality (as evidenced by the final indicator outcomes), health delivery shifted from physician-based (baseline 76%; final 22%) to midwives (baseline 47%; final 88%) and Barangay health workers (baseline 9%; final 65%). This operational shift to community delivery of health services, besides improving access to essential health care from 75% to 89%, also decreased the actual cost of delivering those health services. Given the geographical isolation, poor health infrastructure, and rampant poverty afflicting most of the villages in the Maguindanao region, central and regional health policy-makers should consider expanding this approach to the rest of the province. The effectiveness of a culturally adapted, multi-channel BCC approach focused on health messages is demonstrated by the MCSP behavioral change final outcomes. All nutrition-related behavioral indicators exceeded the original targets, and malnutrition rates declined by over 30%. ARI care-seeking behaviors also improved, resulting in over 80% coverage of pneumonia cases. Likewise, the population’s adoption of new behaviors in community management of diarrheal disease, coupled with prevention-oriented behaviors, was overwhelmingly positive and led to a 12-percentage point reduction in diarrhea prevalence. Interestingly, the effectiveness of this approach is further supported by the fact that demand for immunization services did not increase following a BCC approach that focused solely on the dissemination of health information to increase awareness. Project implementation in these areas would not have been as effective without the proper support from community structures. This is especially true in areas characterized by poverty, weak health infrastructure, and difficult geographical access, where the utilization of local resources based on a highly organized community is a must. A systematic approach to strengthening community structures facilitates this process because it establishes shared organizational objectives, in addition to a common framework and tools. The MCSP community￾centered approach of health mobilization, capacity building, and networking/linkage-building went beyond the narrow vision of providing medical solutions for medical problems. As a result, community solutions to health problems included socio-economic and cultural factors, thereby generating a comprehensive vision for human development. The health targets not achieved by this project, such as full immunization of children under-two and iron supplementation coverage among the population of pregnant and lactating women, shared a common variable in that both interventions were health facility-based. Additionally, the BCC messages were largely focused on awareness raising, premised on the assumption that if availability were improved, utilization would follow. In the particular case of immunization, based on a comparison of FIC with measles coverage, it seems that the “missed opportunity” factor played a role as well. It is important to recall, then, that utilization of health services is contingent not only on availability of services, but also on factors such as client attitudes, perceptions, and past experiences with those services. 41 42 F. ONE-PAGE SUMMARY OF OUTCOMES The Maguindanao Child Survival Project (MCSP) adopted a decentralization approach to community health delivery aimed at shifting delivery of essential health services from the health facility (physicians and nurses) to community health structures (barangay health workers and midwives), and linking both points of the system through a strong referral mechanism. The cornerstone of this approach was the adaptation and implementation of the Community Integrated Management of Childhood Illness (C-IMCI) strategy to increase resolution capacity at the community level, supported by the implementation of a health facility-IMCI to improve quality of services at that level of the health system. The MCSP strategy was based on the CORE’s C-IMCI model and included, besides disease identification and management tools, a locally tailored BCC program, a common approach to community mobilization, and the facilitation of private-public partnerships between IPHO, KFI, CRS, and local municipal governments. The main findings of the evaluation showed quantitative improvements in all but one project indicator. The nutrition and breastfeeding intervention met and exceeded all eight programmatic targets as demonstrated by comparing baseline, midterm, and final data. The overall malnutrition rate in children under two years of age decreased by 34% (target 10%). Breastfeeding initiation within one hour of birth increased 42 percentage points; the rate of exclusive breastfeeding of children under 6 months of age rose by 28 percentage points; complementary feeding rates for children 6 - 9 months grew by 13 percentage points; and growth monitoring coverage increased by 41 percentage points. The findings for the second component of the program, pneumonia case management, showed that all key indicators exceeded the targets set at the start of project implementation. The project improved case identification skills at the household and community levels, access to prompt treatment, and quality of care at health facilities as well as in the community. The ability of mothers to recognize danger signs in children under 2 years of age increased by 31 percentage points from baseline to final data. Pneumonia treatment was received by 83% of children under two by the end of program activities, 13 percentage points over the established target. The CDD intervention led to a 12-percentage point decline in the overall prevalence of diarrheal disease in children under two years of age. All four-target indicators to measure the impact of the CDD intervention were met and exceeded by project end. For example, children under two with diarrhea in the preceding two weeks who received the same amount or more fluids and food increased by 44 percentage points; caregivers of children under two with diarrhea during the preceding two weeks increased their care-seeking behavior by 39 percentage points; the availability of functioning ORT facilities increased by 92 percentage points; and, the availability of trained community health workers increased 100 percentage points. Finally, availability of immunization services was increased during the life of the project: EPI access was 99% among children under two in the project area, possession of a vaccination card increased 51 percentage points, vitamin A coverage increased 33 percentage points, and quality of services also improved, as measured by percentage of health staff trained and a proxy indicator for proper cold chain management. Nonetheless, utilization of vaccination services did not follow service availability. The overall impact indicator for the EPI intervention “percentage of fully immunized children” showed no significant quantitative change when comparing baseline (58%) and final (57.3%) data. Conversely, measles coverage in children under-two increased 51 percentage points, from 27% at baseline to 77.9% at final. Maternal coverage of tetanus toxoid also increased, from 47% to 84.4%.