CARE PERU ENLACE PROJECT 1996-2000 OTUZCO-JULCAN PROVINCES LA LIBERTAD, PERU FINAL PROJECT EVALUATION REPORT October 2000 Prepared by: CARE USA contact: Laura C. Altobelli, DrPH, MPH Judiann McNulty, DrPH Public Health Specialist Deputy Director Child Health This project was funded under the PVO Child Survival Grants Program CS-XII, 1996- 2000, by the United States Agency for International Development, USAID/BHR/PVC. i EXECUTIVE SUMMARY The Enlace Project was designed and implemented by CARE from 1996 to 2000 in the poor mountainous provinces of Otuzco and Julcán in the La Libertad Region of northern Peru. The project was funded by the United States Agency for International Development under the PVO Child Survival Grants Program, CS-XII 1996-2000. Project objectives were focused on three interventions areas. The first area was prevention and control of diarrheal diseases in children through increased breastfeeding and use of liquids, better recognition of dehydration, decreased use of antibiotics and anti-diarrheal medicines for diarrhea, and increased hygiene behaviors. The second set of objectives was on control of acute respiratory infections and pneumonia through improving mothers’ recognition of danger signs and increasing utilization of health services for treatment. The third intervention area was improving maternal health through recognition of danger signs and symptoms in pregnancy, childbirth, and postpartum, increased use of services for preventive care for all pregnant women and for all women with complications in pregnancy, childbirth, and postpartum. Main accomplishments of the Enlace Project were the design and development of several innovative organizational strategies that are a major contribution to international work in community health development. One was the design of organizational structures – APROMSAs and COPROMSAs – for health promoters that allowed them to work within a legalized institutional framework that represents promoters in their relationships with the health system and other community leadership organizations such as municipalities. These structures provided a firm basis for sustainability of health promoter work that is rarely found in community health programs. Another innovation was the development of a complete community surveillance system – SIVICS – that provided an effective methodology for health promoters to identify and monitor community members at high risk of needing health services (i.e. children under age one, pregnant women, and women of reproductive age). SIVICS has gained the attention of the Peruvian Ministry of Health, which is interested in expanding the methodology to the rest of the country. Creative communication techniques, designed based on community preference studies, were an important contribution to changing health behaviors and attitudes toward use of health services. The Enlace project also developed a gender approach to community health through training of “Women Leaders” in communities instead of the more common male health promoters, increasing the cultural acceptability and therefore effectiveness of health education to women on women’s health issues. A unique model was developed for obstetrical emergency evacuation from remote communities, based on organized readiness plans with participation of local authorities, promoters, and community members. Highlights from the comparison of baseline and final evaluation KPC surveys include some of the following improvements in the target population between 1996 and 2000 • 46% to 64% of infants under six months were exclusively breastfed. • 30% to 15% of diarrhea cases were treated with antibiotics or other medicines. • 19% to 56% of mothers recognized two signs of pneumonia in children. CARE PERU ENLACE PROJECT 1996-2000 FINAL PROJECT EVALUATION REPORT ii • 33% to 60% of children with cough, rapid or difficult breathing received medical care. • 3% to 51% of mothers identified two signs or symptoms of obstetrical risk. • 45% to 93% of women with complications in pregnancy, delivery, or postpartum received medical care. Priority conclusions of the project were the following: The project provided important confirmation that the organizational and managerial aspects of a community health program are at least as, if not more, important as the technical approach taken. One of the key features of the Enlace Project was the high degree of co-implementation with the Ministry of Health (MOH). The participatory working style of Enlace strengthened both institutional capability and acceptability of the project products on the part of the MOH. The technical approach in the three intervention areas was effective because it was oriented to best-recommended practices, with strategies based on pre-project analysis of mothers’ beliefs and practices. The focus was on preventive measures, identification of danger signs and symptoms, and referral or emergency evacuation to a health facility. The project identified the importance of using a gender approach to maternal health interventions in communities, using of “Women Leaders” trained to promote maternal health in the community, overcoming resistance to male providers and male health promoters. The project highlighted the advantages of a flexible project design and open management style to be able to respond to community-generated solutions to their needs for structuring health services and designing information delivery systems. Finally, suggestions based on project findings are provided at the end of the report as to how the Ministry of Health should improve its policies to support the development of communitybased health systems in order to more effectively reach health sector goals of reducing needless morbidity and mortality in the most vulnerable populations. CARE PERU ENLACE PROJECT 1996-2000 FINAL PROJECT EVALUATION REPORT iii TABLE OF CONTENTS Executive Summary i Table of Contents iii Abbreviations iv I. Introduction and Background 1 II. Assessment of Results and Impact of the Project 2 A. Results: Summary Chart 2 B. Results: Technical Approach 4 C. Results: Cross-cutting approaches 13 1. Community Mobilization 13 2. Community Health Surveillance 15 3. Communication for Behavior Change 16 4. Capacity Building Approach 18 a) Strengthening the PVO Organization 18 b) Strengthening Local Partner Organizations 19 c) Health Facilities Strengthening 21 d) Strengthening Health Worker Performance 22 e) Training 23 5. Sustainability Strategy 24 III. Program Management 27 A. Planning 27 B. Staff Training 28 C. Supervision of Program Staff 29 D. Human Resources and Staff Management 29 E. Financial Management 30 F. Logistics 30 G. Information Management 30 H. Technical and Administrative Support 32 I. Management Lessons Learned 33 IV. Conclusions and Recommendations 33 ATTACHMENTS A. Evaluation team members B. Assessment methodology C. List of persons interviewed and contacted D. Copy of formal agreement between Agallpampa CLAS-APROMSA-Municipality of Agallpampa E. Promoters trained by topic and micro-network F. Description of APROMSA Development G. Birth Planning Form H. KPC Final Survey Report CARE PERU ENLACE PROJECT 1996-2000 FINAL PROJECT EVALUATION REPORT iv ABBREVIATIONS ADD Acute diarrheal disease APROMSA Asociación de Promotores de Salud (Health Promoter Association) ARI Acute respiratory infection CARE Care And Relief Everywhere COPROMSA Comité de Promotores de Salud (Health Promoter Committee) CLAS Comité Local de Administración de Salud (Local Health Administration Committee) DIP Detailed implementation plan DISA Dirección de Salud Regional (Regional Health Department Office) EOC Emergency obstetrical care HIS Health information system IEC Information, education and communication MOH Ministry of Health of Peru NGO Non-governmental health organization PRONAA Programa Nacional de Asistencia Alimentaria (National Food Assistance Program) PVO Private voluntary organization SIVICS Sistema de Vigilancia Comunal de Salud (Community Health Monitoring System) TBA Traditional birth attendant TOT Training of trainers UPAO Universidad Privada Antenor Orrega UTES Unidad Territorial de Salud (Territorial Health Unit) CARE PERU ENLACE PROJECT 1996-2000 FINAL PROJECT EVALUATION REPORT 1 I. INTRODUCTION AND BACKGROUND This is the final evaluation report of the Enlace Project, carried out by CARE-Trujillo, Peru, under a grant from the PVO Child Survival Grants Program, funded by USAID/BHR/PVC, in the funding cycle CS-XII 1996-2000. Enlace Project beneficiaries were 47,000 women of reproductive age and children 0-5 years of age among an estimated 128,000 inhabitants living in 597 scattered communities in two poor rural provinces of Otuzco and Julcán in the highlands of the Department of La Libertad on the northern Pacific coast of Peru in South America. This inland zone of steep hills and valleys has difficult access from the coastal departmental capital, Trujillo. The area has no asphalt roads, and only the main roads between the five main towns are “firmed” dirt roads. Poverty in the area was exacerbated during most of the project implementation period due to severe flooding (related to the El Niño phenomenon of 1998) and alternating drought. This was alleviated in the final project year with normal rains and a bountiful harvest that, combined with reforestation projects in the area, created a stunningly beautiful landscape. The provinces of Otuzco and Julcán comprise the territory covered by the Territorial Health Unit (UTES) No. 8 of the Regional Health Office of La Libertad. Access to health services increased dramatically during the past five years through a number of mechanisms put into place by the Ministry of Health. The five health centers and 10 health posts mentioned in the 1997 Detailed Implementation Plan (DIP) increased by an additional 11 health posts for a present total of 26 peripheral health facilities in UTES-8, in addition to the small hospital in Otuzco. The “Strengthening Health Services Project” through an IDB loan provided equipment and some management training. Staffing was facilitated by the “Basic Health for All Program” (PSBPT), funded by the public treasury, which provides financial incentives for health professionals to work in isolated facilities under 3- to 6-month contracts. PSBPT provided for lengthening working hours of health facilities from 6 hours to 12 hours, and in some facilities to 24-hour coverage. The majority of facilities are now open seven days a week. Contract renewal under PSBPT depends on performance evaluations and monitoring of service production. These rules resulted in improvements in work habits and increased adherence to MOH norms for patient care. Still, high turnover of personnel was a problem. At the beginning of the Enlace project, 80 of 88 MOH health staff in Otuzco-Julcán were contracted by PSBPT. Another project working in the Otuzco-Julcán area was “Project 2000”, a maternal-child health project financed by USAID Peru during 1995-2000 with goals to increase quality and effectiveness of health services. The project was to provide a series of inputs such as establishing a permanent capability in model hospitals and model health centers for training of health personnel, systems for continual quality improvement, IEC, training of community health workers, and others. Recently extended for a two-year period, Project 2000 has had only partial implementation in Otuzco-Julcán, with initial activities just beginning in clinical training of health personnel and some aspects of quality improvement. Community activities under Project 2000 were not implemented in the Otuzco-Julcán area. CARE Peru is the major sub-contractor to Pathfinder under this project. CARE PERU ENLACE PROJECT 1996-2000 FINAL PROJECT EVALUATION REPORT 2 An important intervention in the Otuzco-Julcán area occurred in 1999, when the Regional Health Office of La Libertad converted all health facilities in UTES - 8 to administration under the “Shared Administration Program”1 , with five Local Health Administration Committees (CLAS) conforming five micro-networks of the 26 health facilities. According to the Regional Health Office of La Libertad, the conversion to CLAS in Otuzco-Julcán was done in recognition of the “high level of community participation in health achieved through Enlace.” One of several advantages of the CLAS model is that personnel are contracted by CLAS under private sector law with additional benefits, so that personnel turnover is reduced and quality is improved due to the social control over the health facility by the community. Any evaluation of Enlace must take into consideration the above-mentioned health inputs that may have influenced project impact. In other words, it may be difficult to attribute improvements in project impact indicators only to Enlace. However, the in-depth key informant interviews in communities, health facilities, the UTES, and DISA leave little doubt of the major and probably lasting effect of Enlace on changing attitudes, paradigms, and organizational structures in the communities of Otuzco-Julcán, and the potential of its strategies for expansion to other areas. The expansion of Enlace strategies is already being seen in their incorporation into other projects and programs through the MOH in other parts of Peru. II. ASSESSMENT OF RESULTS AND IMPACT OF THE PROGRAM A. Results: Summary Chart Baseline and achieved values for all Enlace Project indicators are listed in Table 1in the three main intervention areas: (1) control of diarrheal diseases; (2) control of pneumonia cases; and (3) maternal health. The table also lists the expected values of the indicators, as listed in the 1997 Detailed Implementation Plan (DIP). The confidence interval on the percentage listed as achieved is provided in order to assess how well the achieved value corresponds to the expected value. If the expected value is within the confidence interval of the achieved value, then the objective was successfully reached. Several indicators significantly surpassed the expected value. 1 The Shared Administration Program is a new form of health services administration under the Peruvian health reform in which an elected committee of community members forms a legally constituted private non-profit organization (Comité Local de Administración de Salud - CLAS). The committee receives and manages public funds under a contract with the DISA for implementation of a Local Health Plan. With the health facility medical director as CLAS manager, the CLAS is responsible for monitoring implementation of the Local Health Plan, with decision-making power for hiring and firing health personnel, for allocating of funds gained through charging fees-for-service, among other responsibilities. As a private entity, CLAS can generate additional funding for health-related activities from other public and private sector sources. CARE PERU ENLACE PROJECT 1996-2000 FINAL PROJECT EVALUATION REPORT 3 Table 1 ENLACE PROJECT 1996-2000 OTUZCO Y JULCAN – PERU BASELINE AND FINAL EVALUATION INDICATORS CONTROL OF DIARRHEAL DISEASE CONTROL OF ACUTE RESPIRATORY INFECTION / PNEUMONIA CASES Increase from 33% to 60% the proportion of mothers with 60** children 0-23 months of age who seek appropriate medical care for their child with cough and rapid or difficult breathing. 32 60 51