USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 INCREASED ACCESS TO AND DELIVERY OF QUALITY MATERNAL AND CHILD HEALTH CARE SERVICES IN SELECTED AREAS IN THE DOMINICAN REPUBLIC: MATERNAL AND CHILD HEALTH CENTERS OF EXCELLENCE, AND MATERNAL AND CHILD HEALTH INTEGRATED PROGRAM MAY 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by consultants Marcello Castrillo, Barbara Eguren, Marija Miric, Rafael Montero, and Eddy Perez-Then; Ministry of Health representatives Olga Arroyo and Donatilo Santos; and USAID representatives Veronica Valdivieso and Peg Marshall, through the GH Tech Bridge Project. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 INCREASED ACCESS TO AND DELIVERY OF QUALITY MATERNAL AND CHILD HEALTH CARE SERVICES IN SELECTED AREAS IN THE DOMINICAN REPUBLIC: MATERNAL AND CHILD HEALTH CENTERS OF EXCELLENCE, AND MATERNAL AND CHILD HEALTH INTEGRATED PROGRAM DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 12-01-001) is available in printed or online versions. Online documents can be located in the GH Tech website at www.ghtechproject.com. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com This document was submitted by Development and Training Services, Inc., with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. AID-OAA-C-12-00004. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 i CONTENTS ACRONYMS ........................................................................................................................................ v EXECUTIVE SUMMARY ................................................................................................................. vii Evaluation Methodology ........................................................................................................... vii Evaluation Results ...................................................................................................................... vii Challenges ..................................................................................................................................... xi Future Directions and Recommendations............................................................................ xii I. INTRODUCTION ....................................................................................................................... 1 Evaluation Objectives and Uses ................................................................................................ 1 II. BACKGROUND .......................................................................................................................... 3 The Problem and the Paradox .................................................................................................. 3 Response to the Paradox ........................................................................................................... 3 III. METHODOLOGY ....................................................................................................................... 7 The Evaluation Team ................................................................................................................... 7 Key Evaluation Questions .......................................................................................................... 7 Data Collection and Analysis ..................................................................................................... 7 Assessment Limitations .............................................................................................................. 8 IV. FINDINGS ..................................................................................................................................... 9 Programmatic and Health System ..........................................................................................10 Technical Interventions .............................................................................................................19 Evaluation Question 5 ...............................................................................................................31 V. CONCLUSIONS AND RECOMMENDATIONS ...............................................................33 VI. FUTURE DIRECTIONS ............................................................................................................43 National Level: Ministry of Health, Implementing Partners, and Stakeholders ............59 Hospital San Lorenzo De Los Minas......................................................................................59 Hospital Morilo King. La Vega.................................................................................................60 Hospital Dr Antonio Musa, San Pedro De Macoris...........................................................60 Dominican Republic February–March 2012. ........................................................................71 República Dominicana. Abril–Marzo, 2012..........................................................................91 Formulario de Consentimiento Informado....................................................................... 105 Birth and Immediate Postpartum-newborn Periods ....................................................... 125 Observation Results and Recomendations ....................................................................... 167 ii USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 APPENDICES ANNEX 1. SCOPE OF WORK .....................................................................................................47 ANNEX 2: PERSONS CONTACTED (NOT EXHAUSTIVE)................................................59 ANNEX 3. REFERENCES ...............................................................................................................63 ANNEX 4. MATRIX TO ASSESS THE MATERNAL-INFANT CENTERS OF EXCELLENCE IN THE DOMINICAN REPUBLIC. USAID 2009–2014* . .......69 ANNEX 5. MID TERM EVALUATION FRAMEWORK OF THE RESULTS OF THE MATERNAL-INFANT CENTERS OF EXCELLENCE PROJECT ............71 ANNEX 6. ENTREVISTA A INFORMANTES CLAVES: DIRECCIÓN REGIONAL DE SALUD ....................................................................................................................73 ANNEX 7. ENTREVISTA A INFORMANTES CLAVES: DIRECCIÓN PROVINCIAL DE SALUD ....................................................................................................................81 ANNEX 8. ENTREVISTA A INFORMANTES CLAVES: HOSPITALES REGIONALES Y PROVINCIALES ...........................................................................91 ANNEX 9. FORMULARIO DE CONSENTIMIENTO INFORMADO. ENTREVISTAS A USUARIAS DE SERVICIOS MI Y PERSONAL DE SALUD ......................... 105 ANNEX 10. TEMARIO/GUÍA DE GRUPOS FOCALES: PROVEEDORES DE SERVICIOS DE SALUD........................................................................................... 107 ANNEX 11. TEMARIO/GUÍA DE GRUPOS FOCALES USUARIAS DE SERVICIOS DE SALUD........................................................................................... 111 ANNEX 12. GUÍA DE OBSERVACIÓN CUALITATIVA .................................................... 115 ANNEX 13. GUIDELINES FOR OBSERVER TEAMS 2011 ................................................. 117 ANNEX 14. RESULTADOS DEL EQUIPO EVALUACIÓN EQUIPOS DE GESTIÓN EN HOSPITALES.................................................................................. 133 ANNEX 15: RESULTADOS DEL EQUIPO EVALUACIÓN CUALITATIVA .................. 143 ANNEX 16. MID-TERM EVALUATION OF USAID’S INTERMEDIATE RESULT 4 ..... 167 ANNEX 17. CLINICAL CARE OBSERVATIONS .................................................................. 173 ANNEX 18. RESULTS OF KEY INFORMANT INTERVIEWS AT SRS, SDP AND HOSPITAL LEVELS ....................................................................................... 179 ANNEX 19. PRINCIPLES OF FAMILY-CENTERED MATERNITY CARE ....................... 187 ANNEX 20. SUSTAINABILITY ELEMENTS OF FAMILY-CENTERED MATERNITY ... 189 ANNEX 21. DETAILED METHODOLOGY ........................................................................... 191 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 iii FIGURES Figure IV.1. Intervention Matrix ...................................................................................................... 9 Figure IV.2. Intervention Matrix Applied .....................................................................................11 Figure IV.3. Intervention Matrix Additional Application ..........................................................21 TABLES Table 1. Program Data ....................................................................................................................... 4 Table 2. Qualitative Score Scales and Definitions ....................................................................... 8 iv USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 v ACRONYMS AMTSL Active management of the third stage of labor CAF Common Assessment Framework COEM Cuidados Obstericos Emergencia: Emergency obstetric care CoEx MCH Centers of Excellence DG Discussion group DHS Demographic and Health Survey DIGEPI Dirección General de Epidemiologia - General Directorate of Epidemiology DPS Dirección Provincial de Salud - Provincial Health Directorate DR Dominican Republic FCMC Family-centered Maternity Care FP Family PlanningGH Tech Global Health Technical Assistance Project GODR Government of the Dominican Republic HAIs Healthcare-associated infections HBB Helping Babies Breathe HIV Human Immunodeficiency Virus IEC Information, education, and communication IR Intermediate result KMC Kangaroo Mother Care L&D Labor and delivery LAC Latin America and Caribbean M&E Monitoring and evaluation MCH Maternal and child health MCHIP Maternal and Child Health Integrated Program MOH Ministry of Health MTE Mid-term evaluation PMTCT Prevention of mother to child transmission PNPTV National Program for Prevention of Vertical Transmission of HIV RH Reproductive health SIGHO Sistema Integral de Gestión Hospitalaria: Hospital Management Integrated Information System STI Sexually transmitted infections USAID United States Agency for International Development vi USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 vii EXECUTIVE SUMMARY Although the Dominican Republic (DR) has a high percentage of prenatal care coverage (98.9%), deliveries attended by qualified health personnel (97.8%), and institutional deliveries (97.5%), the maternal mortality rate is still high at 159 deaths per 100,000 live births (DHS 2007). Moreover, between 61.7% and 71.8% of all the maternal deaths reported are considered preventable, raising questions about the quality of health care. In response to this paradox, USAID/Dominican Republic inaugurated the MCH Centers of Excellence Project and the Maternal and Child Health Integration Program (MCHIP). The goal of the Centers of Excellence Project (CoEx) is to help reduce maternal, newborn, and infant morbidity and mortality. MCHIP complemented CoEx through technical support for high￾impact evidence-based newborn health interventions, such as prevention and treatment of sepsis, family-centered maternity and kangaroo mother strategies, strengthening the national newborn health work plan, and use of the Helping Babies Breathe Curriculum to resuscitate newborns. USAID/DR commissioned an external mid-term evaluation (MTE), through the GH Tech Bridge Project, of how responsive these projects are to the Mission’s Intermediate Results 4 (IR4): “Increased access to and delivery of quality maternal and child health [MCH] care services in selected areas.” The eight-member multidisciplinary team, composed of representatives from the Ministry of Health, the Mission, USAID/Washington, and four independent consultants, worked in-country from February 18, 2012, to March 16, 2012. The five evaluation questions it sought to answer were: 1. What programmatic and health systems strengthening aspects have been the most/least successful in ensuring achievement of the project results, and why? 2. What technical interventions have been the most/least successful in ensuring the achievement of the project results, and why? 3. What programmatic and health systems strengthening aspects appear to be the most/least sustainable, and why? 4. What technical interventions appear to be the most/least sustainable, and why? 5. Has the project targeted the most critical and appropriate activities? And what other opportunities exist? EVALUATION METHODOLOGY The team was divided into four groups to (1) interview staff from provincial health directorates and regional health services; (2) interview staff from hospital management teams; (3) hold group discussions with health providers and users of maternal health services; and (4) observe maternal and neonatal care services delivered and interview health care providers. The team as a whole triangulated all results. EVALUATION RESULTS The CoEx has been highly successful, with real impact on the organization and quality of MCH health services in the 10 project hospitals. In all 10 it has met its targets of bringing strategic viii USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 plans current; use of self-assessment tools; and training managerial teams. The project has been less successful in reducing the percentage of cesarean sections performed (still high at 44%) and increasing the number of pregnant women who receive HIV test results and post-test counseling. Hospital management ascribed these problems to the structure of the health system, inefficient supervision, and MOH policies. Programmatic and Health Systems—Evaluation Questions 1 and 3 1. Basic Infrastructure (high performance, fully sustainable): Both health users and providers acknowledge that these project adjustments have vastly improved the quality of care. Hygiene in hospital areas has improved because the support personnel who were trained reported more job satisfaction and more motivation to keep hospital areas clean and functioning. 2. Customer Service System (high performance, fully sustainable): The most visible intervention and the one that gave management teams the most satisfaction was introduction of customer service offices. The CoEx project provided the training, equipment, and necessary infrastructure for the system, including development of the SIGHO software (Sistema Integral de Gestión Hospitalaria). 3. Common Assessment Framework and Strategic Planning (high performance, fully sustainable): Drafting annual operational plans is a task on which all need to cooperate, but hospital management teams found that the project added quality and “motivation.” After training, all facilities drafted strategic plans for 2010–2015 and operational plans for 2010 and 2011; none yet had operational plans for 2012. There did not appear to be a process to assess operational plan quality and monitor its implementation, a weakness confirmed in the M&E component. There also seemed to be an overlap of responsibilities that could undermine the efficiency of project activities. 4. Medicines and Supplies System (high performance, partially sustainable): Pharmacists in hospitals visited reported that they now have a voice in the management team meetings, and that shortages of supplies and medicines are much less frequent. However, PROMESE-CAL, the public sector purchaser and distributor of essential medicines, is failing to consistently provide the necessary supplies. Still, this component is ready for replication and will help reinforce continued improvements in all hospitals. 5. Hospital Management Information System (high performance, partially sustainable): The SOGJP software has improved information systems. Providers were pleased with the possibility of accessing women’s prenatal records in the emergency and labor and delivery (L&D) areas, especially for better management of obstetrical emergencies. However, not all departments had on-site access to all records; for example, in one hospital, neonatology did not have access to the mother’s prenatal records, and the MTE team identified areas of concern in the use of the Perinatal Information System (SIP), a subsystem of SIGHO. Finally, SIGHO clinical data are not routinely employed as a basis for decision-making. 6. Emergency and Disaster Preparedness (low performance, not sustainable): Four of the six sites visited had emergency plans; two of these have done drills, two have made contingency plans, and all four have designated evacuation route and meeting points. Some hospital management teams reported having difficulties coordinating with the MOH Department of Disasters and Emergencies, which insisted that preparedness for emergencies and disasters is their responsibility alone. 7. Monitoring and Evaluation (needs improvement, partially sustainable): El Seybo has done best in terms of completeness and timeliness of reporting to the General Directorate of USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 ix Epidemiology. All three regional hospitals have specific program indicators within their strategic work plans and annual operational plans; however, there is no systematic analysis of the data. 8. Maternal and Child Health Centers of Excellence Certification (needs improvement, not sustainable): The project has drafted certification manuals and tools and trained evaluators in their application. Before certification or accreditation, the MOH must authorize a facility to deliver services by assessing it against a basic set of standards. By the time of the MTE, only one hospital had not been so authorized. One concern, however, is how the MOH can keep the process unbiased by political issues. 9. Replication Model (needs improvement, not sustainable): Regional directorates have used the CoEx experience to establish customer service offices, hospital information systems, etc., in other hospitals, but none has used the complete set of interventions. Region VIII identified financial resources as the main barrier to replication; for Region VI the barrier was human resources. Another difficulty is the continuing lack of true decentralization of health services. 10. Community Participation (low performance, not sustainable): In 9 out of 10 project hospitals, a community member has been involved in the administration council and, in most cases, also trained in order to actively participate in council discussions. However, for the most part this involvement seems to be perceived mostly as a formal fulfilment of a project requirement. 11. Referral System (low performance, partially sustainable): Project performance on this component was in general low and non-functional. Technical Interventions—Evaluation Questions 2 and 4 1. Quality Improvement of Prevention and Treatment of Newborn Sepsis (high performance, partially sustainable): The three hospitals that carried out this intervention evidenced better newborn care, with improvements in the delivery room and neonatal ward that should help reduce infection risks for mothers and newborns. Hygiene in the delivery room and neonatal ward is appropriate for newborn care and has the elements necessary to prevent nosocomial infections. 2. Maternal and Neonatal Mortality Audits (high performance, partially sustainable): All hospitals audit maternal deaths and have a maternal mortality and morbidity committee to reduce them. Since the project began, the percentage of audited maternal deaths has increased from 16% to 88%. However, care during delivery varied widely from site to site and professional to professional. Nurses and medical staff interviewed were not familiar with the latest research on maternal and infant morbidity and mortality due to overuse of cesarean sections, labor inductions, neonatal hypoxia, and other conditions. The physicians recognized that their C-section rates were high (32%–49%); however they did not articulate an approach to reducing them. It appears that common procedures for perinatal care were not always practiced by skilled providers. 3. Kangaroo Mother Care (KMC) (high performance, partially sustainable): KMC was initiated over 18 months ago at the San Franciso de Macoris Provincial Hospital and about one month ago at Centro Materno Infantil San Lorenzo de Los Minas. The provincial hospital program is evolving well: Data are now carefully maintained and the service has earned a prize from the first lady and created an empowered, committed, problem-solving staff. 4. Quality Antenatal Care (needs improvement, partially sustainable): In general, both providers and patients believe the quality of antenatal care has increased since the project began, e.g., the x USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 percentage of pregnant women who had weight, blood pressure, and uterine fundus height measured during last visit reached 90%. However, management of prenatal care is not well￾integrated. 5. Safe Delivery and Emergency Obstetric Care (needs improvement, partially sustainable): All hospitals have been trained in manual vacuum aspiration, emergency obstetric care, management of the neonate, biosecurity, and national norms and protocols; but management of only one SDP reported having applied guidelines to monitor obstetric and neonatal emergency care. Care during delivery varied widely. User discussion groups had little confidence in hospital emergency services. 6. Active Management of the Third Stage of Labor (needs improvement, partially sustainable): AMTSL has been implemented in all sites visited, though not always consistently, and the number and percentage of women receiving AMTSL increased from 32% to 82%. In most facilities, however, AMTSL requires the presence of specialized personnel in the delivery room. The team could not find much evidence of the use of national protocols for maternal and infant care. Health personnel reported knowing such protocols existed, but they were not always able to refer to specific topics and content. It was not possible to identify procedures for monitoring adherence or sanctions for noncompliance. 7. Prevention of Mother to Child Transmission of HIV and Other Interventions (needs improvement, partially sustainable): Given its coverage, with nearly 100% of women in antenatal care accessing PMTCT services at their first visit, this program offers an excellent opportunity for responding to other health needs of pregnant women, including screening for other STIs, partner HIV/STI testing, and partner involvement in prenatal care. In most of the hospitals, these opportunities are currently being missed. Although rapid HIV tests are available in all PMTCT programs in the hospitals visited, in only two are the test results delivered to pregnant women on the same day. 8. Humanization of Care / Delivery (needs improvement, partially sustainable): There is no consistent understanding among hospital authorities and health care providers of the concept of humanization of care. None of the hospitals uses analgesia in labor, and it is only available for special circumstances in one, assuming an anaesthesiologist is available to administer it. However, both women and providers approved of the possibility of a female friend or relative accompanying the mother during labor, and one hospital reported a very positive experience with community leaders trained as doulas. 9. Family Centered Maternity Care (FCMC) (low performance, not sustainable): All 10 components of the Family Centered Maternity strategy could not be implemented at the Antonio Musa Hospital or the San Vicente de Paul Hospital due to cultural barriers and health staff attitudes; the ETA team therefore concluded that there were not enough aspects of this intervention to be observed. 10. Helping Babies Breathe Curriculum for Newborn Resuscitation: The indicators of success of HBB are (1) training of master trainers carried out; and (2) technical assistance provided for implementation at scale and supervision of HBB in all 10 Centers of Excellence. The first indicator was fully completed (100%) in the three hospitals where the strategy is implemented, and after training staff felt empowered, citing situations where the knowledge acquired had been applied. 11. Biosafety (Environment Preserved): The CoEx project called for preservation of the hospital environment through biosecurity systems for prevention and control of infections and negotiation of waste management regulation with local authorities. These aims were achieved or being achieved, with hospital authorities and providers reporting major USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 xi improvements. Limited access zones have adequate signs, and appropriate clothing and other expected procedures are clearly visible. However, some of the indicators for this component exceed the project’s range of action because they depend on local and national policies for biological waste management. 12. Post-Abortion/Partum Care: At simple observation, delivery rooms were clean and had the proper infrastructure to prevent infection. The few observations made during deliveries showed that health staff treated patients with respect, asked them about previous deliveries, and informed them about the procedures. Attention to the newborn was carried out according to norms. However, in two hospitals health staff did not have the patients’ clinical records, background information, or lab results, and the partographs were incomplete. In one hospital, health staff did not examine the integrity of the placenta, and some staff stayed distant from the patient during the expulsion phase. Also HIV and syphilis test results very often do not get into the clinical records. Critical and Appropriate Activities—Evaluation Question 5 During visits to hospitals, the MTE team observed other projects that may bring new opportunities or synergy with the Centers of Excellence: Psicoprofilaxis de parto (Maternidad San Lorenzo de Los Mina) is an intervention for a first pregnancy. It includes exercise and preparation for delivery as well as pain management strategies. Programa de Madres Tutelares (mentoring mothers) uses experienced mature tutors in the community to accompany teenage pregnant women from prenatal care and delivery through care of the newborn. The doulas program at San Vicente de Paul Hospital recruits labor companions or birth workers among community leaders. Doulas provide non-medical support to women and families during labor, childbirth and the postpartum period, contributing to the humanization of care and pain management. The same hospital also trains nurses to attend deliveries. National health norms do not allow non￾physicians to attend deliveries; however, nurses do so with the consent of the doctor in charge. Hospital management has provided additional training to formalize this arrangement, which has been supported by Emory University, with excellent results. CHALLENGES Among challenges to institutionalizing interventions the project may now confront are these: 1. Certification of Centers of Excellence. The certification process is paramount to consolidate and institutionalize CoEx processes, but in carrying out the actual certification process with the MOH problems may arise. However, if that can be done during the life of the project, it would be a remarkable success, regardless of how many hospitals are certified. The challenge would then be to replicate the process regularly. 2. Documentation: The project needs to complete the tools and manuals, and hospitals must continue implementation and make corrections as needed. 3. Replication: The CoEx project has raised MOH expectations; however, only a few senior MOH staff understand the complexity of replicating the “full-blown” model. 4. Quality Maternal-Infant Services. Improving the quality of care is not the responsibility of the project alone. The MOH is aware of its role in the process, and is working to that end. 5. Political Environment. The next national election may affect the project. CoEx should prepare a plan to educate new health authorities about the project and get their endorsement. xii USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 FUTURE DIRECTIONS AND RECOMMENDATIONS The hospital, DPS, and SRS management teams believe the interventions they have been working on will continue to the next administration. However, they also recognize that they need all the CoEx tools to be institutionalized by different units and departments of the health system. The Health Regions were identified as the most suitable locus for the processes to develop a Center of Excellence. The following general suggestions may help decision-makers to build on project successes: 1. The project has successfully improved administration and clinical care areas of the 10 hospitals and trained administrative personnel. However, in the next stage, more resources must be concentrated on quality of health care and management action to transmit competencies transference. More clinicians and specialists should be hired to help field coordinators in this teaching-learning process. In the next stages, field coordinators should emphasize management training for regional directors where further enhancement is necessary to transmit competencies. 2. A plan for continued management training is needed. 3. It might be effective to initiate the process of transferring competencies in one or two regions and later expand to 10 hospitals selected to be Centers of Excellence. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 1 I. INTRODUCTION USAID/Dominican Republic (DR) commissioned an external midterm evaluation (MTE), through the GH Tech Bridge Project, to evaluate implementation of the Mission’s Intermediate Results 4 (IR4), “Increased access to and delivery of quality maternal and child health care services in selected areas.” The aim of the evaluation was to assess the performance of USAID/DR in achieving “increased access to and delivery of quality maternal and child health care services in selected areas.” To achieve the intermediate result, USAID/DR is implementing health activities through the Maternal and Child Health (MCH) Centers of Excellence (CoEx) project (2009– 2014) and the MCH Integrated Program (MCHIP) (2010-2012). EVALUATION OBJECTIVES AND USES The evaluation took place at the critical midpoint in the MCH Centers for Excellence project (March-April 2012), which offers an opportunity to make improvements to ensure that changes are sustainable and the impact is long-term. Thus, the main objectives of the MTE were to help determine what components and project aspects were working well and why and which were not and why; and make recommendations for modifications and midcourse corrections to help guide the MCH CoEx project to a successful consolidation phase with sustainable results. The primary audience for the evaluation report will be the USAID/DR health team and the implementing partners. A report with an executive summary and recommendations will be provided to the Government of Dominican Republic (GODR) and development partners. USAID/DR will use the report to inform its phase-out plan by prioritizing aspects of the program according to performance and potential for sustainability. 2 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 3 II. BACKGROUND THE PROBLEM AND THE PARADOX According to the 2007 DR Demographic and Health Survey (DHS), maternal mortality was then still high (159 per 100,000 live births) even though 98% of all deliveries take place in a hospital. Adolescent pregnancy was also high at 23%; infant mortality was 31 per 1,000 live births; and neonatal mortality accounted for 68% of total infant mortality (22 per 1,000 live births). Although in 2011 maternal mortality was reduced by 23% and infant mortality by 25%, health indicators make it clear that the health system is unable to adequately address the needs of the Dominican population, especially the most vulnerable, such as women and newborns. Although 98% of deliveries are attended at health facilities, the maternal mortality ratio is still high, as are neonatal and infant mortality ratios. Moreover, between 61.7% and 71.8% of all DR maternal deaths are considered to be preventable, which raises questions about the quality of care in the country’s health facilities. The explanation of this apparent paradox is related to the poor quality of sexual and reproductive health services offered in the DR, particularly in the public sector, where 76.0% of all institutional deliveries take place (Miller 2003; Ruminjo 2003; Pérez-Then 2008, 2010; Miric & Pérez-Then 2011). Previous assessments of sexual and reproductive health care in the DR have identified a wide range of limitations, among them (a) insufficient, unprepared, and unmotivated staff; (b) deliveries, dilation and curettage (D&C), and other procedures done by medical residents and students without supervision; (c) difficulties with the system of referrals from lower-level facilities, resulting in over-demand for services in regional and national referral establishments; (d) inadequate infrastructure and unsatisfactory hygiene; (e) generalized disregard of national norms for attention to delivery and obstetrical emergencies; (f) failure to adjust the type of attention offered to the specific needs of health care users; (g) inhumane and degrading treatment of users, particularly where demand is high; and (h) limited integration of information, education and communication (IEC) and family planning (FP) activities into different sexual and reproductive health services. A large proportion of maternal deaths occur postpartum from hemorrhages, infections, and other complications. The World Health Organization states that an increase in the number of trained health assistants present during delivery and postpartum is directly related to a reduction in maternal mortality, yet that has not happened in the DR. RESPONSE TO THE PARADOX Recognizing that the health system was unable to address health needs, the GODR passed two laws, the General Health Law 42-01 and the Social Security Law 87-01, to build the capacity of the government to improve the health system, whether public, private, or nonprofit. These laws were designed to define the health system, decentralize delivery of health services, create a national insurance scheme, and establish demand-driven financing. In addition, in 2006 the GODR launched the Zero Tolerance Strategy to reduce preventable maternal and child deaths, vaccine-preventable diseases, vertical transmission of HIV, and deaths related to dengue and malaria. At the time, USAID/DR funded both the CONECTA Project 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 (2006–2008), implemented by Family Health International, to increase sustainable access to quality maternal and child health services and BASICS, to reduce neonatal sepsis in three public hospitals and the Kangaroo Mother Care program. USAID/DR has since launched two complementary implementing mechanisms: (1) The Maternal and Child Centers of Excellence (CoEx) project (2009–2014) and (2) the MCH Integrated Program (MCHIP) (2010–2012). The two projects were designed to increase access to and delivery of quality MCH care services in 10 pilot hospitals and help to achieve USAID/DR Assistance Objective “Improvement in the health of the most vulnerable populations in the Dominican Republic,” Intermediate Result 4, and sub-intermediate results IR4 Increased access to delivery of quality maternal and child health care services in selected areas: Improvement of hospital management systems Improved quality of prenatal care, delivery, and postpartum and neonatal care Strengthened community oversight role The projects were designed to increase access by improving prenatal, obstetric, and neonatal care and hospital biosafety practices and norms; establishing customer services; and strengthening institutions and management through other activities. Table 1 shows basic facts about each program. Table 1. Program Data Maternal and Child Centers of Excellence (CoEx): Duration: February 2009–February 2014 Total USAID funding: $15.5 million Beneficiaries: Women of reproductive age and newborns Counterpart: Ministry of Health through 10 hospitals, 3 Provincial Health Directorates, and 3 Regional Health Services Geographic coverage: Nationwide Implemented by: Abt Associates Maternal and Child Health Integrated Program (MCHIP) Duration: April 2010–September 2012 Total USAID funding: US $650,000 Beneficiaries: Newborns Counterpart: Ministry of Health through the Centers of Excellence Geographic coverage: Nationwide Implemented by: Partnership of JHPIEGO (prime), JSI, Save the Children, PATH, JHU/IIP, Broad Branch, and PSI USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 5 Since the CoEx program began in February 2009, there has been an observable reduction in maternal deaths, from 201 in 2010 to 168 in 2011—a 16.4% drop. For the 10 project hospitals alone, there were 61 maternal deaths in 2010 (30.5% of the national total), and 31 in 2011 (18.5%); compared with the national figures, the 49.2% reduction in project hospitals was much greater. Although there was a collective, and effective, effort by the MOH, implementing partners, and international supporters to reduce maternal mortality, the CoEx contribution was clearly significant. The goal of the CoEx project is to help reduce maternal, newborn, and infant morbidity and mortality by upgrading management and MCH technical capacity in 10 regional and provincial DR hospitals. The project had the objective of “improving equitable access to quality maternal￾child services focusing on critical interventions and management improvements that enhance quality and efficiency,” and the crosscutting result of “supporting the dissemination of best practices and serving as training centers for other facilities in their respective networks.” To achieve this goal, the project has specified the following intermediate results: Ten hospitals developed as Centers of Excellence; Three Provincial Health Directorates developed as Centers of Excellence; Regional Health Services networks strengthened; and Region V technical interventions consolidated. The project is being implemented in three phases: preparatory, implementation, and consolidation. During the preparatory phase the project provided training on communication, team building, and change management; implemented quality management training using the Common Assessment Framework (CAF) tool; supported sites as they drafted improvement plans; activated hospital administration councils; and initiated the hospital certification process. In the next phase the project implemented interventions (e.g., CAF, the hospital management information system, strategic planning, basic infrastructure, and systems for perinatal information, biosafety and infection control, customer service, referrals, and medicines and supplies) to enable sites to become certified as Centers of Excellence; drafted certification criteria; and began to disseminate lessons learned and best practices to surrounding facilities. The final phase will focus on sustainability, institutionalization, and documentation. This MTE took place at a time when the MCH CoEx project is transitioning from implementation to consolidation, and USAID/DR resources for it are declining. The CoEx project was complemented by MCHIP through technical support for high-impact, evidence-based newborn health interventions: prevention and treatment of newborn sepsis; family-centered maternity care (FCMC) and Kangaroo Mother care strategies; reinforcing the newborn health national work plan in line with the Latin America and Caribbean (LAC) Neonatal Alliance Regional Strategy and Action Plan; and use of the Helping Babies Breathe (HBB) curriculum for newborn resuscitation. Because MCHIP ends in September 2012, the CoEx project will integrate MCHIP newborn health interventions into its consolidation and replication activities. In its collaboration with and technical support to the MOH in the area of newborn health, MCHIP’s specific objectives were to: 6 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Scale up to 2–3 more facilities the intervention for quality improvement of prevention and treatment of newborn sepsis in the CoEx program as part of the regional strategy to improve newborn health (for a total of 4–5 facilities). Reinforce FCMC and Kangaroo Mother care strategies in CoEx sites with trained staff; initiate expansion to at least one more center for a total of 3–4 facilities. Provide technical assistance to the GODR to enhance its newborn health national work plan in line with the LAC Neonatal Alliance Regional Strategy and Action. Use the HBB curriculum for newborn resuscitation in all CoEx facilities. With MCHIP technical support, three CoEx sites continue to participate in activities with the BASICS Project. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 7 III. METHODOLOGY1 THE EVALUATION TEAM A multidisciplinary team drew members from the MOH, USAID/DR, USAID/Washington, and four independent external consultants, totaling 8 team members. Dr. Eddy Perez-Then, Ms. Marija Miric, Dr. Rafael Montero, and Dr. Marcelo Castrillo were the independent consultants; Dr. Olga Arroyo and Dr. Donatilo Santos represented the Department of Maternal and Child Health of the MOH; Mr. Derek Sedlacek represented USAID/DR; and Ms. Veronica Valdivieso and Dr. Peg Marshall represented USAID/Washington. All team members had experience in the country, with USAID’s maternal and infant health interventions, and in evaluation methodologies. The MTE team was then divided into four evaluation groups in charge of: (1) interviews of staff from Provincial Health Directorates and Regional Health Services; (2) interviews of staff from hospital management teams; (3) group discussions with users of maternal-infant health services and health service providers; and (4) rapid observation of the maternal and neonatal care setting, services delivered, and interviews with health care providers of these wards. KEY EVALUATION QUESTIONS The five key questions the MTE aims to respond to are: 1. What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? 2. What technical interventions have been the most/least successful in ensuring the achievement of the project results and why? 3. What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? 4. What technical interventions appear to be the most/least sustainable and why? 5. 5. Has the project targeted the most critical and appropriate activities, what other opportunities exist? DATA COLLECTION AND ANALYSIS The two complementary projects for quality improvement of the maternal-child services, the MCH CoEx and MCHIP were evaluated using rapid Assessment Process principles. The process included an extensive database search; triangulation of information sources (observation, interviews, and literature review); iterative analysis; and additional data collection as needed for a rapid and preliminary understanding of the status of the project. The rapid assessment methodologies applied were: Document review and examination of data on key indicators Interviews with stakeholders at different levels Direct observation 1 See Annex 21 for details. 8 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Group discussions with users of services and hospital staff To classify the evaluation outcomes of the two projects sets of interventions, the MTE team set up a qualitative categorization based on two variables, performance and sustainability (Table 2). Table 2. Qualitative Score Scales and Definitions Performance Sustainability Low: Performance was consistently below expectations in most essential areas of responsibility. Not sustainable: The intervention has little potential for sustainability. Needs improvement: Performance did not consistently meet expectations in one or more essential areas of responsibility. Partially sustainable: Some level of sustainability has been achieved, but external assistance is still needed to maintain the level of effort. High: Performance consistently met expectations in all essential areas of responsibility, at times possibly exceeding expectations, and the quality of work overall was very good. Fully sustainable: Individuals and teams have taken ownership of development processes, including financing, and can sustain the results and outcomes beyond the duration of the project, conditional on the DR MOH level of compromise. ASSESSMENT LIMITATIONS There is one limitation in the approach the MTE team used to assess the USAID Intermediate Result 4 outcomes and consequently funding mechanisms. The evaluation of project processes and effects was qualitative: the MTE team gathered information from a variety of sources and levels and made subjective judgments in rating specific project interventions as relevant, sustainable, and replicable. However, the approach was systematic; issues and facts were explored through predetermined discussion guides and observation lists (Annexes 4 through 13). When the responses of informants and levels were combined, they usually supported one another, and where they were different or there were contradictory points of view, the MTE team kept on exploring in depth to determine if they were true differences or simply perceptions. After data collection, the MTE team met for three days to process the information and formulate conclusions and recommendations. The team brought the assessments perspectives beyond those of the implementing partners and stakeholders about the strengths and limitations of interventions. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 9 IV. FINDINGS This MTE offers an opportunity to reflect not only on the performance of the two USAID funding mechanisms to strengthen the health system and reduce maternal and neonatal mortality, but also on the broader questions of what constitutes a successful process for upgrading a health facility to deliver quality maternal-infant care. Evaluation findings and priority actions are organized to respond to the evaluation questions: Programmatic and Health Systems – What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? – What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? Technical Interventions – What technical interventions have been the most/least successful in ensuring the achievement of the project results and why? – What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? – What technical interventions appear to be most/least sustainable and why? Following the qualitative score categories (see Methodology, section D), the interventions fall into the two-entry matrix (see below) and the scoring is given from left to right and bottom to top. Using these criteria, the interventions were clustered as perceived by the evaluation teams. Figure IV.1. Intervention Matrix 10 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 PROGRAMMATIC AND HEALTH SYSTEM What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? Previous studies conducted on the quality of MCH services in the DR (Miller et al., 2002; Freedman, 2003; Quiterio et al., 2008; Pérez-Then et al, 2008) suggest a need, as a foundation for quality improvement, to strengthen health systems and management capacities by means of strategic planning, monitoring and evaluation (M&E), information systems, quality improvement, and resource management. The CoEx project focused on integration and strengthening of health systems as a foundation for management of maternal-child services; the interface of both was considered to be the basis for design and development of the centers of excellence as a way to expand equitable access to quality MCH services. As ways to reinforce the system, several tasks have been proposed, such as: Strengthen the functions of the management team to initiate change. Strengthen planning and monitoring capacity. Monitor and evaluate performance of management systems. Adopt an automated information system at the facilities (through SIGHO modules). Strengthen the human resources management system. Strengthen the response level and care for service users. The original target of drafting strategic plans and work plans and updating and implementing self￾assessment through the CAF tool, as well as trained managerial teams being familiar with and using management tools successfully, was reached in all ten hospitals (100%). Furthermore, management teams described some of these tasks as “success facilitators,” implying that the time and effort invested in them triggered positive changes in institutional structure, management and, most important, attitudes, aptitudes, and empowerment of health care providers, making “fertile ground” for successful interventions to promote evidence-based and high-quality MCH services. Figure IV.2 depicts project interventions clustered by the interaction of the variables, performance and sustainability. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 11 Figure IV.2. Intervention Matrix Applied Infrastructure (high performance, fully sustainable) According to discussion groups (DGs), hospital infrastructure has shown major improvement. Both health users and providers acknowledge that project adjustments to hospital infrastructure and equipment have greatly enhanced the quality of care and the perception of quality. One aspect that all informants highlighted was hygiene in different hospital areas, achieved mostly by training and empowering the support personnel. After they were included in training, they reported more job satisfaction and more motivation to keep hospital areas clean and functioning. Priority Actions To consolidate sustainability, project interventions should consider promoting maintenance and adequate use by providers of hospital installations and equipment. Customer Service System (high performance, fully sustainable) One of the most visible interventions and the one that gave the management teams the most satisfaction was the Customer Service Office. The CoEx project provided training, equipment, and infrastructure for the system, including development of the SIGHO software (Sistema Integral de Gestión Hospitalaria-SIGHO). The benefits of this system are clear. Hospital staff reported that the difference before and after the system is extreme. This intervention eliminated duplication of clinical records. Medical personnel can easily assess the medical history of a patient over months or years. However, records management continues to need improvement, and the MTE team observed that some clinics had duplicate records and records for new patients were incomplete. Patients in group discussions also reported a perception of improved health facilities. They mentioned that lines to get a ticket to be seen by staff were clear and patients could see their turn displayed on the screen, which significantly reduced waiting time and the related stress. 12 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Some MTE team members felt that there was a minor disconnect between departments, and not all departments had on-site access to all records. For example, in one hospital, neonatology did not have access to the mother’s prenatal records, which would have facilitated clinical care. It is possible that this system will only reach its full potential when all stakeholders are informed, and it is promoted to both providers and patients. Some hospitals visited were not performing all expected tasks. In the group discussions, it was mentioned that there is still confusion as to where to obtain more information about specific services, where to ask questions, and where to present complaints. The MTE team also observed that all hospitals had “suggestion boxes” but there was no privacy; any one could see the notes; because the majority of those who use these services have little education, a suggestion box may not be the best way to express complaints or suggestions. Health staff interviewed reported no clear mechanisms to process the information obtained through suggestion boxes; and no action was ever taken to respond to information received in the boxes. The lack of systematic processing of complaints and actions was confirmed by users of the system, who reported that they do not complain any more because the health personnel would not take it well and might even be vindictive. The CoEx interventions aimed at strengthening hospital systems and management capability have been highly effective and have indirectly had significant impact on the quality of MCH care in project hospitals, which supports their being rated high by the MTE team. Some of them are also being efficiently consolidated and can feasibly be directly replicated by project hospitals and lower-level health care centers. However, as yet these interventions should be considered only partially sustainable, since further interventions and commitment from hospital authorities and health care providers are needed in order to monitor strategic and operational plans, use the information system effectively, make evidence-based decision management decisions, and make customer service offices more responsive to patients. Priority Actions To make interventions related to hospital systems and management capability more effective and sustainable, several priority actions have been identified: Promote and support hospital authorities in M&E of the results of hospital strategic and operational plans, actively involving health care providers in this process. Train and guide maternal and child health care providers in systematic use of relevant clinical data produced by the SIGHO system for the evidence-based decision making, which will help promote an information culture in the hospitals. Ensure that prenatal clinical records are available before, during, and after delivery and in the neonatology wards. Strengthen and expand the current functions of customer service offices, including information mechanisms and complaints management. Continue the successful strategy of transferring basic and advanced technical capabilities to hospital authorities and health care providers, empowering them to define and implement desired changes and goals according to their unique needs. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 13 Design and initiate actions to gradually transfer training and monitoring functions currently handled by CoEx management teams to the DPS and regional services, so they are prepared for full assumption of these functions when the project ends. This process should consider involving highly trained local personnel that participated in the CoEx project within the MOH structure. Common Assessment Framework and Strategic Planning (high performance, fully sustainable) Initially 25 hospitals were invited to participate in the CoEx project; those interested went through a CAF self-assessment exercise 2 and of those, 10 were selected. The self-assessment, and strategic planning adjusted to the specific context of each hospital, have helped to empower hospital authorities and health care providers as stakeholders and active contributors to the quality of MCH and other health care services at each site. Teams in Regions VI and VIII reported that competition and selection was critical, and that commitment will be important in choosing sites to replicate. Management for Change training and the postgraduate diploma in health and social security management were identified as having contributed to the success of the project in the hospitals, and in the region. All hospitals were also supported in creating tailored strategic and quality improvement plans. Management for Change was viewed as a sustainable process and a critical step for replication efforts; previous studies in the DR had also highlighted the importance of attitudes and the responsibilities assumed by individual health care providers to the quality of MHC services. The score on this intervention (high: fully sustainable and high performance) showed that all management teams participated in all the tasks. Management teams also reported that all should be drawing up annual operational plans, but the project added quality and motivation. Motivation was interpreted as what the project called Theory of Change: individuals and groups change because they see the need for change and the potential benefits in terms of their personal interests. In addition, all were supported in creating quality improvement and strategic plans. After training, all centers drafted strategic plans for 2010–2015 and operational plans for 2010 and 2011; none yet had operational plans for 2012. The MTE team did not observe any process to assess the quality of operational plans and their implementation—a weakness that was confirmed when the M&E component was examined. There also seemed to be an overlap of responsibilities that could undermine the efficiency of implementation. Priority Actions To increase both performance and sustainability, a more structured assessment of the quality and monitoring of implementation of strategic and operational plans should be promoted. Roles and responsibilities for this process within the hospital and the health system in general also need to be clearly defined. Medicines and Supplies System (high performance, partially sustainable) The management teams reported that all hospitals now have in place a medicines and supplies system, but the quality of the work varies. Management teams considered the system to represent remarkable progress for the quality of MCH services in their hospitals, and hospital 2 CAF is a tool used by the Ministry of Public Administration to reward good practices in the public sector. 14 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 personnel and authorities were empowered to effectively manage their own medicines and supplies. The SIGHO application used for the supply chain was available in all assessment sites, and medicines and supplies are being much better managed. Pharmacists in the hospitals visited reported that they now have a voice in management team meetings, and shortages of supplies and medicines—particularly for MCH services—are much less frequent. However, PROMESE-CAL, the public sector purchaser and distributor of essential medicines, is failing to consistently provide the medicines needed. The MTE found performance of the medicine and supplies system to be high, particularly in comparison to how it previously functioned before the project implementation, despite aspects of the system that still need improvement. However, it was scored as only partially sustainable, given that external support and hospital-based financing is still crucial to assure availability of medicines and supplies. The teams reported that this component is ready for replication and will help reinforce continued improvements in all hospitals. Hospital Management Information System (high performance, partially sustainable) Management teams reported that using SIGHO software as a fundamental tool has improved the value of their information system. Health care providers considered the possibility of accessing women’s prenatal records in the emergency, labor, and delivery areas to be an important contribution of the project to quality of care, especially better management of obstetrical emergencies. The MTE team identified a number of areas of concern on the Perinatal Information System, a SIGHO subsystem that is being used suboptimally in some of the hospitals. For instance, staff complained that the font was so small that many could not read the chart and wrote the notes on the margins; others said it was too long and required too much information. Also, in a number of hospitals, patient records are not routinely sent to labor and delivery (L&D) when a patient is admitted—even when they are available; thus delivery staffs are not familiar with a patient’s history and antenatal care. Likewise, women delivering at referral hospitals do not come with prenatal records, which clearly affects the quality of care because laboratory and other prenatal data remain unknown. Finally, health personnel and hospital authorities reported that clinical data collected through the SIGHO system are not used very effectively in most hospitals and are not routinely employed for decision-making, strategic planning, and hospital management. This was corroborated by the management teams, who affirmed there is a need to promote and build up the information culture within hospitals in order to change attitudes and previously learned behaviors while raising awareness of the importance of measurable indicators and evidence-based decision￾making. Priority Actions The system needs to be strengthened by promoting use in emergency and delivery areas of information in prenatal and other records that is now readily available through SIGHO. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 15 Emergency and Disaster Preparedness (low performance, not sustainable) The ability of hospitals and their personnel to anticipate and respond to disasters that might affect public health is a core component of national, state, and local emergency preparedness plans. The degree to which hospitals are prepared for potential emergency environments is expected to positively correlate with their ability to respond effectively to the health needs of their patients (Hodge 2009). Hospital preparedness for emergencies and disasters is one component of the CoEx project; including formation of emergency and disasters’ committees, support to hospital authorities in formulating emergency and disaster plans, and structural, nonstructural, and functional evaluations of health establishments. Four of the six sites visited by the MTE team had emergency plans. Two of them have done drills and two have made contingency plans. Hospitals with emergency and disaster plans have designated evacuation route and meeting points, although these were not clearly marked. Some hospital management teams reported that they have had difficulties coordinating with the MOH Department of Disasters and Emergencies, which insisted that preparedness for emergencies and disasters is their responsibility and neither plans nor interventions can be implemented without their direct supervision. The project management team identified this difficulty as a serious obstacle to the success of activities aimed at preparing hospitals for emergency and disaster situations, and thought that effort and resources used for these activities might have more impact on the quality of MCH health services if they were reallocated to other strategic CoEx components. Priority Actions A programmatic decision should be made on whether to (a) continue investing in this project component so as to significantly improve its performance and sustainability and clearly delimit functions and responsibilities with the central level of MOH, or (b) limit these efforts to technical support to hospital authorities in initiatives promoted by other groups while prioritizing other actions that would directly improve the quality of MCH care. Monitoring and Evaluation (needs improvement, partially sustainable) There have been many advances in epidemiology and surveillance. For example, since the project began, El Seybo has been the number one province in terms of completeness and timeliness of information reported to the General Directorate of Epidemiology. This could be related to mandatory reporting and the use of field coordinators as part of the project M&E strategy. The management team at all three regional hospitals reported to the MTE team that they have specific program indicators within their strategic work plans and annual operational plans; however, they still considered that area to be a major weakness. There is no systematic and regular analysis of information, and decision-making processes are not based on quality data. Priority Actions The M&E component needs to produce and use data more effectively. Annual plans need better data and more analysis on which to base service delivery priorities, and to improve the quality of care. It is also recommended that a data quality assessment tool be added to the regular supervision of services. 16 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Maternal and Child Health Centers of Excellence Certification (needs improvement, not sustainable) The MCH CoEx project envisioned winding up hospital strengthening by implementing a certification process for hospitals to become Centers of Excellence in Maternal and Infant Care. The main functional areas of hospitals and DPS would be assessed against specific maternal and infant care standards. The project has drafted all the necessary certification manuals and tools, and a cadre of evaluators has been trained to apply them, so it is prepared for the first round of hospital certification. Before certifying or accrediting a health facility, the MOH has to authorize it to deliver services by inspecting and assessing the services against a basic set of standards. In that regard, the project has trained and provided equipment to the 10 hospitals to pass the MOH inspections; only one project hospital had not been so authorized. When the MTE team asked MOH authorities about certifying a Maternal-Infant CoEx, all reported that they had participated in developing standards, tools, and implementation manuals, and agreed that the MOH will soon take the necessary steps. Nevertheless, there were some questions that the authorities did not properly answer. For instance, how does the MOH acquire the capacity to certify a health facility, not only on maternal-infant care but also for a wider range of services, and which department or unit would be in charge, since services are not yet fully decentralized? How can the MOH maintain an unbiased certification process and avoid political pressures? How can the certification process be kept unaffected by the usually large staff turnover after every change of administrations? The MTE team felt that project staff and government authorities need more discussion and agreement on a clearer path from the current status to where the certification process can be institutionalized in the country. Priority Actions This is the highest priority intervention for institutionalizing the Centers of Excellence model and its long-term sustainability. During the second phase staffs from the project and MOH should together create the certification mechanism, preferably as an independent body or a public-private partnership. 9. Replication Model (needs improvement, not sustainable) The model is being replicated through specific but limited training activities. Regional Directorates have used the CoEx experience to improve filing systems and establish customer service offices, hospital information systems, and other interventions in other hospitals in the region, but none has the entire set of CoEx interventions. Region VIII identified financial resources as the main barrier to replication; for Region VI the barrier was human resources. However, both expressed commitment when resources become available. Another difficulty for the regional and district health authorites in replicating the model is that health services are still not yet truly decentralized. Currently, roles and functions of the SRS and the DPS overlap. Yet when hospital management teams were asked which should be responsible for replicating the model, all said that it should be the SRS. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 17 Priority Actions The team considered a more viable alternative to replication of the complete model would be partial replication of successful experiences and best practices accumulated in different hospitals. Community Participation (low performance, not sustainable) Community participation in health services implies a partnership between local communities and health facilities in planning and utilizing health activities so as to benefit from more local self￾reliance and social control over primary care infrastructure and technology (Rifkin 1990). Multiple benefits of such partnerships have been reported, for both the health system and communities themselves, including better use and sustainability of health services, mutual exchange of resources, positive changes in health behaviors, and general community empowerment in recognizing their own health-related strengths and risk factors (Kahssay and Oakley 1999). Previous studies in the DR, however, report only limited interaction between the health care facilities and communities in terms of MCH services (Pérez et al. 2011), except for isolated projects and interventions that specifically targeted such involvement (Miric et al. 2008). The CoEx project integrates a component that specifically promotes such alliances to Form or strengthen hospital administration councils through participation by community members. Train communities in users’ rights. Create mechanisms for diffusing information. In nine of the project sites, the hospital authorities reported that, as part of the CoEx project, a community member has been serving on the hospital administration council and in most cases had been trained in the skills necessary to actively participate in council discussions. The qualitative interviews and direct observations, on the other hand, suggest that this involvement is treated as formal fulfillment of a project requirement rather than a real, functional, alliance between hospital and community. The one exception is the San Vicente de Paul Hospital in San Francisco de Macorís, where community involvement in hospital activities was established long before the CoEx interventions were planned. In general, however, health care providers and hospital authorities in the other hospitals did not seem to be open to this type of integration; the general attitude was "they (users)” vs. “we (providers)." Hospital San Vicente de Paul also reported successful experiences of community integration, including training of doulas (who provide nonmedical support to women and families during labor and childbirth and postpartum), and volunteer community-based committees integrated into hospital activities and assisting with research in the community. This history may explain why this hospital was more advanced in humanizing care of mothers and infants. While acknowledging the benefits of community participation in hospital activities for the humanization and quality of MCH care, as observed in the experience of San Vicente de Paul Hospital, the management team expressed their concern about whether it will be feasible to achieve this same level of community participation in other project hospitals within the rime remaining for CoEx time. This concern becomes particularly relevant when the numerous other strategic priorities of the CoEx project are taken into account, including the effort still needed for interventions that directly improve the quality of MCH services. 18 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Priority Actions A decision is needed on whether to (a) reinforce and significantly amplify interventions to achieve authentic community participation in project hospitals, or (b) limit these efforts to continuing the formal participation of community representatives in hospital administration councils while giving more attention to actions that directly improve the quality of MCH care. Referral System (low performance, partially sustainable) The lack of connections between different actors providing MCH services in DR, including community-based institutions and providers, has been repeatedly reported as a serious obstacle to integration of services and adapting them to community needs (Pérez-Then et al. 2011; Miric et al. 2006). It was also identified as an obstacle to effective response to obstetrical emergencies (Miric et al. 2008). Recognizing the situation, the CoEx project originally proposed consolidating a documented referral and counterreferral system in each hospital so that the hospital could coordinate actions with other health facilities. This system should also incorporate communication and transportation services that would guarantee timely and safe delivery of laboratory samples and access of mother and child to adequate care. In addition to the referral and counterreferral system itself, this component involves creation of a service portfolio of communication and transportation systems, an appointment management system, and linkages with primary and NGO networks. At this point, referral and counterreferral mechanisms are only vaguely defined in project hospitals. Some higher-level facilities receive patients from lower levels, but there are no procedures in place for counterreferrals, and the original provider does not receive feedback on outcomes of treatment of the referred patient. Usually patients are simply just sent on to the referral site without a clinical record or even a note. Usually a physician who wants to refer a patient calls the hospital and asks the doctor in charge to accept the patient. One team reported that the referral hospital in Santiago does not accept any patients without a cell phone call from the physician making the reference; the patient appearing without that would have to work out how to get to another hospital. Thus, referral depends heavily on primary relationships between specific physicians rather than on clear institutional mechanisms and procedures. In some of the hospitals, it was possible to observe specific positive referral and counterreferral experiences, such as use of mobile phones for direct communication between facilities at different levels, and community participation in post-delivery follow up of women who gave birth in the hospital. However, these experiences are not yet structured and generalized enough to constitute an effective system of referral and counterreferral. In general, this component was rated low and nonfunctional, reducing the chances for timely response to urgent MCH needs, including follow up and monitoring of treatment outcomes. A project priority should therefore be to build up this component by drafting through manuals that define clearly step-by- manner referral and counterreferral procedures to be followed by each hospital involved. CoEx managers thought that most of the aims of this component were beyond direct project reach and that, while it might be able to help promote actions that reinforce connectedness within the health services network, the project should not expend resources and efforts on it USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 19 beyond such actions. In that sense, CoEx might decide to focus on building up direct communication between directors of ob/gyn departments at higher-level facilities with lower￾level facilities that often refer patients to it, particularly to promote effective response to obstetric emergencies. The project managers thought that only limited effort should be invested in transportation systems. Priority Actions The project management team should analyze interventions within this component so as to set priorities and define feasible strategies to strengthen specific aspects of the referral and counterreferral system. Some successful strategies at least partially in place in some of the hospitals visited, such as use of technology (teleconferences, social networks, mobile phones) for inter-hospital and interprovider communication, could perhaps be generalized to other settings. Closer direct communication between the directors of tertiary ob/gyn departments lower￾level facilities that often refer patients to it could promote more effective response to obstetric emergencies. TECHNICAL INTERVENTIONS What technical interventions have been the most/least successful in ensuring the achievement of the project results and why? What technical interventions appear to be most/least sustainable and why? As noted in the Background section, the DR paradox of both high coverage of antenatal care and institutional deliveries and high maternal and infant mortality, has been repeatedly linked to poor quality of care, particularly in public health establishments, where 76% of all institutional deliveries take place (Miller 2003; Ruminjo 2003; Pérez-Then 2008, 2010; Miric & Pérez-Then 2011). Previous studies have highlighted the lack of familiarity and observance of national norms and protocols by health care providers (Pérez-Then 2008, 2010). Although these norms emphasize, among other aspects, the importance of humanizing care, including the guarantee of warm and respectful treatment, a safe delivery, and postpartum follow-up, several fairly recent studies suggest that degrading treatment often ignores women’s basic human rights (Miller et al. 2003; Miric et al. 2008). Surprisingly, then, the levels of satisfaction reported by users of DR public maternal and child health services tend to be relatively high (Miller et al. 2003; Hernández & Romero 1993; Pérez-Then et al. 2011), though this is mostly explained by sociocultural factors (Miric et al. 2006, 2008). That is why caution is needed in taking user satisfaction as a direct indicator of health care quality. 20 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 The CoEx project focused on promoting and consolidating evidence-based quality MCH services in the hospitals selected in order to: Build up the quality of maternal-child care. Implement the CoEx strategy. Improve newborn care. Improve follow-up of mother and newborn in the first 72 hours postpartum. According to the projects FY 2010 and FY 2011 figures, the CoEx has brought up a number of outcome indicators (USAID 2012) as a result of compliance with national norms and protocols (see also Figure IV.3), specifically: The number and percentage of women receiving active management of the third stage of labor (AMTSL) increased from 32% (base line) to 82%.3 The percentage of pregnant women who had their weight, blood pressure and uterine altitude measured during their last visit reached 90%. The percentage of episiotomies reached an acceptable level for a referral hospital (25%). The percentage of audited maternal deaths increased from 16% to 88%.4 The project has been less successful in reducing the percentage of cesarean sections performed in the target hospitals, which remains high at 44%, or increasing the number of pregnant women who receive their HIV test results and post-test counseling. The management teams reported that the impact on these indicators was suppressed by the current structure of the health system, inefficient supervision mechanisms, and MOH health policies. 3 Data provided by the Centers of Excellence Project M&E staff. 4 Data provided by the Centers of Excellence Project M&E staff. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 21 Figure IV.3. Intervention Matrix Additional Application Quality Improvement of Prevention and Treatment of Newborn Sepsis (high performance, partially sustainable) Worldwide, infections claim an estimated 1.4 million newborn lives each year and are thus responsible for about one-third of the world's 4.0 million neonatal deaths. Until recently neonatal deaths, which now constitute about 40% of deaths in children under 5 years, were largely ignored as a global health concern. To reduce neonatal mortality by half between 2000- and 2015 and thus achieve the Millennium Development Goal for child survival, many challenges must be overcome. To mitigate newborn sepsis in the DR, MCHIP is working to: Continue quality improvement activities for prevention and treatment of newborn sepsis in the hospitals that participated in BASICS, Musa and Los Mina. Expand neonatal sepsis prevention and treatment to 2–3 new facilities that participate in the CoEx project. Obtain baseline information on rational use of use of antibiotics for newborn sepsis in participating CoEx hospitals. The project reached these objectives. The 2010 baseline study found deficient hand-washing practices, lack of cleaning supplies, and failure to correctly practice clean delivery. The MTE team concluded that, in the three hospitals where MCHIP operated (Musa-SPM, Los Mina-Santo Domingo and San Vicente de Paul Hospital-SFM), there is evidence that newborn care improved. In observing deliveries and attention to the newborn, the team concluded that care of the newborn met high standards. The Improvements observed in the delivery room and neonatal ward will help reduce the infection risk for mothers and newborns. Delivery room and neonatal 22 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 hygiene is appropriate and has the elements necessary to prevent nosocomial infections. The Antonio de Musa Hospital decreased nosocomial infections from 5% (2010) to 3% (March 2012), Los Mina Hospital from 37% (2011) to 30% (March 2012), and the San Vicente de Paul Hospital from 19% (2011) to 9% (March 2012). Priority Actions To ensure that this intervention is the effective and sustainable there is a need for action at different levels: Medical school curricula and hospital in-service training must cover the basic elements of preventing newborn infections. Nurse staff must have more training and be empowered to prevent and manage newborn infections. Hospital Infection control committees must enforce hygiene practices in all clinical areas; periodically audit consistency and adequacy of hand washing; insist that clean water, soap, and towel be available in all clinical areas; share monthly infection data with all clinical areas; and graph performance and post it on the wall of clinical units. Hospital administrators must be held accountable for placing sinks in all key clinical areas. (The MTE team observed nurseries and labor and delivery [L&D] units without sinks. This is absolutely non-negotiable for giving high quality care.) The neonatal ward should ensure that the supply of medicines is adequate; improve neonatal mortality audits; and take action to prevent and treat cases of neonatal sepsis. Laboratory services to the early detection of the organisms causing neonatal sepsis must be improved. Neonatal mortality audits must be made an integral component of the CoEx project. The national referral and counterreferral system needs to be improved. Actions that have proven effective in other hospitals should be replicated to continue reducing child mortality. New staff should be oriented to project strategies and neonatal care. Maternal and Neonatal Mortality Audits (high performance, partially sustainable) All hospitals audit maternal deaths and have a functioning maternal mortality and morbidity committee to reduce them. The percentage of audited maternal deaths has increased from 16% to 88%. The committees use the perinatal information system (SIP) as the basis for the audit. The government declared a “zero tolerance” policy for maternal mortality in 2006. The policy says “Zero tolerance does not mean that cases and deaths would not occur, but it means that cases and deaths will not be tolerated or go unnoticed for the society and the health system.” This policy had brought increased attention to the need to reduce maternal mortality due to obstetrical complications and maternal death audits have been introduced. The result is a dramatic decrease in maternal mortality, particularly in recent years. The drop has been even greater in project hospitals than in other hospitals. Any obstetric complication detected USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 23 immediately becomes a high priority; even the Minister of Health and the head of the Maternal Child Department would call a hospital if they learned that there was a “complicated or high￾risk case” as referred by hospital management teams. On the other hand, one MTE team noted that care during delivery varied widely from site to site and professional to professional. Nurses and medical staff interviewed were not familiar with the latest studies on excess maternal and infant morbidity and mortality due to overuse of cesarean sections, labor inductions, neonatal hypoxia, and other conditions. Although physicians recognized that their C-section rates were high (32%–49%), they could not articulate an approach to reducing them. Most of the nurses interviewed did not have a real sense of what a reasonable rate is, and most did not link postpartum early formula feeding to excess surgical interventions and condition of the mother postpartum. Interviews with skilled providers indicated that they did not always master or practice common skills for perinatal care. One obstetrician stated she cannot resuscitate a baby or manage a patient on magnesium sulfate. There seemed to be little understanding of how to manage assisted deliveries, which probably contributes to excess cesarean sections and deaths. This hospital had a high volume of deliveries per year and family practice residents completed their rotation on ob/gyn. The MTE team rated this intervention as “partially” on the sustainability axis and “high” on the performance axis. The audits have been carried out efficiently, but more work is needed to analyze the causes of maternal and infant deaths and morbidity and move to reduce them, as well as consolidate audits as routine procedures within each health establishment. Priority Actions Complement maternal death audits with supervision of the quality of maternal and infant services delivered. Kangaroo Mother Care (KMC) (high performance, partially sustainable) KMC is a special way of caring for low-birth-weight babies. It fosters their health and wellbeing by promoting effective thermal control, breastfeeding, infection prevention, and bonding: the baby is kept in continuous skin-to-skin contact by the mother and breastfed exclusively to the utmost extent possible. KMC is initiated in the hospital and continued at home. KMC has been shown to have a beneficial effect on breastfeeding and thermal control through prolonged skin-to-skin contact that reduces the risk of hypothermia. For stable babies, KMC is at least equivalent to conventional care with incubators in terms of safety and thermal protection.5 KMC was initiated more than 18 months ago at the San Franciso de Macoris Provincial Hospital (San Vicente De Paul Hospital) and about a month ago at the time this evaluation at Centro Materno Infantil San Lorenzo de Los Mina in Santo Domingo after a team (neonatologist, nurse, and psychologist) was sent to the Kangaroo Foundation in Colombia for training. The program in the former has undergone a positive evolution since. Initially data were not recorded and hospital support was tepid. Now data are carefully maintained and the service has become the darling of the hospital enjoying lots of publicity, earning a prize from the first lady, and creating an empowered, committed, problem-solving staff. 5 Kangaroo Mother Care Clinical Practice Guidelines. 24 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 In Los Minas the project does not yet have dedicated space. In spite of space and equipment shortages, however, the staff enthusiastically enrolled 33 babies the first month. They were delighted to report that none of these babies had needed to be re-hospitalized—the usual rate is 20 per cent. Enthusiasm is very high and clearly there are committed change agents working to make it happen. Priority Actions There is considerable evidence that this low-cost, high-benefit strategy decreases morbidity and mortality in preterm, low-birth-weight neonates born in resource-limited settings. Given the experience of the two hopitals where this intervention is currently functioning, for its proper replication, it is crucial to appoint a specific person as kangaroo project manager, and establish clear staff responsibilities and communication channels. It is recommended that CoEx integrate this strategy as part of its technical interventions and continue supporting MCHIP staff to scale up KMC to other hospitals. The intervention can be complemented by providing continuous education to staff at referral facilities. Staff from San Pedro de Macoris noted that some neonate deaths were due to lack of proper referral, e.g. newborns not kangarooed for transport arrived with severe hypothermia. Quality Antenatal Care (needs improvement, partially sustainable) In general, quality of antenatal care in the hospitals visited has increased since the project began, according to both health care providers and their patients. For example, the percentage of pregnant women that had weight, blood pressure, and uterine fundus height measured during the last visit reached 90%. However, the MTE team’s qualitative observations suggest that prenatal care is not managed in an integrated manner, missing opportunities to offer the complementary health care services required by pregnant women. One case observed during the evaluation was a user in mourning because her partner had died. The staff never approached her, nor was any effort made to refer her to a service that could provide appropriate mental health or emotional support care. Although there was a noticeable advance in the quality of antenatal care in program hospitals, it still needs to be improved to assure, for instance, compliance with national norms and protocols and to provide more integrated health care to pregnant women, mothers, their infants, and families in general. At the current level of functioning, the quality of antenatal care is only partially sustainable—further training, and above all, supervision are needed to assure quality. Priority Actions Widely disseminate the project/MOH quality of care supervision tools and support the Provincial Health Directorate to draft a plan to supervise all health facilities and provide training on-site. Safe Delivery and Emergency Obstetric Care (needs improvement, partially sustainable) Management teams reported that all hospitals have received training in manual vacuum aspiration, emergency obstetric care, management of the neonate, and biosecurity and that the national norms and protocols have positively impacted clinical management. The MTE team verified the existence of guides to monitor obstetric and neonatal emergency care, but only one SDP management team reported having applied them. However, all hospitals reported having specific indicators for M&E of maternal and infant case management. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 25 The MTE teams noted that care during delivery varied widely from site to site and professional to professional, and again nurses and medical staff were not aware of to latest findings on overuse of cesarean sections, labor inductions, neonatal hypoxia, and other conditions. In several facilities, providers reported that they place the baby on the mother’s abdomen until the cord stops pulsing, as established by national protocols, but group discussions with mothers users did not always corroborate these reports. Mothers who had delivered recently reported that the babies were brought to them usually within two hours after the delivery. On the other hand, even though, according to national norms, the nurses are not authorized to conduct deliveries by themselves, this seems to be common practice. The “clandestine” nature of this practice means that institutional deliveries are not accurately registered or supervised, and the nursing staff are not well trained for these functions. In one hospital, however, as promoted by other international agencies, nurses had received in-depth training for conducting deliveriess, with adequate support of the hospital authorities. The results were excellent for the hospital’s L&D services. This suggests that nursing personnel can be central to initiatives for improving the quality of MCH care in public hospitals in DR, where specialized medical doctors are not always available for all institutional deliveries. It was reported that nursing skills had been upgraded in only two hospitals, one through UPR in one case and the other through Emory University. Service users who participated in discussion groups expressed limited confidence in the capacity of hospital emergency services, mostly because service providers were unavailable in off hours, and waiting time was long. This suggests that there is a need to heighten both the quality of the emergency services and their perception in the communities, giving special attention to timely detection of and response to obstetrical emergencies. Priority Actions The MTE team considered that the emergency and disaster preparedness in hospitals visited needed improvements, mostly through clear delimitation of functions and responsibilities with the MOH, personnel training, and regular drills. As currently implemented this intervention was considered only partially sustainable because further external technical assistance and supervision are needed to consolidate emergency plans within hospitals. Active Management of the Third Stage of Labor (needs improvement, partially sustainable) The MTE team verified that AMTSL was implemented in all sites visited, although it was not done consistently. According to CoEx figures for FY 2010 and FY 2011, the number and percentage of women receiving MTSL increased from 32% to 82%. The user focus groups acknowledged this practice as a strength; a common statement was "they cleaned me so well that I barely bled after delivery.” In most establishments, however, AMTSL is conditioned on the presence of specialized personnel in the delivery room, which is not always possible, particularly not in lower-level facilities during weekends and night shifts. The MTE team could not obtain sufficient evidence about the use of national protocols for case management of maternal and infant care. Health personnel reported that they know the protocols exist, but they were not always able to refer to specific topics and content. Medical residents from different specialties did not recall being trained in using case management 26 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 protocols but reported they have heard about them during training about issues like management of obstetric emergencies, neonatal care, etc. The MTE team could not identify procedures for monitoring adherence to case management protocols or sanctions for noncompliance. These findings are consistent with previous studies conducted on compliance with DR case management protocols for MCH care in the country (Pérez-Then 2008; Pérez-Then et al. 2010.). Staff pointed out that the duty shift, when health staff analyze cases and recommend procedures to follow, is an opportunity to ensure compliance with standards and protocols. The evaluation team rated this intervention as partially sustainable and needing improvement in performance. Priority Actions Although there were visible improvements, there is still a need to universalize AMTSL and adherence to national protocols (see detailed MTE team observations, Annexes 14 –17). Prevention of Mother-to-Child Transmission of HIV and Other Interventions (needs improvement, partially sustainable) In general, the hospitals visited had little interdepartmental integration and communication, including management of clinical records. The National Program for Prevention of Vertical Transmission of HIV (PNPTV), for instance, operates with more autonomy than others and is seen as a world apart by providers, and sometimes hospital authorities. In this context, suppliers at one of the CoEx centers referred to the need for greater interdepartmental interaction at the hospital, to provide more comprehensive care to the client. Given its coverage, with nearly 100% of women in antenatal care accessing prevention of mother-to-child transmission (PMTCT) services at their first visit, this program offers a major opportunity for responding to the other health needs of pregnant women, including screening for other sexually transmitted infections (STIs), partner HIV/STI testing, and partner involvement in prenatal care. Except for one hospital where free rapid (same-day) VDRL testing was provided together with HIV testing, and another where pre-test HIV counseling was used to deliver other health-related information, most hospitals are missing these opportunities. Although rapid HIV tests are available in all PMTCT programs in the hospitals visited, in only two are test results delivered to pregnant women the same day, mostly due to logistical issues and lack of personnel. This is a serious weakness of the PMTCT services, since many women do not come back to pick up their results and approach the end of their pregnancy without knowing their HIV status. Priority Actions Provision of HIV test results on the same day; integration of the HIV program into the regular constellation of hospital services; and increasing male partner involvement and screening for other STIs should all be on the agenda. Humanization of Care / Delivery (needs improvement, partially sustainable) The MTE team believed there was variation in the understanding of hospital authorities and health care providers about the concept of humanization of care. It is is occasionally reduced to “not speaking harshly” to the women or calling them “mi amor.” Humanization of care includes USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 27 problem-solving, such as finding comfortable chairs for breastfeeding women, integrating fathers into care, flexible visiting hours, etc. Because it also has a rights and equity dimension, it needs empowered staff who see problem-solving as an integral and exciting part of their job. None of the hospitals visited uses analgesia in labor; it is actually available only in one for those cases providers might consider it is needed, assuming an anesthesiologist is available to administer it. The women in general ignore type of analgesia and actually identify pain during labor as positive (the more pain, the faster the delivery). The management team identified pain management as an issue the program should approach, through recognizing it could be complex given how MCH services currently function in DR and the financial and human resources limitations. They suggested introducing alternatives to analgesia for pain management in labor, such as psycho-prophylactic courses for pregnant women and involvement of doulas, already evidenced as positive experiences in two project hospitals. Nevertheless, analgesia in labor by epidural with an anesthesiologist present is not required. IM analgesics are well within the scope of prepared nurses. Having someone at the bedside giving attention and support has been found to equal administration of Demerol—another reason why families must be integrated into all areas of care. Having good labor support is the number one recommended intervention. IM and epidural analgesics, though important, should be regarded as second line. Women and health care providers had positive perceptions of the possibility of being accompanied by a female friend or relative during L&D, even though, due to infrastructure limitations, it would not yet be feasible to propose male partner involvement. One of the project hospitals has reported very positive experience with community leaders trained as doulas; they accompany women throughout the pregnancy, preparing them for labor and pain management, which, according to hospital authorities and others, has been a major factor in humanizing delivery in their facility. Project managers expressed concern about the feasibility of effectively impacting all dimensions of humanized care, given the complexity of this indicator and the limitations already noted. It was suggested that the CoEx team clearly delimit areas of impact and viable strategies that the project might introduced in the next phase, with realistic expectations about long-term outcomes. One strategy for which there was enthusiasm was empowerment of health care users about their health-related human rights and mechanisms for reclaiming them in different health system settings. Priority Actions Provide continuous in-service training and supervision to assure that health care providers and hospital authorities are familiar and comply with DR national norms and protocols established for MCH services. Consolidate maternal death audit procedures in hospitals and strengthen audits of infant, particularly neonatal, deaths. Disseminate the CoEx/MOH quality of care supervision tools and support the Provincial Health Directorate in drafting a plan to supervise all health facilities and provide on-site training. 28 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Provide training and attitude-change interventions so as to effectively transmit the concept of humanizing health care services and translate this concept into the day-to-day practice of health and support personnel. Among other interventions, nurses and other personnel might be specifically trained in psychological crisis management and enhancement of referrals within the same health facility so as to adequately respond to women’s needs from an integral perspective. Empower health care users by educating them about their health-related human rights and mechanisms for reclaiming them in all health system settings. Family-Centered Maternity Care (FCMC) (low performance, not sustainable) FCMC is characterized by 10 principles (Annex 19) and requires a fundamental shift from a professional-centered control model to a collaborative model in which the mother and her family become active partners in their health care during the childbearing experience. Services revolve around the needs and expectation of women and their families rather than those of the hospital staff (Phillips 2006). MCHIP has implemented this strategy at the Antonio Musa Hospital in San Pedro de Macoris, and the San Vicente de Paul Hospital in San Francisco Macoris. All 10 FCMC components could not be implemented in either. The main reason stated by health staff was that the intervention was not appropriate for the DR and consequently they did not make much effort to implement it. The MTE team concluded there were not enough aspects of this intervention to be observed, since it was not being fully implemented in any site visited. Nevertheless, the implementing partners believe some strategy components could be implemented in the DR with additional funding and personnel. BASICS and MCHIP experience showed that prenatal control, rooming-in, unrestricted access of parents to the neonatal ward, and skin-to-skin contact could be acceptable. Annex 19 shows recommendations as to how this strategy could be implemented in the second half of the CoEx project. Priority Actions This intervention has low priority and could be discontinued as a holistic approach to MCH health services in the second half of the project, except for principles shown to be feasible for application in the DR. Helping Babies Breathe Curriculum for Newborn Resuscitation The objective of HBB is to train birth attendants in developing countries in the skills essential to newborn resuscitation, with the goal of having at least one person who is skilled in neonatal resuscitation at the birth of every baby. HBB focuses on practices that all persons who care for babies at birth can.6 Iindicators of success with HBB are Master trainers trained. Technical assistance provided for implementation at scale and supervision of HBB in all 10 Centers of Excellence. The MTE team found that indicator was 100% completed in the three hospitals where HBB is being implemented (Musa, San Lorenzo de Los Mina, and San Vicente de Paul). Staff who had received training in these interventions clearly felt empowered. They referred to practical 6 See http://www.helpingbabiesbreathe.org/about.html. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 29 situations where the knowledge acquired was applied, e.g., an obstetrician resuscitated a newborn as trained because the pediatrician was not present. Priority Actions Given the experience of the three establishments where HBB is currently functioning, for replication, integrate BBB as part of CoEx technical interventions and continue support of MCHIP staff to scale up HBB to other hospitals. Provide continuous education to staff at all referral facilities. Staff from San Pedro de Macoris noted that some deaths of neonates were due to lack of proper referral. Biosafety (Environment Preserved) Previous evaluations of DR MCH services report serious limitations in compliance with basic biosecurity norms (Pérez-Then et al. 2008, 2010; Miric et al. 2006), due both to a lack of essential supplies, such as soap and water in delivery wards, and to bad habits and learned behaviors of health care providers. Some providers just assumed shortages of water and essential supplies in maternity areas was a reason for not washing hands before attending a delivery (Miric et al. 2006), which highlights the interdependence in health facilities between infrastructure and equipment certification and the biosecurity component. Those studies also reported poor waste management, which is a hazard for both the health of facility personnel and the local environment in general. Responding to this situation, the CoEx project integrated a component dedicated to preservation of the hospital environment, incorporating a biosecurity system for prevention and control of infections and negotiation of appropriate waste management regulation with local civil authorities. Among other interventions corresponding, project hospitals were expected to: Design and comply with a program for continual improvement of biosecurity measures, which would be constructed and periodically updated based on specific risks as they are identified. Adopt an integral program for management of waste and solid hospital residues, to preserve the environment. Introduce a structured program for prevention, detection, and control of health-care associated infections (HAIs). These interventions are interdependent with activities developed within the CoEx evidence￾based quality MCH care component (see above) and with such specific MCHIP interventions as newborn sepsis control, as a part of HAI prevention efforts. By the time of this evaluation, these aims had been or were being achieved in most of the hospitals; hospital authorities and health care providers attested to significant improvements in biosecurity measures. Limited access zones have been adequately marked with signs, and appropriate clothing and other expected biosecurity procedures are clearly visible. Sanitary flowcharts and waste management procedures were established and generally complied with in all hospitals visited. These procedures included appropriate clothing of the concierge and cleaning staff, and immunization of staff against Hepatitis B and tetanus. Likewise most hospitals have signed collaboration agreements with local civil authorities for management of hospital solid waste, which is now collected and deposited separately from household waste. Some 30 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 agreements have been extended to cover management of waste from nearby private health facilities, which suggests that the CoEx project is having an indirect impact on environmental preservation in DR beyond the 10 project hospitals. One of the CoEx strategies that has had major impact, according to by all stakeholders, including the project management team, has been training concierge and cleaning staff. They have become familiar with biosecurity measures in their facilities and empowered do something about hygiene in different hospital areas. MCH services users consistently reported this as the major improvement in the quality of those services over the past two years. Concierge staff who have been trained reported more job satisfaction and higher motivation to keep hospital areas clean and functioning well. Despite these advances, some of the indicators included in this CoEx component, exceed the range of action of this project and depend, on a much larger scale, on local and national policies for managing biological waste. One specific challenge the project management team highlighted was appropriate management of placentas, which due to their high contamination potential should be chemically deactivated before transportation and deposit with other waste. Despite the notable impact of CoEx biosecurity activities, the limitations in the waste management interventions, due mostly to factors beyond CoEx control, support the MTE team’s evaluation of this component as needing improvement. Considering the current success indicators for this component, it is not sustainable because the waste management interventions exceed the reach of the project. The currently successful efforts need further consolidation, such as bringing hospital authorities into implementation and monitoring of waste management operational plans. Priority Actions The CoEx team should clearly delimit areas of impact and success indicators for the rest the project, concentrating on areas that are clearly within the scope of project activities and have proved to be highly successful, such as definition of and compliance with biosecurity procedures, training of personnel, signalizing of hospital areas, and hospital waste management protocols. Bring hospital authorities and health care providers into adjustments, implementation, and compliance with biosecurity measures so as to further the positive impact of this component on the quality of MCH care. Periodically offer health care providers and support staff (concierges, etc.) refresher training in biosecurity procedures, to maintain their compliance and personal identification with the protocols. Combine the efforts of this CoEx component with other successful interventions for improving MCH care in project hospitals, such as newborn sepsis prevention and control, which are currently implemented by MCHIP (see section 4). USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 31 EVALUATION QUESTION 5 Has the Project targeted the most critical and appropriate activities, if not what opportunitywas missed? The project has chosen intervention, both systems strengthening and technical, based on self￾diagnosis and priority actions to improve maternal and infant health care; therefore, the mix of interventions and activities was appropriate for the DR. During visits to hospitals, the MTE team observed other projects that may bring new opportunities or synergy with the Centers of Excellence: Psicoprofilaxis de parto at Maternidad San Lorenzo de Los Mina is an intervention for women during first pregnancy. It includes both exercise and preparation for delivery and pain management strategies. It could offer opportunities for humanizing care. Programa de Madres Tutelares (Mentoring Mothers) is about designating tutors in the community (experienced, middle-aged women) who would accompany teenage pregnant women through prenatal care, delivery, and care of the newborn, including growth monitoring and immunizations. The Doulas program at Hospital San Vicente de Paul supports creation of labor companions or birth workers who are community leaders. Doulas provide nonmedical support to women and families during labor, childbirth, and postpartum, helping to humanize care and management of pain. Training nurses to attend deliveries, as Hospital San Vicente de Paul does. National health norms do not allow nonmedical personnel to attend deliveries, but many do so with the consent of the doctor in charge. Recognizing that reality, hospital management provided additional training to formalize the situation; this has been supported by Emory University, with excellent results for the quality and humanization of care. 32 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 33 V. CONCLUSIONS AND RECOMMENDATIONS This matrix summarizes the main MTE findings, with emphasis on actions that should be priorities, recommendations for future directions, and who should be responsible for the intervention. Since a single priority action may need more than one responsible entity, the cells are broken down horizontally to specify which entity would do what. Note that these are only recommendations. Project and MOH staff will ultimately decide on which recommendations are most appropriate and who should be responsible. Programmatic and Health Systems: What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? Strategy Implementation Issues/Constraints Priorities Recommended Actions Responsible Entity Basic Infrastructure Certified Since there is general agreement that hospital infrastructure has shown major improvements, the only issue is how to sustain the improvements. Medium: Keep equipment functioning and keep infrastructure renovated to evolving standards. Monitor how equipment and basic infrastructure are used and maintained. Set up a system for guiding hospitals and health care providers to ensure that recommendations for proper use of equipment are followed and that procedures for regular maintenance are followed. DPS and hospital management team with CoEx project support Customer Service System Records management needs improvement. Staff as well as users are confused about where to obtain information about specific services, where to ask questions, and where to present complaints. The privacy of suggestion boxes is not protected, and in any case most users of health care services are not well￾educated and may therefore not be comfortable using a suggestion box. High: Make clinical records available to all clinical departments. Establish a system for handling complaints. Ensure that all staff are trained in how use of clinical records can enhance patient care. Ensure that prenatal clinical records are available during and after delivery and in neonatology wards. Revamp technology related to clinical records to allow access for all clinical departments. Train care providers in systematic use of clinical data for evidence-based decision-making. Redefine the responsibilities of the customer service office. Promote and support hospital authorities and health care providers in monitoring and evaluating actualization of strategic and operational plans. DPS and hospital management team with CoEx project support 34 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Programmatic and Health Systems: What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? Strategy Implementation Issues/Constraints Priorities Recommended Actions Responsible Entity Not all departments have on￾site access to all records. Continue to transfer basic and advanced technical capabilities to hospital authorities and health care providers so that they can define the changes needed to meet their unique needs. In transferring knowledge consider involving the highly trained local personnel that participated in the CoEx project in the MOH structure. DPS and hospital management teams with CoEx project support Common Assessment Framework and Strategic Planning There were minor disconnects between departments and services. Not all relevant staff were involved in planning. There is no process for assessing the quality of operational plans and practices. Overlapping responsibilities could undermine efficiency. Strategic plans need improvement. Medium: Provide a structure for planning and evaluation. Assign responsibilities for coordinating the drafting of strategic and operational plans and monitoring their realization. Draft plans in terms of patient needs. Ensure that all departments affected are included in the planning. Offer facilities written guidance on the elements of a strategic plan and how to measure whether it is effective in practice. Address quality and integration of services in annual work plans and set up mechanisms for regular monitoring Hospitals, DPS, and SRS with CoEx project support Assess client satisfaction and offer opportunities for a wider constellation of services according to patients’ identified needs. DPS and hospitals with CoEx project support Medicines and Supplies System All hospitals have systems but they vary in quality. PROMESE_CAL, the government distributor, is failing to consistently provide the necessary supplies of medicines. Low: Although this is a national priority for the MOH, it is beyond the reach of the CoEx project. Continue supporting this intervention within hospital and regional structures, monitor the logistic system, and give PROMESE-CAL feedback to help solve the stock- out problem. SRS and DPS USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 35 Programmatic and Health Systems: What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? Strategy Implementation Issues/Constraints Priorities Recommended Actions Responsible Entity Hospital Management Information System The font used by Perinatal Information System (SIP) in some facilities was so small that many could not read the chart and had to write notes on the margins. Forms are too long and require too much information; In several hospitals, patient records were not routinely sent to L&D when a patient admission is an emergency. Clinical data collected through the SIGHO system is not used effectively or routinely employed as a basis for decision-making, strategic planning, and facility management. High: Ensure that formats are easy for providers to use. Assess data quality, from source documents through monthly reports. Set up procedurss for records to follow patients through the system. Revamp SIP documentation. Carry out audits of data quality: completion, timeliness, and consistency,among other characteristics. DPS and hospitals with CoEx project support Medium: Assess the routine information collected against such indicators as flow of data and patient records among hospital services, levels of data aggregation and use for decision￾making and strategic planning; and quality of services delivered. Promote use in emergency and delivery areas of such SIGHO data as prenatal records. Train and support MCH health care providers on the relevance, systematization, and utilization of SIGHO clinical data for decision making to help promote an “information culture within each hospital. DPS and hospitals with CoEx project support Monitoring the quality of services delivered. Set up a system to ensure that prenatal clinical records are available before, during, and after delivery and as in the neonatology wards. Hospital management teams with CoEx project support Safe Hospital for Emergencies and Disasters Most, but not all, hospitals visited had emergency plans. The MOH Department of Disasters and Emergencies insists that it alone is responsible for preparedness for emergency and disasters and no facility plans or interventions can be implemented without its direct supervision. If so, the effort and resources used for these activities might have more impact on the quality of MCH services if invested in other High: The project managers need to decide whether to invest significantly in this component to improve its sustainability and clarify responsibilities within the MOH, or limit efforts to technical support to hospital management on initiatives promoted by other groups. CoEx needs to decide in which direction it wishes to take this component. Hospitals, DPS, and regional directorates need to institutionalize this component by means of training and replicating it throughout other provincial or regional hospitals. CoEx; if the component is continued, SRS, DPS and hospital management teams 36 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Programmatic and Health Systems: What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? Strategy Implementation Issues/Constraints Priorities Recommended Actions Responsible Entity strategic components of CoEx. Maternal and Child Health Centers of Excellence Certification The questions about this component relate only to the Ministry of Health: Although documentation and tools are available to support certification, does the MOH have the capacity to certify Centers of Excellence? How can the MOH resist political pressures that might bias the certification process? High: The certification process is a high priority for the second half of the project, but much depends on the answers to questions about the MOH. However, the CoEx team can make some recommendations to the MOH Include Provincial Directorates in the pre￾certification authorization process; encourage the MOH to delegate this function to the provincial level. DPS with CoEx project support Support DPS authorities in monitoring and evaluating implementation of strategic and operational plans. DPS Publicize the results of the authorization and certification processes. DPS with CoEx project support Training Centers The CoEx project has helped to place well-equipped training centers in each target facility; drafted training curricula and job aids; and trained trainers. However, regional and DPS directorates believe it will be difficult to replicate the model because health services have not yet been truly decentralized. Hospital management teams recommend that replicating the model should be the responsibility of the SRS. High: CoEx needs to decide whether to continue to pursue replication of this model generally or pursue partial replication based on successful experiences and best practices accumulated to date. Training curricula and annual training plans to replicate the CoEx approach are needed. Training of trainers is needed to develop a cadre of regional and provincial faculty by specialty to support training. Promote gradual transformation of hospital training centers to also cover: Continuous education; research and critical analysis of recent MCH scientific information and evidence-based medicine through, e.g., journal clubs or discussion forums. Systematization of data produced by hospital services, clinical and community-based research, and scientific publications; SRS with CoEx project support Communities Participating From qualitative interviews and direct observations, it appears that community involvement is perceived as merely formal fulfillment of a project requirement rather than a real functional alliance. High: CoEx administration should decide whether to reinforce and significantly amplify interventions in this component in order to achieve authentic community participation, or limit efforts to continuing formal involvement and participation of community representatives in Hospitals, DPS, and regional directorates can use training to institutionalize this component and replicate it in other hospitals. SRS, DPS, and hospital management teams USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 37 Programmatic and Health Systems: What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? Strategy Implementation Issues/Constraints Priorities Recommended Actions Responsible Entity hospital administration councils and work on other components to improve the quality of MCH care more directly. The Referrals System There are no well-defined systems for referrals and counter-referrals in project hospitals: Patients are “just sent” to a referral site without a note or clinical record. Lack of referral systems reduces chances for timely response of the health system to patient needs, including follow up and monitoring of treatment outcomes. High: The CoEx administration needs to decide whether to continue investing efforts and resources into this project component. Analyze interventions to set prioritis and strategies required by a functioning referral system. Strengthen direct communication between directors of hospital ob/gyn departments with other facilities that refer patients to it, in order, e.g., to promote an effective response to obstetric emergencies. Exploit technology such as videoconferencing, for clinical exchanges and advice. SRS, DPS, and hospital management teams Technical Interventions: What technical interventions have been the most/least successful in ensuring the achievement of the project results and Why? What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What technical interventions appear to be most/least sustainable and why? Strategy Implementation Issues/Constraints Prioritization Recommended Actions Responsible Entity Prevention and Treatment of Newborn Sepsis Although there were visible improvements; the intervention was not consistently implemented in all project hospitals. Handwashing and other practices are deficient, and there is a shortage of cleaning supplies. Many providers are not practicing clean delivery. High: Prevention and treatment of newborn sepsis needs to be institutionalized within the MOH by replicating the successful CoEx experience. Medical school curricula and hospital in-service training must cover the basics of preventing newborn infection. MOH, Ministry of Education, and hospital management teams Nursing staff should be trained and enabled to move to prevent and manage newborn infection. Hospital infection control committees must enforce hygiene practices in all areas. DPS and hospital management teams with CoEx project support Improve referrals within the national SRS, DPS, and 38 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Technical Interventions: What technical interventions have been the most/least successful in ensuring the achievement of the project results and Why? What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What technical interventions appear to be most/least sustainable and why? Strategy Implementation Issues/Constraints Prioritization Recommended Actions Responsible Entity health system. hospital management teams Ensure that neonatal wards have an adequate supply of medicines; improve neonatal mortality audits; and move to prevent and treat neonatal sepsis. Improve laboratory services for early detection of organisms causing sepsis. DPS and hospital management teams with CoEx project support Make neonatal mortality audits an integral component of the Centers of Excellence project. DPS and hospital management teams Replicate actions that have proven effective elsewhere to reduce child mortality SRS Orient new staff on project strategies and neonatal care. DPS and hospital management teams with CoEx project support Maternal and Neonatal Mortality Audits. Neonatal mortality audits are not conducted regularly. CoEx project indicators for 2011 reported no neonatal death audits by the safe motherhood committees. Medium: Consolidate maternal death audit procedures in the hospitals, and strengthen audits of infant, particularly, neonatal, deaths Monitor the quality of service delivery and adherence to hygiene and clinical norms Transfer CoEx tools and methodology to the SDPs to monitor adherence to national protocols. DPS and hospital management teams Kangaroo Mother Care Enthusiasm for this component is high and there are committed change agents working to make it happen. The only issue is how to make it more effective. Medium: Support institutionalization of Kangaroo Mother Care in the MOH by replicating the successful CoEx experience. Appoint a specific person as Kangaroo Project manager. Clarify responsibilities and communication channels among staff. Make this strategy part of CoEx technical interventions. Continue to support MCHIP staff to scale this up to other hospitals. DPS and hospital management teams with CoEx project support Keep staff informed about all referral DPS and hospital USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 39 Technical Interventions: What technical interventions have been the most/least successful in ensuring the achievement of the project results and Why? What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What technical interventions appear to be most/least sustainable and why? Strategy Implementation Issues/Constraints Prioritization Recommended Actions Responsible Entity facilities. management teams with CoEx project support Quality Antenatal Care: Risk Focus and HIV/AIDS Management Because prenatal care is not managed in an integral manner, opportunities are missed to offer complementary care by pregnant women. Because HIV test results are not delivered to pregnant women on the same day, they must come back to the hospital two or even three times to receive counseling, schedule a lab appointment, and get test results. The National PMTCT Program operates with more autonomy than others and is seen as somewhat remote by providers and sometimes hospital authorities. High: Monitor the quality of service delivery overall. Medium: Integrate other health care opportunities into prenatal care and streamline processing of lab tests. Disseminate the quality of care supervision tools developed by the project and the MOH. Support the Provincial Health Directorate in drafting a plan to supervise all health facilities and provide additional training on-site Expand maternal death audits to more comprehensively monitor management of obstetrical complications and use clinical rounds to discuss evidence-based case management. Expand audits to neonatal deaths and case management, Introduce MCHIP priorities, such as helping Babies Breath” and “Kangaroo Mother Care.” DPS and hospitals management teams with CoEx project support Safe Delivery and Emergency Obstetric Care. Care during delivery varied widely from site to site and provider to provider: Nurses and medical staff were not familiar with new research on the relation of maternal and infant morbidity and mortality to overuse of cesarean sections, labor inductions, and neonatal hypoxia, etc. . Physicians recognized that their C-section rates were high but did not articulate an approach to reducing them. Most nurses interviewed did High: CoEx administration should decide whether to continue investing efforts and resources in this component so as to significantly improve its performance and sustainability and clearly delimit functions and responsibilities within MOH, or limit these efforts to technical support to hospital authorities for initiatives promoted by other entities and give more attention to other components that directly improve the quality of MCH care Use improved tools for monitoring and evaluation. Institutionalize standard protocols for evidence-based case management, with close monitoring by the MOH. Reinforce clinical rounds and literature review and,link them to real cases and audits, to keep all health care staff current on advances in MCH care., SRS, DPS, and hospital management teams 40 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Technical Interventions: What technical interventions have been the most/least successful in ensuring the achievement of the project results and Why? What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What technical interventions appear to be most/least sustainable and why? Strategy Implementation Issues/Constraints Prioritization Recommended Actions Responsible Entity not realize what was a reasonable rate and did not link postpartum early formula feeding to excess surgical interventions and condition of the mother postpartum. Knowledge of how to manage assisted deliveries seemed weak, and probably contributes to excess cesarean sections. Although nurses are not authorized to conduct deliveries, it is a common practice. This clandestine practice does not allow accurate registry and supervision of deliveries; nor are nurses properly trained for these functions. User discussion groups had problems with service providers not being available at off hours and long waiting times. Active Management of the Third Stage of Labor (AMTSL) Although AMTSL is operational in all the sites visited, its application is not consistent. It is often conditioned on the presence of specialized personnel in the delivery room, which is not always possible, especially during weekends and night shifts. Medium: Disseminate the quality of care supervision tools developed by CoEx and the MOH, and support the Provincial Health Directorate in drafting a plan to supervise all health facilities and to provide training on-site. Make supervising the quality of service delivery a priority for the second half of the project and for SDP staff. Reinforce M&E with improved tools and use of standard protocols for evidence-based case management and ensure close monitoring by the MOH. Reinforce clinical rounds and literature review and,link them to real cases and audits, to keep all health care staff current on advances in MCH care. DPS and hospital management teams with CoEx project support USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 41 Technical Interventions: What technical interventions have been the most/least successful in ensuring the achievement of the project results and Why? What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What technical interventions appear to be most/least sustainable and why? Strategy Implementation Issues/Constraints Prioritization Recommended Actions Responsible Entity Humanization of Care and Delivery There was a lack of a common understanding among hospital authorities and health care providers about what humanization of care meant. None of the visited hospitals uses analgesia in labor. Women in general consider pain during labor as normal. The management team identified pain management issues to be addressed by the CoEx. Project managers were concerned about whether full￾blown humanization of care was feasible given its complexity and the limitations imposed by the health structure and functionality. Medium: Disseminate the quality of care supervision tools developed by CoEx and MOH, and support the Provincial Health Directorate in drafting a plan to supervise all health facilities and provide training on-site Provide training and attitude change interventions to effectively transmit the concept of humanization of health care services and translate this concept into everyday practice of health and support personnel in the hospitals. Train the nurses and other personnel in psychological crisis management and enhancement of inter-consultation referrals within the same facility in order to adequately respond to women’s needs from an integral perspective. DPS and hospital management teams with CoEx project support Community-based education and patients’ orientation during consultation visits Empower health care users by educating them about their health￾related human rights and mechanisms for reclaiming them in all health system settings. DPS and hospital management teams with CoEx project support Family Centered Maternity Care MCHIP has tested this strategy in two hospitals, but it could not be implemented due to cultural barriers and health staff attitudes to some of the strategy components. Low: Replicate only the components that have proven to be culturally appropriate for the DR, and improve the delivery of quality maternal and neonatal services. Consider discontinuing this as a holistic approach Among strategy components that could be implemented in the DR with additional funding and personnel, are prenatal control; rooming-in; unrestricted access of parents to the neonatal ward, and skin-to-skin contact. SRS Helping Babies Breathe Curriculum for Newborn Resuscitation Although this intervention reached an optimal level of performance in three hospitals, there is no action plan to replicate the experience Low: Support institutionalization of the HBB Curriculum for Newborn Resuscitation in the MOH by replicating the successful experience through CoEx. Integrate this strategy into CoEx technical interventions. Continue to support the MCHIP staff in scaling it up to other hospitals. DPS and hospital management teams with CoEx project support 42 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Technical Interventions: What technical interventions have been the most/least successful in ensuring the achievement of the project results and Why? What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? What technical interventions appear to be most/least sustainable and why? Strategy Implementation Issues/Constraints Prioritization Recommended Actions Responsible Entity Biosafety (Environment Preserved) There have been major advances in this area, but some aspects of this component exceed the range of action of the hospitals, such as appropriate management as solid waste of placentas, which have high contamination potential. Medium: Clearly demarcate areas of impact and success indicators for this component for the remainder of the project, concentrating on areas that are clearly within project reach. Continue training personnel, signalizing hospital areas, and introducing hospital waste management protocols. Consolidate the positive impact of this component on the quality of maternal and child health care. Provide refresher training to health care providers and support staff (concierges, etc.) in the biosecurity measures and procedures to maintain and strengthen compliance and personal identification with protocols. Combine the efforts of this component with other interventions, such as newborn sepsis prevention and control, currently implemented by MCHIP. DPS and hospital management teams with CoEx project support USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 43 VI. FUTURE DIRECTIONS All the people interviewed, without exception, agreed that the project had caused profound changes that they had never believed possible. It changed a paradigm that quality is a long and complex process by using tools already available and taking hospital teams through a systematic process of self-assessment, devising solutions within their capabilities, and setting realistic goals and deadlines. Project field managers worked closely with hospital teams, with an attitude of learning from each other. The hospital, DPS, and SRS management teams expect that the interventions they have learned will continue to stay with no change and will transition to the next administration. However, they also recognize that their formation is not yet complete and they need all the tools used by the project to be adopted and institutionalized by different units and departments of the health system. People interviewed identified the health regions as the most suitable level for transmitting the processes necessary to become a Center of Excellence. The MTE teams perceived that positive attitude from the moment they arrived at a site visited. Criticisms raised earlier were intended to improve project performance, but by no means do they underrate the excellent work already accomplished. The following more general suggestions may help decision-makers to build on project successes in the time remaining: 1. The project has made real progressed in improving administration and clinical care areas of the 10 hospitals and in training administrative personnel. Perhaps, however, in this next project stage more resources should be directed to quality of health care and management action to transfer competency. In other words, more clinicians as well as specialists in obstetrics and neonatology should be hired to help field coordinators with teaching-learning process until the certification process is reached. In the next project stages, field coordinators should emphasize management training for regional directors, where further enhancement is necessary to transmit competencies. 2. To complete transmittal of project competencies throughout the DR health system, continued management training is needed that could launch the process of identifying breaches as well as limitations and benefits of this process with fieldwork coordinators and regional directors. It might identify components that are absent and quantify the time it takes to adopt activities or projects that have been evaluated as feasible to incorporate into the DR health system. 3. One option would be to initiate the process of transferring competencies in one or two regions and later expand it to all 10 hospitals selected as Centers of Excellence. 4. Experience to date suggests that the managers of the two projects being evaluated might consider further orientation about management and direct services provided to users to modify indicators of project success in at least six main categories: a. Implement the process of transferring competencies that seem to be feasible in the three regional health areas and the three DPS currently within the project. This process should begin as soon as possible in order to test the certification process. The principal objective is to train regional directors in process management and M&E of activities related to maternal-child health. 44 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 b. Provide training by ob/gyn specialists in direct service to users, including supervision of personnel for compliance with standards for antenatal, intrapartum, and postpartum care. c. Provide training by specialists in pediatrics and neonatology in direct care of children, including supervision of personnel for compliance with standards for child care and use of sustainable strategies to reduce child mortality (kangaroo mother care, neonatal respiratory assistance, prevention of neonatal sepsis, etc.). d. Involve the general director of the CoEx project in empowering middle-level MOH managers to draft protocol standards to help standardize hospital financing, quality control, M&E, and forming teams to replicate activities that have demonstrated success in reducing maternal-child mortality. This would also sustain the project’s momentum and demonstrate its value to the incoming administration. e. Identify opportunities within the health system for enhancing actions related to reducing maternal-child mortality, such as (1) training for nurses in completing deliveries with the MOH authorization; (2) adoption of programs to prevent, e.g., perinatal transmission of HIV and congenital syphilis to improve the quality of life for mother and child (e.g., male involvement); and (3) building strategic alliances with community-based organizations to promote programs to prevent morbid events (e.g., breastfeeding, vitamin A supplies, administration of oral rehydration salts); follow-up (prenatal care); and case referral (for maternal hypertension or severe dehydration of the newborn). f. Establish a process for certifying hospitals as Centers of Excellence—a high priority for the second half of the project. This intervention, which can become a program of its own, would set the course for improving the quality of care not only at project hospitals but also across the health system. The current MOH’s Authorization Department envisions certification as the next step for the MOH n its way to an accreditation system. g. Senior MOH staff interviewed all agreed that this is something that would be in the interests of DR; however, there was no clarity about how exactly to get there. Senior staff mentioned some possible difficulties, such as politicization of the process; and not enough resources to manage certification engage hospitals; or create an incentive system. Nevertheless, senior officers admitted that the project had made major advances toward a certification process for maternal-infant care. This is an experience that would create high expectations among central decision-makers. The MTE team believes that the project managers should take into account the following when the MOH is considering whether to establish the certification process: 1. Certification/accreditation in health cannot be put together quickly. It requires both political will and institutional maturity. 2. It is important to carefully project the costs of operationalizing technical management and evaluation. 3. The director of the accreditation system would not be the head of any government structures. The government should supervise but not intervene. 4. Demand for standards will rise gradually. 5. Other interested institutions should have a voice. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 45 6. Mechanisms to oversee the certification process should involve the government but be protected from local partisan politics. This might best be done by a private organization or a private-public partnership with a board of independent directors. Academic and research institutions and the private health sector should have a role. 7. It would be useful to create an incentive mechanism that would go beyond the simple reputation of being certified as a Center of Excellence; perhaps financial or in-kind incentives might be offered to the health staff or the hospital. 46 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 47 ANNEX 1. SCOPE OF WORK Global Health Technical Assistance Bridge Project GH Tech Contract No. AID-OAA-C-12-00004 SCOPE OF WORK: (Final 02-08-12) I. TITLE: USAID/DOMINICAN REPUBLIC (DR): MATERNAL CHILD HEALTH EVALUATION Contract: Global Health Technical Assistance Bridge Project (GH Tech) II. PERFORMANCE PERIOD Evaluation preparations should begin in mid-February 2012 depending on the availability of the selected consultants. Work is to be carried out over a period of approximately six weeks, beginning on or about (o/a) mid-February, 2012, with field work completed in late March 2012 and final report and close out concluding by early May 2012. With the Dominican presidential elections scheduled for May 20, 2012, it is critical that the evaluation be implemented o/a this timeline. III. FUNDING SOURCE Mission-funded IV. PURPOSE OF ASSIGNMENT The overall aim of the evaluation is to assess the performance of USAID/Dominican Republic (DR) in achieving “Increased access to and delivery of quality maternal and child health care services in selected areas.” USAID is implementing activities through the Maternal and Child Health (MCH) Centers of Excellence project (2009-14) and the Maternal and Child Health Integrated Program (MCHIP) (2010-12 USAID/DR) to achieve this intermediate result. V. BACKGROUND Health indicators demonstrate the inability of the health system to adequately address the health needs of the population, particularly the most vulnerable. According to the 2007 Demographic and Health Survey (DHS), maternal mortality remained high (159 per 100,000 live births) despite the fact that 98% of all deliveries take place in a hospital setting. Additionally adolescent pregnancy remains high at 23%. Infant mortality is 31 per 1,000 live births, and neonatal mortality accounts for 68% of total infant mortality (22 per 1,000 live births). Vaccination rates have improved but continue to be low (72.6% for DPT3 and 64.8% for polio). HIV prevalence among pregnant women is 2.3% with specific areas where prevalence is as high as 8.2% (Elias Pina) and 8.4% (Monte Cristi). The Dominican Republic is one of the countries in the continent with the highest tuberculosis incidence rates at 80-90 new cases/100,000 inhabitants. The geographical location and climate conditions put the Dominican Republic in a vulnerable position in terms of hurricanes and tropical storms, which cause major damage and create significant public health issues. 48 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Project Approach and Implementation Project Approach To achieve the project intermediate and subintermediate results highlighted above, USAID is leveraging two complementary implementing mechanisms, each with its own approach. Maternal and Child Health Centers of Excellence The overall goal of the MCH Centers of Excellence project is to contribute to reducing maternal, newborn, and infant morbidity and mortality in the Dominican Republic. The project will reach this goal by achieving the following objective: Improve equitable access to quality maternal and child health services focusing on critical interventions and management improvements that enhance quality and efficiency. To achieve this goal, the following intermediate results1 were identified: Ten hospitals developed as Centers of Excellence Three Provincial Health Directorates (DPS) developed as Centers of Excellence Regional Health Service Networks strengthened Region V technical interventions consolidated The project is being implemented in three phases: preparatory, implementation, and consolidation. During the preparatory phase the project provided trainings on communication, team building, and change management; implemented a Quality Management training using the Common Assessment Framework (CAF) tool; supported sites to develop improvement plans; activated hospital administration councils; and initiated the hospital certification process. During the implementation phase the project implemented interventions (e.g. CAF, Hospital Management Information System, Strategic Planning, Perinatal Information System, Biosafety and Infection Control System, Customer Service System, Referral and Counter-referral System, Medicines and Supplies System, Habilitation of Basic Infrastructure) to enable sites to become certified as Centers of Excellence; developed the certification criteria; and began the dissemination of lessons learned and best practices to surrounding facilities. The final consolidation phase will focus on sustainability, institutionalization, and documentation. Maternal and Child Health Centers of Excellence Contract No: GHS-I-00-07-0003-00 Project Dates: February 3, 2009 – February 2, 2014 Project Funding: $15,555,856 (Original: $14,825,814.00) Implementing Organization: Abt Associates (Prime), Subcontractors: Sistemas y Procedimientos WP, S.A. (SISPROSA-Local), Instituto Tecnológico de Santo Domingo (INTEC￾Local), El Centro de Estudios Sociales y Demográficos (CESDEM-Local), and Cultural Practice Contracting Officer’s Technical Representative (COTR): Sarah Majerowicz 1 More detail about the intermediate results can be found in Annex A. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 49 Maternal and Child Health Integrated Program The goal of USAID’s Maternal and Child Health Integrated Program (MCHIP) is to scale up evidence-based, high-impact maternal, newborn, and child health (MNCH) interventions toward reductions in maternal and child mortality. Building upon the experiences and successes described above, the newborn team at MCHIP submitted a plan to collaborate and provide technical support to the USAID’s new Maternal and Child Centers of Excellence Project in the Dominican Republic in the area of newborn health. The objectives and activities below are aligned with some of the characteristics and expected results of the Maternal and Child Centers of Excellence project. Objective 1: Scale up the intervention for quality improvement of prevention and treatment of newborn sepsis in the Maternal and Child Centers of Excellence as part of the regional strategy to improve newborn health to 2-3 additional facilities (for a total of 4-5 facilities) Objective 2: Strengthen the implementation of Family Centered Maternity and Kangaroo Mother Care Strategies in Centers of Excellence with trained staff; initiate expansion efforts to at least one additional center for a total of at least 3-4 facilities Objective 3: Provide technical assistance to the GODR for the strengthening of the newborn health national work plan in line with the Latin America and Caribbean (LAC) Neonatal Alliance Regional Strategy and Action Objective 4: Implement the "Helping Babies Breathe" (HBB) Curriculum for newborn resuscitation in all Centers of Excellence – Maternal and Child Health Integrated Program (MCHIP) – Contract No: GHS-A-08-0002-000 (Field Support) – Project Dates: April 1, 2010 – March 31, 2012) – Project Funding: $650,000 – Implementing Organization: Jhpiego (Prime), PATH, John Snow, Inc. (JSI), Save the Children, Johns Hopkins University-Institute for International Programs (JHU-IIP), Broad Branch, PSI, and Macro International. – Activity Manager: Damani Goldstein Major Changes and Project Modifications Hispaniola Cholera Outbreak Cholera emerged in the Dominican Republic in November 2010. While the overall response has been adequate, case fatality rates remain unacceptably high—at 1.7% they are well above the international standard of 1.0%. Poor quality of care and weak primary healthcare have been important contributors to these high mortality figures—the same issues that have plagued maternal and child health for the past few decades. The emergence of cholera shifted the focus of the Ministry of Health and slowed project implementation, specifically affecting many of the activities scheduled for the 3rd quarter of FY2011. While there were no specific modifications 50 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 made to the contract, USAID encouraged its implementers to look for synergistic opportunities which would both improve maternal and child health as well as bolster the cholera response. Recent Reductions in Maternal and Infant Mortality Compared to the previous year, 2011 saw dramatic reductions in both maternal and infant mortality, which have declined by 23 percent and 25 percent2 respectively. The ten hospitals supported by the USAID project were substantial contributors to this reduction. There is an ongoing study to better understand the reasons behind these recent reductions. The study will be completed before initiation of the evaluation and it is expected that the evaluation team will utilize the findings to inform the report. VI. SCOPE OF WORK Evaluation Objectives This external evaluation comes at the chronological mid-point of the Maternal and Child Health Centers of Excellence project. It is a mid-term, formative evaluation whose objectives are to help determine what components and project aspects are working well and why, which perhaps are not and why, and to make modifications and mid-course corrections, if necessary, to help guide the Maternal and Child Health Centers of Excellence project over its second half. Examined should be the sustainability of each of these aspects given that Maternal and Child Health funding for the Dominican Republic is on a downward trajectory and it is necessary to plan an orderly phase-out and handover to the Government of the Dominican Republic (GODR). The evaluation should provide pertinent information, findings, and recommendations to inform USAID, its Implementing Partners, the GODR, and other development partners on what is being accomplished regarding health systems strengthening and improving maternal, newborn, and infant health, as well as what relevant management, financial, and cost efficiencies are revealed. In summary, the evaluation will help all involved to better understand the initial results and contributions of the project, and where needed re-focus and strengthen it. Audience and Intended Uses The audience for the evaluation report will be the USAID/Dominican Republic Mission, specifically the health team, and the implementing partner. A report with an Executive Summary and recommendations will be provided to the GODR and relevant development partners. USAID/Dominican Republic will use the report to inform its phase-out plan by prioritizing aspects of the program according to performance and potential for sustainability; Abt Associates and its subcontractors will learn about their strengths and weaknesses and adjust their programs accordingly; the GODR will learn more about what is working and what is not in order to improve maternal, newborn, and infant health; and the relevant development partners will use the findings to identify priority interventions for scale-up. Evaluation Questions The evaluators will identify and explain the following: 1. What programmatic aspects have been the most/least successful in ensuring the achievement of the project results and why? 2 It is estimated that over 60% of infant mortality is neonatal; hence the focus on newborn health through MCHIP. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 51 2. What technical interventions have been the most/least successful in ensuring the achievement of the project results and why? 3. What programmatic aspects appear to be the most/least sustainable and why? 4. What technical interventions appear to be the most/least sustainable and why? 5. Has the project targeted the most critical and appropriate activities? if not, what opportunity was missed? Evaluation questions will be finalized during the team planning meeting and will be provided to USAID/DR with the team’s evaluation framework for review and approval. It is expected that in addition to a narrative that the evaluators will utilize the matrix outlined below to depict both the performance and sustainability of each programmatic aspect and technical intervention. In order to better focus the evaluation questions USAID/DR has developed an illustrative list of programmatic aspects and technical interventions that are referenced in the questions. Performance Sustainability Prioritization Matrix Programmatic Aspects Technical Interventions Change Management Humanization of Care Common Assessment Framework Quality Antenatal Care Visits Hospital Management Information System Perinatal Information System (SIP) Strategic Planning Emergency Obstetric Care Biosafety and Infection Control System Active Management of the Third Stage of Labor Customer Service System Emergency Obstetric Care Referral and Counter-referral System Helping Babies Breathe Medicines and Supplies System Kangaroo Mother Care Habilitation of Basic Infrastructure Family Centered Care Emergency and Disaster Preparedness Prevention of Newborn Sepsis 52 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Programmatic Aspects Technical Interventions Community Participation Post-Abortion/Partum Care Replication Model Maternal and Neonatal Mortality Audits Certification Prevention of Mother to Child Transmission VII. METHODOLOGY The final methodology will be developed by the team once the evaluation questions have been finalized and in collaboration with USAID/DR prior to any in-country evaluation work. Document and Literature Review: USAID/DR will provide background documents to GH Tech for distribution to the evaluation team at least one week prior to the inception of the in-country work. The documents will include but are not limited to the MCH Centers for Excellence and MCHIP proposal documents, project reports, training curricula, technical documents, USAID/DR MCH strategy- and planning-related documents, GODR background reports, and evaluation studies and reports from other countries with similar interventions. Team planning meeting (TPM): The assignment work will commence with a two-day TPM. This meeting will allow the evaluators to meet with the USAID/DR staff to be briefed on the assignment. It will also allow USAID/DR to present the team with the purpose, expectations, and agenda of the assignment. In addition, the team will clarify team members’ roles and responsibilities; review and develop final evaluation questions; review and finalize the assignment timeline and site visit calendar and share with USAID; develop data collection methods, instruments, tools, guidelines, and analysis; review and clarify any logistical and administrative procedures for the assignment; establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion; develop a preliminary draft outline of the team’s report; and assign responsibilities for the final report. Interviews and focus groups: Key informants at GODR, regional health authorities, partner organizations, beneficiaries, and other key stakeholders. Surveys and site visits: Convenience sample of Centers for Excellence (hospitals, health facilities, DPS), health facility staff, hospital boards, and community members. Preliminary and final debriefings: Participatory review of findings and discussion of recommendations to inform USAID, implementing partners, GODR, and other development partners. USAID/DR will provide a detailed contact list of key informants, focus group participants, and list of Centers for Excellence locations and key points of contact to the consultants during the document review period, so that appointments, interviews, and site visits can be set up for the team’s arrival in-country. USAID/DR will also provide a draft schedule for field visits including duration of stay at various sites to inform the team’s time in-country. VIII. TEAM COMPOSITION, SKILLS, AND LEVEL OF EFFORT (LOE) GH Tech will identify and hire an evaluation team composed of a team leader, three local experts, and a local logistics coordinator. The team leader will be responsible for team coordination and performance, and for ensuring the timeliness and quality of deliverables. USAID may also propose representatives from USAID/Washington (1-2), USAID/DR (3-4), and USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 53 MOH (1-2) to participate in parts of the evaluation and/or travel with the consultant team to site visits, specific event participation to be determined in coordination with the team leader. The evaluation team leader may ask the USAID and/or MOH staff person(s) to recuse themselves from some key informant interviews when the topic is USAID program management or other sensitive administrative topics. All attempts will be made for the team to be comprised of a representative number of male and female members. 1. Team Leader/Senior Evaluation Specialist (1) should have a postgraduate degree in health or an applicable social sciences field. S/he should have extensive experience in conducting qualitative evaluations/ assessments in Latin America and the Caribbean and strong familiarity with health systems strengthening and maternal and child health. Excellent oral and written skills in English and Spanish are required. The Team Leader should also have experience in leading evaluation teams and preparing high-quality documents. 2. The Team Leader will take specific responsibility for assessing and analyzing the organization‘s progress toward targets, factors for such performance, and benefits/impact of the strategies, and compare with other possible options. S/he will also suggest ways of improving the present performance, if any. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high-quality document. S/he will write the final report. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Dominican Republic team and other major partners. 3. Health Systems Technical Advisor (1) should have a postgraduate degree in public health or related subject. S/he should have several years’ experience with health systems strengthening and institutional development in the Dominican Republic. S/he should be knowledgeable in program assessment and evaluation methodologies. S/he should have extensive experience, and demonstrate state-of-the-art knowledge, in conducting qualitative evaluations/assessments around improving capacity for service delivery. 4. Maternal and Child Health Technical Advisors (2) should have a degree in public health, nursing, or a related subject. S/he should have several years’ experience with maternal and child health in the Dominican Republic. S/he should be knowledgeable in program assessment and evaluation methodologies. S/he should have experience, and demonstrate state-of-the-art knowledge, in conducting qualitative evaluations/assessments around improving capacity for service delivery. 5. Local Logistics Coordinator (1) will join the team on a part-time basis to provide support to schedule stakeholder meetings, key informant interviews, and focus group discussions; and to organize field visits. Required qualifications include: – Minimum 6 years of progressively responsible experience within GODR and/or NGO work settings handling complex logistics, such as coordinating business travel and meetings. – Demonstrated ability to be resourceful and to successfully execute complex logistical coordination; ability to multitask, work well in stressful environments, and perform tasks independently with minimal supervision. 54 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 – Ability to work collaboratively with a range of professional counterparts at all levels, including those from host country governmental and nongovernmental organizations, U.S. Government agencies and other donors. – Capacity for effective time management and flexibility. – Ability to interact effectively with a broad range of internal and external partners, including international organizations, host country government officials, and NGO counterparts. – Fluency in English and Spanish. – Proven ability to communicate clearly, concisely, and effectively both orally and in writing. As a lesson learned from previous evaluations, the mid-term evaluation needs to be carried out in a participatory fashion, forming a team that, in various places and times, includes a range of GODR staff, partner agency staff, and stakeholders. The process and findings are expected to enable USAID and its partners to clearly and easily evaluate the quality of programming over the last few years. Level of Effort (LOE) An illustrative table of the LOE is found below. Dates may be modified based on availability of consultants and key stakeholders, and amount of time needed for field work. Activity Team Leader Sr. MCH Advisor HS and MCH Experts Local Logistics Coord. Period of Performance (illustrative, depending on start date) Mission sends background documents to GH Tech and team members - - - Mid-February Review documents and begin drafting evaluation protocol and survey instruments; begin logistical arrangement. 4 3 2 Mid-February Team planning conference call with USAID/DR; modify protocol and tools according to discussion 1 1 - Mid-February Interview scheduling - - 5 Mid-February Travel to country 1 - Late February In-briefing with USAID, team planning meetings and interviews with key stakeholders in Santo Domingo; finalize evaluation framework, protocol, and survey tools; organize logistics for field work 5 4 5 and Spanish.t in English. t of ng Facilitator Late February Logistical preparations - - 3 Mid February Fieldwork (including travel days) 5 5 - Early March USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 55 Activity Team Leader Sr. MCH Advisor HS and MCH Experts Local Logistics Coord. Period of Performance (illustrative, depending on start date) Preliminary data analysis and synthesis; drafting report and presentation materials with additional follow up meetings 3 3 3 - Mid-March Stakeholders presentation on preliminary findings 1 1 1 - Mid-March Evaluation team incorporates feedback from debriefings into draft report, submits draft report 3 3 - Late March Team departs country 1 - - Late March Mission sends technical feedback/comments on draft to Team Leader - - - Early April Draft revised by Team Leader and team. GH Tech submits final report to Mission 5 2 - Mid-April Translation of final report - - - Late April Mission approves report - - - Late April-Early May Total LOE 29 8 23 10 A six-day work week is approved while in-country. IX. LOGISTICS GH Tech will be responsible for all international travel and consultant logistics. USAID/DR will send letters of introduction informing key MOH staff and other high-level partners of the nature, timing, and scope of the evaluation and the evaluation team members. X. DELIVERABLES AND PRODUCTS The evaluation team will complete the following deliverables: Evaluation framework including revised evaluation questions; detailed approach/methodology to be used, including the documents to review, key informants to interview, sampling frame, evaluation protocols and instruments; and plans for analysis and dissemination of findings. The Team Leader will submit the evaluation framework to USAID/DR and GH Tech after the in￾country TPM. USAID/DR will then review the proposed work plan/methodology and submit comments to the team leader within two business days. The evaluation team will revise the work plan/methodology and send the final version to USAID/DR and GH Tech. The evaluation framework must be approved prior to the initiation of the interviews and site visits. Weekly status reports. The Team Leader will provide weekly status reports on work plan implementation to USAID/DR and GH Tech. 56 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Presentation slides (in MS PowerPoint) used during stakeholder meeting and debriefing to USAID/DR staff on the preliminary findings and recommendations. The PowerPoint presentations will be shared with GH Tech prior to the USAID and stakeholder debriefings. Draft report in English no longer than 30 pages with an executive summary, introduction, methodology, findings, and recommendations that address each of the three objectives and subsequent questions with bibliography and annexes. The Team Leader will submit the first draft report to USAID/DR and GH Tech at the end of the evaluation team’s visit. The Mission will provide consolidated, written comments to the evaluation team and GH Tech within 5 working days of receiving the draft report. Final report will address the Mission’s comments. The Team Leader will submit the final unedited report to USAID/DR and GH Tech within 10 working days after the team receives consolidated comments from USAID. GH Tech will provide the edited and formatted final document approximately 30 days after USAID provides final approval of the content. Procurement sensitive information will be removed from the final report and incorporated into an internal USAID Memo. The remaining report will then be released as a public document on the USAID Development Experience Clearinghouse (DEC) (http://dec.usaid.gov) and the GH Tech project web site (www.ghtechproject.com). XI. RELATIONSHIPS AND RESPONSIBILITIES GH Tech will coordinate and manage the evaluation team and will undertake the following specific responsibilities throughout the assignment: Recruit and hire the evaluation team. Make logistical arrangements for the consultants, including travel and transportation, country travel clearance, lodging, and communications. USAID/DR will provide overall technical leadership and direction for the evaluation team throughout the assignment and will provide assistance with the following tasks: Before In-Country Work SOW. Respond to queries about the SOW and/or the assignment at large. Consultant Conflict of Interest (COI). To avoid conflicts of interest or the appearance of a COI, review previous employers listed on the CVs for proposed consultants and provide additional information regarding potential COIs with the project contractors evaluated/assessed and information regarding their affiliates. Documents. Identify and prioritize background materials for the consultants and provide them to GH Tech, preferably in electronic form, at least one week prior to the inception of the assignment. Local Consultants. Assist with identification of potential local consultants, including contact information. Site Visit Preparations. Provide a list of site visit locations, key contacts, and suggested length of visit for use in planning in-country travel and accurate estimation of country travel line item costs. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 57 Lodgings and Travel. Provide guidance on recommended secure hotels and methods of in￾country travel (i.e., car rental companies and other means of transportation) and if necessary, identify a person to assist with logistics (i.e., visa letters of invitation, etc.). During In-Country Work Mission Point of Contact. Throughout the in-country work, ensure constant availability of the Point of Contact person and provide technical leadership and direction for the team’s work. Meeting Space. Provide guidance on the team’s selection of a meeting space for interviews and/or focus group discussions (i.e., USAID space if available, or other known office/hotel meeting space). Meeting Arrangements. Assist the team in arranging and coordinating meetings with stakeholders. Facilitate Contact with Implementing Partners. Introduce the evaluation team to implementing partners and other stakeholders, and where applicable and appropriate prepare and send out an introduction letter for team’s arrival and/or anticipated meetings. After In-Country Work Timely Reviews. Provide timely review of draft/final reports and approval of deliverables. XII. MISSION CONTACT PERSON Derek R. Sedlacek, MPH Health Officer USAID/Dominican Republic Email: dsedlacek@usaid.gov Phone Number: 809 731-7045 XIII. XIII. COST ESTIMATE GH Tech will provide a cost estimate for this activity. 58 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 59 ANNEX 2: PERSONS CONTACTED (NOT EXHAUSTIVE) NATIONAL LEVEL: MINISTRY OF HEALTH, IMPLEMENTING PARTNERS, AND STAKEHOLDERS Goldie Mazia MCHIP, Washington, DC, USA Dra. Nieves Rodriguez MCHIP, Dominican Republic Dr. Lucas Gomez Director General of Specialized Centers for Health, Ministry of Health Dr. Jose de Lance Maternal-Child Health Director, Ministry of Health Dra. Francine Placencia Director General of M&E Quality Assurance Dra. Raquel Pimentel Director of Epidemiology Dr. Lopez Monegro Director of Decentralization of Regions Dra. Martha Martinez Liaison, Department of Authorization and Accredidation Lic. Clotilde Duran Program Coordinator Lic. Lidia Marite Information Systems Dario Pena Kangaroo Mother Team Carmen Pequeno Kangaroo Mother Team Solange Nunez Kangaroo Mother Team Leonidas Reyes Kangaroo Mother Team Hortencia Hernandez Kangaroo Mother Team Dra. Rosa Maria Suarez Abt Associates, Director, Centers of Excellence Project Cecilia Corporan Abt Associates, M&E Coordinator, Centers of Excellence Project Dr. Carlos Cuellar Abt Associates, Ex-Director, Centers of Excellence Project HOSPITAL SAN LORENZO DE LOS MINAS Dr. Gregorio Porfirio Diaz Regional Director of Health Dr. Paulino Díaz Osoria Hospital Director Lic. Bartolina santos Administrator Dr. Andres Rivas Ex-Director Dr. José Shouwer Head of Epidemiology Dra. Nalda De oleo Head of Perinatology Group Discussion 6 Users of health services Women 19-26 years of age who have delivered with the past three years. One was a pregnant adolescent and three had high-risk pregnancies. 60 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 4 Health Providers Ob/Gyn residents, 2nd, 3rd, and 4th year. Posted on gynecological surgery, post-delivery, and post-surgery wards HOSPITAL MORILO KING. LA VEGA Dr. Juan de La Cruz Director of the hospital Dr. José Luis Mate Subdirector Dr. Freddy Hidalgo Regional Director of Health, Region 8 HOSPITAL INMACULADA CONCEPCION, JUAN SANCHEZ RAMIREZ Dr. Bolivar Matos Regional Director of Health Dra. Mayra Bido Hospital Director Not Available Accountant of the Center Not Available Epidemiologist Dra. Julissa Robles Subdirector Group Discussions 14 Users of health services Women 16-33 years of age who had delivered with the past three years. Note: appointed by the hospital director 9 Health providers Maternity Ward nurses; working experience 4 to 28 years in the same hospital HOSPITAL SAN VICENTE DE PAUL, SAN FRANCISCO DE MACORIS Dr. Ángel Ambrosio Rosario Hospital Director Lic. Hortensia Hernandez Administrator Dra. Ramona Mercedes Pediatrician Lic. Sorangel Morillo Warehouse/Pharmacy Manager Group Discussions 7 Health providers Medical doctors providing outpatient services. Note: rotation is every three months among the different departments. HOSPITAL ALEJANDRO CABRAL, SAN JUAN DE LA MAGUANA Dr. Hector Ortiz Hospital Director Dr. Ramon Evangelista Chief of Gynecology and Obstetrics Lic. Daysi Hernandez Strategic Manager Group Discussion 7 Users of health services Pregnant women of 20-27 years of age, on prenatal care, who had at least one previous delivery in the same hospital 7 Health providers Emergency ward health staff (medical residence, family health residents, cleaning staff and counselor) HOSPITAL DR ANTONIO MUSA, SAN PEDRO DE MACORIS Guillermo Hernández Director USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 61 Félix Mejía Sub-Director Silvio Tomas Chief of Gynecology and Obstetrics Amaira González, Chief of Instruction Magalys Vidal Chief Nurse Group Discussion 9 Users of health services Pregnant women of 21-34 years of age, on prenatal care, who had at least one previous delivery in the same hospital 62 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 63 ANNEX 3. REFERENCES ABT Associates Report. Annex 2 Summary of Performance Indicators. August 6, 2009. Abt Fully Executed Contract DR TO GHS-I-05-07-00003-00. Baez, J., and E. Perez-Then. “Situación de los servicios de salud materno infantil de la Republica Dominicana." Serie de Reuniones Tecnicas I, Centro Nacional de Investigaciones en Salud Materno Infantil (CENISMI), 2009. Bahl R, J. Martines, N.Ali et al. “Research Priorities to Reduce Global Mortality From Newborn Infections by 2015.” Pediatric Infectious Disease Journal, 2009; 28(1): S43-S48. Bases del Sistema de Certificacion CDEx, 8 de febrero 2012. Modificada. Canario A., E. Pérez-Then, A. De Moya, et al. “Proceso de abordaje rápido para describir las condiciones de salud y ecológicas de bateyes localizados en el suroeste de la República Dominicana.” Boletín CENISMI, 16(3):26-27, 2006. Centro de Estudios Sociales y Demográficos (CESDEM). Línea de base: Nivel de satisfacción de las usuarias y los usuarios de los servicios pediátricos y obstétricos, en hospitales seleccionados del ministerio de salud pública de la República Dominicana. 2010. CESDEM. Línea de Base: Medición de la Capacidad de Gestión en Instituciones Seleccionadas del Ministerio de Salud Pública de la República Dominicana. 2010. Curbing Maternal Mortality rates in the Dominican Republic. USAID. January 17, 2012. Dominican Republic Country Program. Quarterly Report 1 2012. October 1, 2011–December 31, 2011. “Environmental Mitigation Report (EMR), USAID Dominican Republic Mission. Title of IP Activity: USAID Improving Maternal & Child Care/Health Systems Project.” Abt Associates Inc. Funding Period: Fiscal Year 2009 - 2010. Evaluation of REDSALUD and USAID/DR Strategic Support to Reform of the Dominican Health Sector. Deloitte Touche Tohmatsu Emerging Markets, Ltd. May 28, 2004. Freedman, LP. “Human rights, constructive accountability and maternal mortality in the Dominican Republic: A commentary.” International Journal of Gynecology and Obstetrics, 2003. Vol.82, pp.111–114. Hernández, C. y Romero, M. “Atención médica y grado de satisfacción de la paciente Maternidad Nuestra Señora de la Altagracia,” 1993. En OMS/OPS/INSALUD, Análisis del desempeño: Maternidad Nuestra Señora de la Altagracia. Santo Domingo, 1996. Hernández, C. y Romero M. “Informe Proceso de Programación: Actitudes y cambios,” 1996. En OMS/OPS/INSALUD, Análisis del desempeño: Maternidad Nuestra Señora de la Altagracia. Santo Domingo, 1996. 64 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Hodge, J.G., Garcia, A.M., Anderson E.D., Kaufman, T. “Emergency Legal Preparedness for Hospitals and Health Care Personnel.” Disaster Medicine and Public Health Preparedness, 2009. Vol 3, Suppl. 1, S37-S44. Informe de Consultoria Luis Morales. Reorganizacion de la Cartera de Servicios en el Servicio Regional de Salud. El Valle, SRS R-VI. Noviembre 23 de 2011 a enero 31 de 2012. 30 Enero 2012. Johnston, A.M., C.E. Bullock, J.E. Graham et al., “Implementation and Case-Study Results of Potentially Better Practices for Family-Centered Care: The Family-Centered Care Map.” Pediatrics 118: Supplement 2 (November 1, 2006). Kangaroo Mother Care Clinical Practice Guidelines; KMC India Network. Available at http://www.newbornwhocc.org/pdf/KMC%20Network%20Booklet.pdf. Accessed April 14, 2012. Kahssay HM & Oakley P (Eds.) Community involvement in health development. A review of the concept and practice. World Health Organization (WHO), Geneva, 1999. Maternal & Child Centers of Excellence. Abt Associates, Inc. Implementation Plan 2009-2014, August, 2009. Maternal and Child Health Integrated Program (MCHIP) Annual Report FY11 Program Year 3, October 1, 2010 to September 30, 2011. Dominican Republic. MCHIP Dominican Republic Country Program. Quarterly Report 1, 2012. October 1, 2011 – December 31, 2011. MCHIP Dominican Republic Country Program. Quarterly Report. April 1, 2010–September 30, 2012 Miller, S., A. Tejada, P. Murgueytio, J. Diaz, R. Dabash, P. Putney, S. Bjegovic, and G. Caraballo, USAID, SESPAS and the Population Council. Strategic Assessment of Reproductive Health in the Dominican Republic. February 15, 2002. Miller, S., M. Cordero, A.L. Coleman, J. Figueroa, S. Brito-Anderson, R. Dabagh, V.Calderone, F. Cáceres, A. J. Fernandez , and M. Nunez, “Quality of Care in Institutionalized Deliveries: The Paradox of the Dominican Republic.” International Journal of Gynecology & Obstetrics. 2003 Jul; 82(1):89-103. Ministerio de Administración Pública, Republica Dominicana. Common Assessment Framework (CAF). Una herramienta inspirada en el Modelo de Excelencia de la Fundacion Europea de Gestion de la Calidad (EFQM) y en el Modelo de la Universidad Alemana de Ciencias Administrativas de Speyer. Versión piloto, mayo de 2000. Primera versión, 2002. Segunda versión, 2006. Miric, M., Diplan, J., Martinez, M., Díaz, L., De la Rosa, J. “Análisis situacional del uso de planificación familiar en programas posparto, postaborto y de la reducción de la transmisión vertical del VIH/SIDA en la República Dominicana. Enfoque cualitativo.” Population Council: Fronteras de la Salud Reproductiva / Centro de Estudios Sociales y Demográficos (CESDEM). Santo Domingo, Unpublished report, 2006. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 65 Miric, M., Campos, I., Figueroa, J., Báez, C. “Assessment of access and use of obstetric emergency services (COEm) in the Dominican Republic.” UNFPA / Public Health Ministry (MSP) / Consultoras Asociadas para el Desarrollo (CONSAD). Unpublished report, 2008. Miric, M., y Pérez-Then, E. (Eds.). “Salud Materno-Infantil en República Dominicana: Situación actual y perspectivas futuras a la luz de los nuevos indicadores.” Serie de Reuniones Técnicas I, CENISMI. Santo Domingo, 2011. Miric, M., Pérez-Then, E., Santos, L., Mendoza, R., Sehuoerer, J.A., Bautista, C., Martínez M., Díaz, L., Rosario, M.L., Fernández, B., Orne-Gliemann, J., Tchendjou, P., Dabis, F., Shor￾Posner, G. “Impacto de la Consejería prenatal del VIH centrada en parejas de países con baja y mediana prevalencia del VIH (Proyecto Prenahtest): Fase de factibilidad.” Boletín CENISMI, 2008. 18(1): 2–6. Pérez-Then, E. (Ed.). “Calidad de la atención de la embarazada y del niño menor de un año en Centros de primer nivel de atención de la Regiones III, IV, V, VI de la República Dominicana.” CENISMI. Serie de Reuniones Técnicas IV, 2008. Pérez-Then, E., et al. “Evaluación de la calidad de atención a la embarazada y al niño sano en centros de primer nivel de atención de las Regiones de Salud III, IV, V y VI de la República Dominicana: Resultados de la segunda fase.” Boletín CENISMI, vol. 20, no 2. 2010. Pérez-Then, E., Gómez, V., Fernández, J., Guerrero, S., Marte, J., De la Cruz, C., Medina, Y., Espinal, R., Diaz, A., Machuca, M. “Uso y abuso de antibióticos después de la realización de una intervención eductaiva en centros de salud públicos y privados de la República Dominicana.” Boletín CENISMI, 2010. Vol. 20, No 2. Pérez-Then, E., Miric, M., Canario, J.A., Martínez, E., Martínez, M. “Encuesta nacional de percepción de mujeres usuarias respecto de los servicios públicos de maternidad en República Dominicana.” Informe inédito. Organización Panamericana de la Salud (OPS), Santo Domingo, 2011. Pérez-Then, E., Miric, M., Castro, A. “Integración de la Atención Prenatal con los Procesos de Detección y Manejo Clínico del VIH y de la Sífilis en la República Dominicana.” UNICEF, Santo Domingo, 2011. Phillips, C.R. (Ed.), Family-Centered Maternity Care. Jusburry, MA: Jones and Barlett Publishers, 2006. Proyecto USAID/Centros de Excelencia Materno-Infantil. Manual de estandares para la certificacion de hospitales y direcciones provinciales de salud como centros de excelencia materno infantil. Republica Dominicana, Enero de 2012. Proyecto USAID/Centros de Excelencia Materno-Infantil. Manual de preparacion para la certificacion de hospitales y direcciones provinciales de salud como centros de excelencia materno infantil. Republica Dominicana, Enero de 2012. Proyecto USAID/Centros de Excelencia Materno-Infantil. Instituto Tecnológico de Santo Domingo (Intec). Línea de base para la implementación del sistema de vigilancia, prevención y control de infecciones intra - hospitalarias en hospitales seleccionados para desarrollarse como Centros de Excelencia Materno-Infantil. República Dominicana, 2010. 66 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Quiterio, G., Molina, M., Miric, M., Vernon, R. y Rivero-Fuentes, E. “Situation Analysis of the Integration of Family Planning Services in Postpartum, Postabortion and Prevention of Mother to Child Transmission Programs in the Dominican Republic.” Population Council: Frontiers in Reproductive Health/USAID, 2008. Rifkin, S.B. “Community Participation in MCH/FP Programmes: An Analysis Based on Case Study Material.” WHO/UNICEF, Geneva, 1990. Rivero-Fuentes, E., Miric, M., Vernon, R., Quiterio, G., Molina, M. “Calidad de la consejería durante la atención posaborto en República Dominicana.” En Billings, DL y Vernon R. (Eds.), Avances en la atención posaborto en América Latina y el Caribe: Investigando, aplicando y expandiendo. Population Council, Mexico, 2007. Ruminjo J, Cordero C, Beattie KJ y Wegner MN (2003). “Quality of care in labor and delivery: a paradox in the Dominican Republic; commentary.” International Journal of Gynecology and Obstetrics. Vol.82, pp.115-119. Secretaria de Estado de Salud Pública y Asistencia Social (SEPSAS). Disposicion sobre el desarrollo de las funciones y organización de las direcciones provinciales de salud (DPS). Disponible en: http://www.salud.gob.do/download/docs/Disposiciones/DispDesfunOrg.pdf. Acceso: 15/04/2012. Sistema Nacional de Vigilancia Epidemiológica en República Dominicana. Available at http://www.sespasdigepi.gob.do/sinave/sistema_nacional_de_vigilancia.pdf. Access: 04/15/2012 Tec. 1 Guía Metodológica para la implementación de un Sistema de Gestión de Riesgos y Bioseguridad. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 2 Guía Metodológica para la implementación de un Sistema de Gestión de Residuos y Desechos Hospitalarios. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 3 Guía Metodológica para la implementación de un Sistema de Atención de Usuarios. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 4 Guía Metodológica para la implementación de un Sistema de Gestión de Suministros en Hospitales. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 5 Guía Metodológica para la formulación de un plan estratégico de desarrollo institucional. Dra. Amparo Rueda de Chaparro Consultora Internacional. Unversidad CES, Medellín, Colombia. Abril de 2010. Tec. 6 Guía Metodológica para el Análisis de situación y elaboración de los perfiles epidemiológicos. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 7 A SIGHO-Suministros_Manual_Usuarios. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 67 Tec. 7 B SIHA—Manual Operativo de Usuarios. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 7 C SINAC—Manual Operativo de Usuarios. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 7 D Manual Operativo de Usuarios—RIV. Maternal & Child Centers of Excellence. Ministerio de Salud Pública. Republica Dominicana, 2012. Tec. 8 Guía para elaboración Planes Hospitales de Emergencia. Thacker, S.B., R.G. Parrish, F.L. Trowbridge FL. “A Method for Evaluating Systems of Epidemiological Surveillance.” Available at: http://ukpmc.ac.uk/abstract/MED/3269210/reload=0;jsessionid=60QrXaSTQkjJ2JMgihW N.4. Access: 04/15/2012. USAID Project Maternal & Child Centers of Excellence: Annex 1: Performance-Based Management System: Objectives and Performance indicators by Task and Result. August 6, 2009. USAID Project Maternal & Child Centers of Excellence. Implementation Plan 2009-2014: Annex 1. Selected hospitals by location, region, level of care and potential for synergy and replication. USAID Project Maternal & Child Centers of Excellence. Implementation Plan 2009-2014: Annex 2 Gantt_Implementation Plan_EN. 68 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 69 ANNEX 4. MATRIX TO ASSESS THE MATERNAL-INFANT CENTERS OF EXCELLENCE IN THE DOMINICAN REPUBLIC. USAID 2009–2014* . *The evaluation was carried out during February and March 2012. Evaluation questions Objective Results Phases Evaluation Strategy Source What programmatic aspects have been the most/least successful in ensuring the achievement of the project results and why? Improve equitable access to quality maternal-child services focusing on critical interventions and management improvements that enhance quality and efficiency. Result 1 Ten hospitals developed as Centers of Excellence have enhanced capacity for clinical & resource management and administrative processes in support of MCH programmatic areas. Induction and preparatory Implementation Consolidation Replication Rapid Assessment Procedure (RAP) that includes document review, direct observation, in-depth interviews and group discussions. Documents and references, databases, direct observation, and in￾depth and group interview results What technical interventions have been the most/least successful in ensuring the achievement of the project results and why? Result 2 Three Provincial Health Directorates developed as Centers of Excellence have enhanced stewardship capability for epidemiological surveillance and response, public health programs, and certification/accreditation of public and private health providers. Induction and preparatory Implementation Consolidation Replication Rapid Assessment Procedure (RAP) that includes document review, direct observation, in-depth interviews and group discussions. Documents and references, databases, direct observation, and in￾depth and group interview results What programmatic aspects appear to be the most/least sustainable and why? Result 3 Regional network developed and tested serves to identify best approaches for development of regional networks. Induction and preparatory Implementation Consolidation Replication Rapid Assessment Procedure (RAP) that includes, documents review, direct observation, in-depth interviews and Group Discussions. Documents and references, databases, direct observation, and in￾depth and groups interview results What technical interventions appear to be the most/least sustainable and why? Has the project targeted the most critical and appropriate activities, what other opportunities exist? 70 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 71 ANNEX 5. MID TERM EVALUATION FRAMEWORK OF THE RESULTS OF THE MATERNAL-INFANT CENTERS OF EXCELLENCE PROJECT DOMINICAN REPUBLIC FEBRUARY–MARCH 2012. Result 1 Implementation Phase Categories Obtained from Rosa Maria Suarez and adapted by Eddy Perez, evaluator Implementation Phase Indicators Evaluation Methodology Source of Information Ten hospitals developed as Centers of Excellence have enhanced capacity for clinical & resource management and administrative processes in support of MCH programmatic areas Systems and management capability strengthened Improvement of hospital organization in the past two years, including filing cabinets and software systems Key informant interviews (KII) Group discussion (GD) In-depth Interviews (IDI) Questions 16-27, 28-30, and 82-89 of the GD and IDI Evidence-based quality maternal & child services Training curricula on health interventions Adaptation of clinical guidelines and procedures 12 supervision check lists to assess quality of services delivered Maternal Mortality Audit Forms Monitoring key indicators Key informant interviews (KII) Group discussion (GD) In-depth Interviews (IDI) Questions 47-71 of the GD and IDI Interconnected within the health services network Referral and counter-referral system; project support for transportation of patients Group discussion (GD) In-depth interviews (IDI) Questions 74-77 of the GD and IDI Training center developed The hospital has a training center, and training activities are carried out. Key informant interviews (KII) Group discussion (GD) In-depth Interviews (IDI) Questions 31-34 of the GD and KII Basic infrastructure certified Improvement of quality indicators, humanized attention and exit interviews Key informant interviews (KII) In-depth interviews (IDI) Questions 35-46 of the KII and IDI Environment preserved Toxic waste disposal containers (bags and boxes); waste transit route; intermediate storage; immunization of staff; agreement with municipal authorities Group discussion (GD) In-depth interviews (IDI) Questions 72-73 of the GD and IDI Community participating Strategies to increase community participation Group discussion (GD) Questions 95-96 of the GD Safe hospital for emergencies and disasters Evacuation routes; drills and committees of risks Group discussion (GD) In-depth interviews (IDI) Questions 90-94 of the GD and IDI 72 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 73 ANNEX 6. ENTREVISTA A INFORMANTES CLAVES: DIRECCIÓN REGIONAL DE SALUD EVALUACIÓN DE TÉRMINO MEDIO DEL PROYECTO CENTROS DE EXCELENCIA MATERNO INFANTIL REPÚBLICA DOMINICANA. ABRIL-MARZO, 2012 Entrevista a Informantes Claves: DIRECCIÓN REGIONAL DE SALUD La presente guía de entrevistas es para el personal de la Dirección Regional de Salud. Por favor seleccione entrevistar aquellas personas del staff que tiene más tiempo en el servicio y conoce sobre el proyecto “Centros de Excelencia Materno-Infantil.” Por favor entreviste a la persona exhaustivamente, sin dejar ninguna pregunta sin contestar. Las preguntas son semi-estructuradas y abiertas, es decir, se deben llenar las respuestas como las describa el entrevistado. En los casos de que sea si o no, serán como filtros para profundizar en el tema. Utilice su experiencia y criterio para profundizar en algunos temas si le parecen interesantes, pida mayor información o evidencia si es necesario. Por favor use hojas sueltas si necesita expandir las respuestas, y adjúntelas con grapas al finalizar la entrevista. Nombre del entrevistador: Fecha: Nombre del entrevistado: Posición/titulo: Distrito/provincia: Pueblo/ciudad: Conocimientos generales acerca del proyecto: 1. Describa brevemente sus responsabilidades e involucramiento con el proyecto sobre Centros de Excelencia desde mediados del 2009 a la fecha. (Nota para el entrevistador: Profundice sobre su participación y por cuanto tiempo) Selección de los Centros de Excelencia (CDEx) 2. Explique que criterios se aplicaron para seleccionar el(los) hospitales en su región? 3. ¿Esta de acuerdo con el proceso de selección? ¿Como lo mejoraría, si tiene que elegir otros? 74 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Creación de capacidad gerencial 4. Explíqueme en que sentido cambió o mejoró su estilo gerencial gracias al acompañamiento del proyecto CDEx? (Nota para el entrevistador. Explore sobre los conceptos de gestión para el cambio o Change Management?) 5. ¿Son replicables estas intervenciones? ¿Cómo los extendería a todos los hospitales de su región? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 6. ¿Explique cuales fueron las conclusiones de la aplicación del Plan Estrategico que se realizó en el año 2010 y el 2015? Cuales fueron las prioridades para la mejoría de la calidad en esos mismos años? (Nota para el entrevistador. Explore sobre CAF- Common Assessment Framework) 7. ¿Superviso la región la elaboración de los planes de acción de los hospitales CDEx 2011? (Nota para el entrevistador. Pida que se lo muestre) Pida que le describa el proceso que sigue para elaborar el plan operativo y registre en el cuadro a continuación (Nota para el entrevistador: puede mostrar la lista. Anote en la columna de observaciones los comentarios que haga): USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 75 Estructura, funciones y capacidad Si Parcial mente No Comentarios/observaciones 8. ¿Tiene documentado el plan operativo? Muéstrelo por favor ☐ ☐ ☐ 9. Se usan datos estadísticos (de la región) para la priorización de actividades ☐ ☐ ☐ 10. ¿Incluyen estrategias específicas para la prestación de servicios en la región? ☐ ☐ ☐ 11. ¿Tiene metas anuales (o el periodo que se usa)? ☐ ☐ ☐ 12. ¿Tiene un calendario de actividades? ☐ ☐ ☐ 13. ¿Tiene un presupuesto? ☐ ☐ ☐ 14. ¿Están en capacidad de desarrollar el próximo plan operativo? ¿Necesitan algún tipo de apoyo? (Nota para el entrevistador: Si es si, o no, por favor elabore y especifique) Monitoreo, Evaluación y Sistemas de Información 15. ¿Cuáles son las brechas más importantes en la recolección, análisis y uso de información estratégico? 16. ¿Cómo está apoyándole el proyecto CDEx en la recolección, análisis y uso de información estratégico? 17. ¿Cuáles planes tienen para seguir mejorando la recolección, análisis y uso de información estratégico? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 76 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 18. ¿Con que frecuencia revisa el sistema informático perinatal (SIP)? Estructura, funciones y capacidad Si Parcial mente No Comentarios/observaciones 19. ¿Hay un plan de monitoreo y evaluación para la región? ☐ ☐ ☐ 20. ¿El sistema de información recolecta datos del nivel primario, la red de ONGs y el sector privado? ☐ ☐ ☐ 21. ¿Existe mecanismos de revisión de la calidad de la información (exactitud, integridad, oportunidad y confidencialidad) recibida de los niveles de inferiores que la reportan? ☐ ☐ ☐ 22. ¿Hay un plan de capacitación en M&E para el personal de la región? ☐ ☐ ☐ Formatos y herramientas para recoger información y presentar informes Si Parcial mente No Comentarios/observaciones 23. ¿El sistema de información que utilice está automatizado? ☐ ☐ ☐ 24. ¿Si múltiples organizaciones (públicos y privados) están implementando actividades bajo el programa/proyecto, todos usan los mismos formatos de informes y reportan de acuerdo a los mismos lineamientos y plazos de tiempo? ☐ ☐ ☐ 25. ¿La información recogida a través del sistema de información, es suficientemente precisa para medir el/los indicador/es y desagregados (p.ej. género, edad, etc.)? ☐ ☐ ☐ 26. ¿La región realiza análisis de datos en forma regular? ☐ ☐ ☐ 27. ¿la región realiza retroalimentación del análisis de la información con los hospitales? USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 77 Gestión de Recursos 28. ¿Cuáles son las diferentes fuentes de financiamiento para la región y cuales estrategias esta implementando para optimizar la adquisición de recursos financieros? 29. ¿Cuáles son las brechas más importantes en el manejo financiero? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 30. ¿Cómo esta apoyándoles el proyecto CDEx en manejo financiero? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) Pregunte y profundice sobre las respuestas recibidas # Que evidencia observo? Comentarios/observaciones 31. Capacitación del personal: a) hay un plan de capacitación y una evaluación de las necesidades de capacitación, de acuerdo a los protocolos actualizados, y estrategias e intervenciones prioritarias b) La capacitación se produce en forma aislada, pero no ligada a los protocolos actualizados, y estrategias e intervenciones prioritarias c) Hay un plan de capacitación y esta de acuerdo a los protocolos actualizados, y estrategias e intervenciones prioritarias, pero no se cumple a cabalidad d) Hay un plan de capacitación y evaluaciones continuas de las necesidades de capacitación en base a los protocolos actualizados, y estrategias e intervenciones prioritarias y se cumplen completamente 32. ¿Qué capacitaciones ha recibido del proyecto? 78 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 33. ¿Cuentan con un equipo responsable de las capacitaciones del proyecto? 34. ¿Cuáles son las capacitaciones prioritarias para difundir al resto de los establecimientos de salud en la región? (Nota para el entrevistador. Incluye capacitaciones del proyecto y fuera del proyecto) Gestión de cartera de servicios 35. ¿Cuáles son las brechas más importantes en la gestión de cartera de servicios? 36. ¿Cómo esta apoyándole el proyecto CDEx para asumir el rol de la gestión de cartera de servicios? Articulación entre los diferentes niveles 37. Descríbame por favor el sistema actual de referencia y contra-referencia 38. ¿Como esta apoyándole el proyecto de CDEx en mejorar este sistema de referencia y contra-referencia? 39. Descríbame por favor el sistema actual de traslado de pacientes. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 79 40. ¿Como esta apoyándole el proyecto de CDEx en mejorar este sistema de traslado de pacientes? 41. Descríbame por favor el sistema actual de traslado de muestras de laboratorio. 42. ¿Como esta apoyándole el proyecto de CDEx en mejorar este sistema de traslado de muestras de laboratorio? Participación comunitaria 43. ¿Cuáles estrategias se está implementando para aumentar la participación ciudadana en la región? 44. ¿Como esta apoyándole el proyecto de CDEx en mejorar este proceso de participación ciudadana? Lecciones Aprendidas 45. Si tuviera que replicar todo el proceso de Centros de Excelencia ¿Cuáles estrategias no funcionaron y cuales si, y que cambiaria? (Nota para el entrevistador. Anote todo lo que diga) Muchas gracias por su participación 80 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 81 ANNEX 7. ENTREVISTA A INFORMANTES CLAVES: DIRECCIÓN PROVINCIAL DE SALUD EVALUACIÓN DE TÉRMINO MEDIO DEL PROYECTO CENTROS DE EXCELENCIA MATERNO INFANTIL REPÚBLICA DOMINICANA. ABRIL-MARZO, 2012 Entrevista a Informantes Claves: DIRECCIÓN PROVINCIAL DE SALUD La presente guía de entrevistas es para el personal de la Dirección Provincial de Salud. Por favor seleccione entrevistar aquellas personas del staff que tiene mas tiempo en el servicio y conoce sobre el proyecto “Centros de Excelencia Materno-Infantil.” Por favor entreviste a las personas exhaustivamente, sin dejar ninguna pregunta sin contestar. Las preguntas son semi-estructuradas y abiertas, es decir, se deben llenar las respuestas como las describa el o los entrevistados. En los casos de que sea si o no, serán como filtros para profundizar en el tema. Utilice su experiencia y criterio para profundizar en algunos temas si le parecen interesantes, pida mayor información o evidencia si es necesario. Por favor use hojas sueltas si necesita expandir las respuestas, y adjúntelas con grapas al finalizar la entrevista. Nombre del entrevistador: Fecha: Nombre del entrevistado: Posición/titulo: Distrito/provincia: Pueblo/ciudad: Conocimientos generales acerca del proyecto: 1. Describa brevemente sus responsabilidades e involucramiento con el proyecto Centros de Excelencia desde mediados del 2009 a la fecha. (Nota para el entrevistador: Profundice sobre su participación y por cuanto tiempo) Capacitación en gestión 2. Explíqueme en que sentido cambió o mejoró su estilo gerencial gracias al acompañamiento del proyecto CDEx? (Nota para el entrevistador. Explore sobre los conceptos de gestión para el cambio?) 82 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 3. ¿Son replicables estas capacitaciones o intervenciones? ¿Cómo los extendería a otras DPS? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 4. ¿Explíqueme cuales fueron las conclusiones de la auto-evaluación que se realizó en el año 2010 y el 2011? ¿Cuales fueron las prioridades para la mejoría de la calidad en esos mismos años? ¿Presentaron memorias al premio de la calidad, puede mostrarla, han recibido y discutido informes de retorno de los evaluadores externos? (Nota para el entrevistador. Explore sobre CAF- Common Assessment Framework) 5. ¿Cual fue su plan estratégico 2010 y 2015? (Nota para el entrevistador. Pida que se lo muestre) 6. Pida que le explique: Cual es el periodo del plan Estrategico? Cuando fue aprobado el plan? Quien/quienes aprueban el plan? Pida que le describa el proceso que sigue para elaborar el plan operativo y registre en el cuadro a continuación . (Nota para el entrevistador: puede mostrar la lista. Anote en la columna de observaciones los comentarios que haga): Estructura, funciones y capacidad Si Parcial mente No Comentarios/observaciones 7. ¿Tiene documentado el plan operativo? Muéstrelo por favor ☐ ☐ ☐ 8. ¿Se usan datos estadísticos (de la provincia) para la priorización de actividades? ☐ ☐ ☐ USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 83 Estructura, funciones y capacidad Si Parcial mente No Comentarios/observaciones 9. ¿Incluyen estrategias específicas para la provincia? ☐ ☐ ☐ 10. ¿Tiene metas anuales (o el periodo que se usa)? ☐ ☐ ☐ 11. ¿Tiene un calendario de actividades? ☐ ☐ ☐ 12. ¿Tiene un presupuesto? ☐ ☐ ☐ 13. Tienen Sala de Situación ☐ ☐ ☐ 14. Realizan el análisis de los datos dentro de la sala de situación ☐ ☐ ☐ 15. ¿Están en capacidad de desarrollar el próximo plan operativo? ¿Necesitan algún tipo de apoyo? (Nota para el entrevistador: Si es si, o no, por favor elabore y especifique) Monitoreo, Evaluación, Sistemas de Información, Vigilancia y Epidemiologia 16. ¿Cuáles son las brechas más importantes en la recolección, análisis y uso de información estratégica? 17. ¿Cómo está apoyándole el proyecto CDEx en la recolección, análisis y uso de información estratégica? 18. ¿Cuáles planes tienen para seguir mejorando la recolección, análisis y uso de información estratégica? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 84 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 19. ¿Con que frecuencia revisa el sistema informático perinatal (SIP)? (Nota para el entrevistador. Esta información se analiza en el Hospital y si debe analizarse después como consolidado debe ser en el Servicio Regional, responsable de los servicios) Estructura, funciones y capacidad Si Parcialmente No Comentarios/ observaciones 20. ¿Hay un plan de monitoreo y evaluación de los programas para la provincia? ☐ ☐ ☐ 21. ¿El sistema de información recolecta datos del nivel primario, la red de ONGs y el sector privado? ☐ ☐ ☐ 22. ¿Existen mecanismos de revisión de la calidad de la información (exactitud, integridad, oportunidad y confidencialidad) recibida de los niveles de inferiores que la reportan? ☐ ☐ ☐ 23. ¿Hay un plan de capacitación en M&E para el personal de la DPS? ☐ ☐ ☐ Los Formatos y herramientas para recoger información y presentar informes Si Parcialmente No Comentarios/observa ciones 24. ¿El sistema de información que se utiliza está automatizado? ☐ ☐ ☐ 25. ¿Si múltiples organizaciones (públicos y privados) están implementando actividades bajo el programa/proyecto, todos usan los mismos formatos de informes y reportan de acuerdo a los mismos lineamientos y plazos de tiempo? ☐ ☐ ☐ 26. ¿La información recogida a través del sistema de información, es suficientemente precisa para medir el/los indicador/es y desagregados (p.ej. género, edad, etc.)? ☐ ☐ ☐ 27. ¿La provincia realiza análisis de datos en forma regular? ☐ ☐ ☐ 28. ¿Se realizan retroalimentación del análisis de la información con los establecimientos y la región? ☐ ☐ ☐ USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 85 Sobre el análisis de la información Si Parcia lment e No Comentarios/observaciones 29. ¿Poseen Salas de Situación, los datos están expuestos, los datos (curvas endémicas y otros) están actualizados? El evaluador debe ver que así sea ☐ ☐ ☐ 30. ¿Quien actualiza los datos periódicamente en la sala de situación ☐ ☐ ☐ 31. ¿Han realizado el análisis de situación de salud de su provincia y sus perfiles epidemiológicos? (pueden mostrarlos) ☐ ☐ ☐ 32. Explíqueme como ha apoyado el proyecto CDEx en el Sistema de Vigilancia Epidemiologica de la Provincia. 33. ¿Cómo han sido capacitado el personal que participa en la Vigilancia epidemiológica? 34. ¿Participa la DPS en la auditoria de las muertes materna? Gestión de Recursos 35. ¿Cuáles son las diferentes fuentes de financiamiento para la provincia y cuales estrategias esta implementando para optimizar la adquisición de recursos financieros? 86 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 36. ¿Cuáles son las brechas más importantes en el manejo financiero? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 37. ¿Cómo esta apoyándoles el proyecto CDEx en manejo financiero? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) Pregunte y profundice sobre las respuestas recibidas # Que evidencia observo? Comentarios/observaciones 38. Capacitación del personal: a) hay un plan de capacitación y una evaluación de las necesidades de capacitación, de acuerdo a los protocolos actualizados, y estrategias e intervenciones prioritarias b) La capacitación se produce en forma aislada, pero no ligada a los protocolos actualizados, y estrategias e intervenciones prioritarias c) Hay un plan de capacitación y esta de acuerdo a los protocolos actualizados, y estrategias e intervenciones prioritarias, pero no se cumple a cabalidad d) Hay un plan de capacitación y evaluaciones continuas de las necesidades de capacitación en base a los protocolos actualizados, y estrategias e intervenciones prioritarias y se cumplen completamente 39. ¿Cuáles capacitaciones ha recibido del proyecto? USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 87 40. ¿ La DPS cuenta con una estructura, procesos y metodologías para la capacitación? 41. ¿La DPS cuenta con personas capacitadas como tutores y multiplicadores de las intervenciones del proyecto de CDEx? 42. ¿Cuáles son las capacitaciones prioritarias para difundir al resto de los establecimientos de salud en la provincia? Redes programáticas de Salud Colectiva 43. ¿Cuáles son las brechas más importantes en la ejecución del rol del rector del sistema de salud? 44. ¿Cómo esta apoyándole el proyecto CDEx para asumir el rol de rectoría? 45. ¿Cuáles son las brechas más importantes en la gestión de los programas de promoción y prevención de la salud y el PAI? ¿Conoce de algún estudios de evaluación de buenas prácticas del PAI realizado en la región __, puede decirnos en que le sirve para su trabajo? 46. ¿Cual la cobertura de Vacuna del PAI, para la Provincia? 88 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 47. ¿Cómo esta apoyándole el proyecto CDEx para mejorar la gestión de estos programas? Plan de Manejo de Emergencias y Desastres 48. ¿Cuáles son las brechas más importantes en la preparación para emergencias y desastres? 49. ¿Cómo está apoyándole el proyecto CDEx en la preparación para emergencias y desastres? (Nota para el entrevistador. Si le menciona la formulación del Plan de Respuesta ante Emergencias y Desastres favor de pedirle que se lo muestre) 50. ¿Cuáles planes tienen para seguir mejorando la preparación para emergencias y desastres? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) Estructura, funciones y capacidad Si Parcial mente No Comentarios/observaciones 51. ¿Hay un plan de manejo de emergencias y desastres? ☐ ☐ ☐ 52. ¿Hay un comité de emergencias funcionando? ☐ ☐ ☐ 53. ¿Hay un vínculo con el COE Provincial? ☐ ☐ ☐ 54. ¿La participación comunitaria está incluida dentro del plan de manejo de emergencias y desastres? ☐ ☐ ☐ USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 89 Proceso de Habilitación 55. ¿Tiene la DPS un comité de Habilitación? 56. ¿Disponen de una guía o protocolo (Instrumentos de Inspección) para el proceso de habilitación? 57. Describa el proceso de reconocer un centro como habilitado 58. Cuántos de los establecimientos de salud en su provincia están habilitados? Lecciones Aprendidas 59. Si tuviera que replicar todo el proceso de Centros de Excelencia ¿Cuáles estrategias no funcionaron y cuales si, y que cambiaria? (Nota para el entrevistador. Anote todo lo que diga) Muchas gracias por su participación 90 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 91 ANNEX 8. ENTREVISTA A INFORMANTES CLAVES: HOSPITALES REGIONALES Y PROVINCIALES EVALUACIÓN DE TÉRMINO MEDIO DEL PROYECTO CENTROS DE EXCELENCIA MATERNO INFANTIL República Dominicana. Abril-Marzo, 2012 Entrevista a Informantes Claves: Hospitales Regionales Y Provinciales La presente guía de entrevistas es para el personal de la los hospitales regionales o provinciales. Por favor seleccione entrevistar aquellas personas del staff que tiene más tiempo en el servicio y conoce sobre el proyecto “Centros de Excelencia Materno-Infantil.” Por favor entreviste a la persona exhaustivamente, sin dejar ninguna pregunta sin contestar. Las preguntas son semi-estructuradas y abiertas, es decir, se deben llenar las respuestas como las describa el entrevistado. En los casos de que sea si o no, serán como filtros para profundizar en el tema. Utilice su experiencia y criterio para profundizar en algunos temas si le parecen interesantes, pida mayor información o evidencia si es necesario. Por favor use hojas sueltas si necesita expandir las respuestas, y adjúntelas con grapas al finalizar la entrevista. Nombre del entrevistador: Fecha: Nombre del entrevistado: Posición/titulo: Distrito/provincia: Pueblo/ciudad: Nombre del hospital regional/provincial: Conocimientos generales acerca del proyecto: 1. Describa brevemente sus responsabilidades e involucramiento con el proyecto sobre CDEx desde mediados del 2009 a la fecha. (Nota para el entrevistador: Profundice sobre su participación y por cuanto tiempo) Selección de los Centros de Excelencia (CDEx) 2. Explique el proceso competitivo que tuvo para ser seleccionado como Centro de Excelencia. 92 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 3. ¿Esta de acuerdo con este proceso? ¿Como lo mejoraría, si tiene que recomendar a sus colegas para que apliquen? Planificación 4. Explique en que sentido cambió o mejoró su estilo gerencial gracias a las intervenciones del proyecto CDEx? (Nota para el entrevistador. Explore sobre los conceptos de gestión para el cambio o Change Management?) 5. ¿Explíqueme cuales fueron las conclusiones de la auto-evaluación que realizó el 2010 y el 2011? Cuales fueron las prioridades que determinó en esos mismos años? ¿Han elaborado memorias de participación al premio de la calidad, pueden mostrarlas? ¿Ha recibido y discutido informes de retorno de esta participación en el premio de la calidad, la ha discutido con su equipo ¿Cuáles fueron los objetivos estratégicos del plan de direccionamiento estratégico? (Nota para el entrevistador. Explore sobre CAF- Common Assessment Framework) 6. ¿Cual fue su plan para la mejora continua de la calidad, también para el 2010 y 2011? (Nota para el entrevistador. Pida que se lo muestre) 7. Pida que le explique: Cuales fueron los objetivos del plan de direccionamiento estratégico? Cuando fue aprobado el plan? Quien/quienes aprueban el plan? (Nota para el entrevistador. Esta actividad usa otra metodología no es el CAF) USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 93 8. Pida que le explique: ¿Cual es periodo del plan estratégico? ¿Cuando fue aprobado el plan? Quien/quienes aprueban el plan? 9. ¿Cuáles estrategias y/o acciones se han realizado para cerrar brechas o debilidades identificadas en su plan operativo? Estructura, funciones y capacidad Si Parcialm ente No Comentarios/observaciones Pregunte sobre los cambios en los últimos 2 años 10. ¿Tiene documentado el plan operativo? Muéstrelo por favour ☐ ☐ ☐ 11. ¿Se usan datos estadísticos (del hospital y su área de cobertura) para la priorización de actividades ☐ ☐ ☐ 12. ¿Incluyen estrategias especificas para la entrega de servicios del hospital? ☐ ☐ ☐ 13. ¿Tiene metas anuales (o el periodo que se usa)? ☐ ☐ ☐ 14. ¿Tiene un calendario de actividades? ☐ ☐ ☐ 15. ¿Tiene un presupuesto? ☐ ☐ ☐ 16. ¿Están en capacidad de desarrollar el próximo plan operativo? ¿Necesitan algún tipo de apoyo? (Nota para el entrevistador: Si es si, o no, por favor elabore y especifique) 94 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Monitoreo, Evaluación y Sistemas de Información 17. ¿Cuáles son las brechas más importantes en la recolección, análisis y uso de información estratégico? 18. ¿Cómo está apoyándole el proyecto CDEx en la recolección, análisis y uso de información estratégico? 19. ¿Cuáles planes tienen para seguir mejorando la recolección, análisis y uso de información estratégico? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) Estructura, funciones y capacidad Si Parcialmente No Puede mostrarlo Si o no Comentarios/observa ciones Pregunte sobre los cambios en los últimos 2 años 20. ¿Hay una estructura organizacional documentada ☐ ☐ ☐ 21. ¿Se dedica todo el equipo a los sistemas de M&E y a manejar la información? La función solo se realiza si existe un equipo? O cada gerente debe manejar la información requerida para su toma decisión? ☐ ☐ ☐ 22. ¿Las funciones las realiza si existe un equipo de M&E, o cada gerente debe manejar la información requerida la toma decisiones? ☐ ☐ ☐ 23. ¿Hay un plan de capacitación en M&E que incluye a todo el personal del servicio? ☐ ☐ ☐ USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 95 Estructura, funciones y capacidad Si Parcialmente No Puede mostrarlo Si o no Comentarios/observa ciones Pregunte sobre los cambios en los últimos 2 años 24. ¿Existe personal asignado a la revisión de la calidad de la información (exactitud, integridad, oportunidad y confidencialidad ) antes de enviar a la región y otros niveles? ☐ ☐ ☐ Definiciones de indicadores y lineamientos para los informes Si Parcia lment e No Puede mostrarl o Si o no Comentarios/observaci ones Pregunte sobre los cambios en los últimos 2 años 25. ¿Se ha documentado y compartido la definición de los indicadores del programa materno-infantil con el personal que trabaja en materno-infantil y genera los informes de progreso? ☐ ☐ ☐ 26. ¿Existe una descripción de las intervenciones de materno-infantil relacionados a cada uno de los indicadores medidos por el programa? ☐ ☐ ☐ 27. ¿Se evalúan periódicamente los indicadores del proyecto y se realiza retroalimentación con el personal del hospital? ☐ ☐ ☐ Formatos y herramientas para recoger información y presentar informes Si Parcial mente No Puede mostrarl o Si o no Comentarios/observaci ones Pregunte sobre los cambios en los últimos 2 años 28. La información recogida a través del sistema de información, ¿es suficientemente precisa para medir el/los indicador/es y desagregados (p.ej. género, edad, etc.)? ☐ ☐ ☐ 29. Realiza su hospital análisis de datos en forma regular? ☐ ☐ ☐ 30. ¿Funciona el sistema informático peri￾natal? ☐ ☐ ☐ 96 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Gestión de Recursos 31. ¿Cuáles son las diferentes fuentes de financiamiento del hospital y cuales estrategias esta implementando para optimizar la adquisición de recursos financieros? (Nota para el entrevistador. ) 32. ¿Cuáles son las brechas más importantes en el manejo financiero? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 33. ¿Cómo esta apoyándoles el proyecto CDEx en manejo financiero? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) 34. ¿Cuáles capacitaciones ha recibido del proyecto? 35. ¿El hospital cuenta con una estructura, procesos y metodologías para capacitar? 36. ¿El hospital cuenta con personas capacitadas como tutores y multiplicadores de las intervenciones del proyecto de CDEx? ¿En que áreas? (Nota para el entrevistador. Especifique en que áreas tiene el personal/tutores) USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 97 37. ¿Cuáles son las capacitaciones prioritarias para continuar en el futuro? Atención Humanizada Observe el ambiente del hospital y llene la siguientes lista Estructura, funciones y capacidad Si Parcial mente No Puede mostrarlo Si o no Comentarios, observaciones 38. ¿Las sillas para los pacientes son adecuadas (cómodas)? ☐ ☐ ☐ 39. ¿El área de espera es cerrada? ☐ ☐ ☐ 40. ¿Hay agua potable? ☐ ☐ ☐ 41. ¿Hay baño o letrina disponible? ☐ ☐ ☐ 42. ¿Hay un área para desechar los desechos? ¿Están clasificados? ☐ ☐ ☐ 43. ¿Hay posters (afiches) con información visible? ☐ ☐ ☐ 44. Si es así ¿El lenguaje es claro de entender? ☐ ☐ ☐ Cuidados a la madre Si Parcial mente No Puede mostrarlo Si o no Comentarios, observaciones 45. ¿El área de consulta tiene privacidad? ☐ ☐ ☐ 46. ¿Cuántos consultorios para control prenatal hay? __________ 47. ¿Cuántos consultorios para control prenatal tienen ventilación? __________ 48. ¿La iluminación es apropiada? ☐ ☐ ☐ 49. ¿Es un ambiente limpio? ☐ ☐ ☐ Atención del Parto: Estadísticas del servicio Anote las siguientes estadísticas: (Nota para el entrevistador: Si el entrevistado no sabe estos datos, pídale al encargado de estadística. Si la información no esta disponible, marque ND) 98 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 2010 2011 Puede mostrarlo Si o no 50. Numero total de pacientes (solo registrados, no el numero de controles) para control pre￾natal 51. Numero de partos vaginales atendidos (normales) 52. Numero de partos vaginales – con complicaciones atendidos 53. Numero de partos por cesárea 54. Numero de muertes maternas 55. Numero de muertes maternas auditadas? 56. Causas de muerte materna: Eclampsia Hemorragia (en cualquier estadio) Infección (en cualquier estadio) Otros ………………… ………………… ………………… ………………… ………………… ………………… ……………… ………………… ………………… ………………… …………… …… …………… …… …………… …… …………… …… …………… …… 57. Numero de muertes neonatales 58. Causa de muertes neonatales Prematuridad Sepsis Asfixias Otros (especifique) ………………… ………………… ………………… ………………… ……………… ………………… …………… …… …………… …… …………… …… 59. Numero de mujeres embarazadas transferidas a otro hospital 60. Numero de parto que se realizo Manejo Activo 61. Numero de partos realizado sin episiotomía 62. Numero de resolución del post-aborto por AMEU 63. Numero de parto que fuero inducidos USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 99 Intervenciones Prioritarias Indica si el hospital está implementado los siguientes intervenciones Si Parcial mente No Comentarios/observaciones Pregunte sobre los cambios en los últimos 2 años 64. Manejo Activo del Tercer Etapa del Parto ☐ ☐ ☐ 65. Cuidados Obstétricos de Emergencia ☐ ☐ ☐ 66. Ayudando a Bebes Respirar (ABR) ☐ ☐ ☐ 67. Madre Canguro ☐ ☐ ☐ 68. Prevención de sepsis neonatal ☐ ☐ ☐ 69. Prevención de Transmisión Vertical de VIH ☐ ☐ ☐ 70. Maternidad centrada en la familia ☐ ☐ ☐ 71. Atención post-aborto/parto ☐ ☐ ☐ Mortalidad materna 72. ¿Hace auditorias sobre muertes maternas? Solicite la documentación (Nota para el entrevistador. Indague sobre las muertes ocurridas en el ultimo trimestre de 2011. Verifique si esta la causa de la muerte, y que medidas se tomaron sobre la atención en el hospital) Mortalidad Infantil 73. ¿Hace auditorias sobre muertes infantiles? (Nota para el entrevistador. Indague sobre las muertes ocurridas en el ultimo trimestre de 2011. Verifique si esta la causa de la muerte, y que medidas se tomaron sobre la atención en el hospital) Control de infecciones 74. ¿Cómo se monitorea las infecciones intrahospitalarias? 100 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 75. ¿Cómo se monitorea el uso racional de antibióticos y medicamentos? 76. ¿Tiene diseñada y señalada la ruta y horas de recoger los desechos? 77. ¿Cual es el nivel de coordinación con el municipio para la recogida y disposición final de los desechos hospitalario? ¿Cómo esta funciona? Articulación entre los diferentes niveles 78. Descríbame por favor el sistema actual de referencia y contra-referencia 79. ¿Como esta apoyándole el proyecto de CDEx en mejorar este sistema de referencia y contra-referencia? 80. Descríbame por favor el sistema actual de traslado de pacientes. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 101 81. ¿Como esta apoyándole el proyecto de CDEx en mejorar este sistema de traslado de pacientes? Habilitación 82. ¿Este establecimiento está habilitado oficialmente? Si no, ¿Qué le falta? 83. ¿Como esta apoyándole (o ha apoyado) el proyecto de CDEx en alcanzar la habilitación? Oficina de Atención al Usuario 84. ¿Cómo ha mejorado el desempeño de la oficina de atención al usuario? Enumere los pasos que han realizado para mejorar las OAU y los Archivos (Nota para el entrevistador. Durante los últimos dos años) 85. ¿Cómo está apoyándole el proyecto CDEx en la atención al usuario? Observe y anote los siguientes componente Si Parcial mente No Comentarios/observaciones Pregunte sobre los cambios en los últimos 2 años 86. ¿Tiene estantería para los expedientes clínicos? (Describa su condición) ☐ ☐ ☐ 87. ¿Están usando el software para la gestión hospitalaria automatizada? ☐ ☐ ☐ 88. Verifique si existe el ultimo reporte del sistema automatizado. ☐ ☐ ☐ 102 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Sistema de Gestión de Suministro 89. ¿Cómo ha mejorado el desempeño del sistema de gestión de suministro? Enumere los pasos que han realizado para mejorar este proceso (Nota para el entrevistador. Durante los últimos dos años) 90. ¿Cómo está apoyándole el proyecto CDEx en la distribución de medicamentos e insumos? Observe y anote los siguientes componente Si Parcial mente No Comentarios/observaciones Pregunte sobre los cambios en los últimos 2 años 91. ¿Tiene estantería para los insumos médicos? ☐ ☐ ☐ 92. Describa las condiciones del ambiente (temperatura, limpieza, no le da la luz del sol directamente) ☐ ☐ ☐ 93. Verifique si hay un control de inventarios ☐ ☐ ☐ Plan de Manejo de Emergencias y Desastres 94. ¿Cuáles son las brechas más importantes en la preparación para emergencias y desastres? 95. ¿Cómo está apoyándole el proyecto CDEx en la preparación para emergencias y desastres? USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 103 96. ¿Cuáles planes tienen para seguir mejorando la preparación para emergencias y desastres? (Nota para el entrevistador. Ponga la situación de que no hay recursos adicionales, solo con los fondos existentes) Estructura, funciones y capacidad Si Parcial mente No Comentarios/observaciones 97. ¿Hay un plan de manejo de emergencias y desastres? ☐ ☐ ☐ 98. ¿La participación comunitaria está incluida dentro del plan de manejo de emergencias y desastres? ☐ ☐ ☐ Participación comunitaria 99. ¿Cuáles estrategias se está implementando para aumentar la participación ciudadana en el mejoramiento del hospital? 100. ¿Como esta apoyándole el proyecto de CDEx en mejorar este proceso de participación ciudadana? Lecciones Aprendidas 101. Si tuviera que replicar todo el proceso de Centros de Excelencia ¿Cuáles estrategias no funcionaron y cuales si, y que cambiaria? (Nota para el entrevistador. Anote todo lo que diga) Agradezca por su participación y termine con la entrevista 104 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 105 ANNEX 9. FORMULARIO DE CONSENTIMIENTO INFORMADO. ENTREVISTAS A USUARIAS DE SERVICIOS MI Y PERSONAL DE SALUD EVALUACIÓN DE MEDIO TÉRMINO: INCREMENTAR EL ACCESO A SERVICIOS MATERNO INFANTILES DE CALIDAD EN ÁREAS SELECCIONADAS DE LA REPÚBLICA DOMINICANA: COMPONENTE CUALITATIVO FORMULARIO DE CONSENTIMIENTO INFORMADO Entrevistas Fecha: / / Saludos: Estamos realizando un estudio para conocer la percepción que tienen las mujeres sobre los diferentes servicios de salud materno-infantil que se ofrecen en este Hospital. Por esto le pedimos participar en una entrevista, a manera de conversación con la entrevistadora, lo cual le tomará aproximadamente media hora. Los temas de la conversación tienen que ver con las experiencias que usted u otras personas que conoce han tenido con los servicios que se ofrecen en este Hospital, incluyendo sus necesidades y sugerencias relacionadas con los mismos. La conversación será grabada en audio para facilitar su posterior procesamiento y análisis. Su participación en este estudio no representa ningún riesgo para usted. Aun así, si alguno de los temas llega a causarle incomodidad, usted no está obligada a opinar al respecto, y tiene derecho a retirarse del estudio cuando lo desee. Le aseguramos que toda la información que usted nos proporcione será manejada de manera confidencial, y sólo como grupo, nunca individual. La grabación de la conversación será identificada solamente con un número, sin incluir nombres ni señas que la identifiquen. Su participación en todo momento debe ser voluntaria. Si bien no tendrá ningún beneficio directo por su participación, los resultados del presente estudio le ayudarán a las autoridades de este Hospital a tomar decisiones y proveer servicios de salud que mejor se ajusten a las necesidades de las mujeres que los utilizan. Si ha incurrido en algún gasto de transporte para llegar hasta acá, este le será rembolsado por nuestro equipo. Este estudio no forma parte de los servicios ofrecidos por este Hospital ni por cualquier otra institución. Por lo tanto, usted no pierde ninguno de sus derechos vinculados a estos servicios por negarse a participar en ella. Si decide ayudarnos, le pedimos responder con la mayor sinceridad a cada una de las preguntas, y se sienta libre de expresarse. Si tiene alguna observación o pregunta sobre este estudio o su participación en el mismo, puede hacerla ahora o, si así lo prefiere, puede contactar más adelante al Dr. Marcelo Castrillo, en el teléfono 829-873-8272. Usted recibirá una copia de este formulario de consentimiento para su uso personal. Si desea puede consultarlo con familiares, otros profesionales o amigos antes de firmarlo. 106 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Si está de acuerdo con participar en la entrevista, bajo estas condiciones, por favor, coloque su nombre y firma en la línea señalada más abajo, para otorgar su consentimiento informado. Esto indica que ha decidido ser voluntario/a en el estudio, y que entiende el objetivo del mismo y sus derechos. Nombre del/a entrevistador/a Nombre de la participante Firma del/a entrevistador/a Firma de la participante Si la participante es menor de 18 años y/o no sabe leer y escribir ► Nombre del/a testigo Firma del/a testigo USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 107 ANNEX 10. TEMARIO/GUÍA DE GRUPOS FOCALES: PROVEEDORES DE SERVICIOS DE SALUD EVALUACIÓN DE MEDIO TÉRMINO: INCREMENTAR EL ACCESO A SERVICIOS MATERNO INFANTILES DE CALIDAD EN ÁREAS SELECCIONADAS DE LA REPÚBLICA DOMINICANA Componente Cualitativo (Grupo 3) Temario / Guía de Grupos Focales Proveedores de Servicios de Salud Actividades Preliminares Llenado de la ficha de participantes FASE I. Presentación del grupo / introducción Presentación del facilitador / co-facilitador del grupo. Presentación / explicación de los objetivos de la discusión. Presentación de los participantes en el grupo Reglas de discusión (confidencialidad, participación equitativa, respeto mutuo, etc.) FASE II. Desarrollo / Discusión de tópicos 1. Percepción general de la calidad de los servicios de salud materno-infantil que ofrece en el Centro, tomando en cuenta: (NOTA: en el caso de referirse a un servicio específico, como el Programa de Madres Canguro, orientar las preguntas a este servicio). – Condiciones de la planta física (amplitud, higiene, adecuación, etc). – Disponibilidad de equipos, materiales y suministros requeridos para la entrega de servicios. – Manejo de normas y protocolos clínicos (diseño de protocolos, adecuación al contexto, familiaridad del personal con su existencia / contenidos, mecanismos de control, resultados percibidos, etc.) – Bioseguridad y respeto al ambiente (Manejo de desechos, flujo de residuos, depósitos intermedios, vacunación del personal, etc.). – Preparación del Hospital frente a posibles situaciones de desastre (rutas de evacuación, simulacros, Comité de Riesgo, etc.) – Preparación y actitud de los recursos humanos involucrados en estos servicios. – Desarrollo de actividades docentes / formación continua. Explorar el Centro de Capacitaciones y su funcionamiento en el Hospital. – Actividades de IEC dirigidas a las usuarias (contexto, temas, personal encargado, etc.) – Unidad de Atención a la Usuaria (funcionamiento actual, servicios provistos, etc.) – Integración y unificación de esfuerzos entre diferentes Programas. 108 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 – Sistema de referencia y contra-referencia (interacción con centros de otro nivel de atención, traslado de pacientes, etc.) – Principales obstáculos que encuentra el Hospital / proveedores de servicios en estas áreas. – Partidas presupuestarias asignadas a estos servicios, en función de las necesidades sentidas. – Estrategias a emplear para mejorar la calidad de estos servicios. 2. Integración comunitaria – ¿Existe algún Comité Comunitario que se integra al trabajo de este Hospital? ¿Funciona? ¿Cómo se vincula ese Comité al trabajo del Hospital? ¿Qué hace? ¿Cuáles son las fortalezas y debilidades de este Comité? – ¿Funciona en el Hospital algún Comité de Ética? ¿De qué tipo? Describa su funcionamiento, fortalezas y debilidades. Integra el Comité de Ética a algún representante de la Comunidad. Describa. – ¿Se integra algún miembro de la Comunidad al Consejo Administrativo del Hospital? Describa el proceso de su reclutamiento, entrenamiento y la participación que tiene en el Consejo. – ¿Se le explica a las usuarias sobre las condiciones del hospital, sus programas y estado financiero? ¿Considera necesario darles estas explicaciones? ¿Por qué? 3. Proyectos de USAID implementados en el Hospital – A lo largo de los últimos dos años, ¿ha notado alguna diferencia en la provisión de servicios de salud materno-infantil en este Hospital? ¿En qué sentido? ¿A qué atribuye las diferencias observadas? – ¿Se ha implementado algún Proyecto / Programa que ha incidido de manera importante en la calidad de servicios en este Hospital? ¿Cuál? ¿Con qué financiamiento? – Se ha implementado alguna intervención financiada por USAID en este Hospital en los últimos años. ¿Cuál(es)? ¿En qué ha(n) consistido? – ¿Cómo se han diseñado estas intervenciones? ¿Quién(es) participaron en la toma de decisiones al respecto? Explique. – ¿Cómo evaluaría el impacto de las intervenciones financiadas por USAID en la calidad de servicios de salud materno-infantil en este Hospital? ¿Cuál destacaría como particularmente exitoso / poco exitoso? Explique cada caso. – Especifique las fortalezas y las debilidades de las intervenciones auspiciadas por USAID en este Hospital. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 109 – ¿Qué tipo de cosas podría hacerse para optimizar el impacto de estas intervenciones en el futuro? ¿Cuáles serían las necesidades del Hospital no cubiertas por estas intervenciones? – Una vez no esté disponible el financiamiento del USAID, ¿cuáles son las perspectivas de que se mantenga el impacto de estas intervenciones en el Hospital? ¿Por qué? Identifique los factores que contribuyen / dificultan la sostenibilidad de estas intervenciones. 4. Evaluación general / sugerencias relativas a servicios de salud materno-infantil en el Centro – Luces y sombras de los servicios de salud materno-infantil que se ofrecen en este Centro de salud. – ¿Recomendaría los servicios que ofrece este Hospital a una amiga o familiar? ¿Por qué? – Sugerencias con respecto a cómo mejorar los servicios en un futuro / posible papel de USAID. FASE III. Cierre de la discusión – Ofrecer la oportunidad de agregar comentarios sobre temas adicionales a las participantes. – Asegurar con la co-facilitadora que no haya preguntas pendientes. – Resumen de la discusión / agradecimiento. 110 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 111 ANNEX 11. TEMARIO/GUÍA DE GRUPOS FOCALES USUARIAS DE SERVICIOS DE SALUD EVALUACIÓN DE MEDIO TÉRMINO: INCREMENTAR EL ACCESO A SERVICIOS MATERNO INFANTILES DE CALIDAD EN ÁREAS SELECCIONADAS DE LA REPÚBLICA DOMINICANA Componente Cualitativo (Grupo 3) Temario / Guía de Grupos Focales Usuarias de Servicios de Salud Actividades Preliminares Consentimiento informado / Llenado de la ficha de participantes Entrega de gastos de transporte FASE I. Presentación del grupo / introducción Presentación del facilitador / co-facilitador del grupo. Presentación / explicación de los objetivos de la discusión. Presentación de los participantes en el grupo. Reiteración de los contenidos del Consentimiento Informado / reglas de discusión. FASE II. Desarrollo / Discusión de tópicos 1. Experiencia de las usuarias con servicios de salud materno-infantil en el Centro de Salud – Servicios que conocen / han recibido / están recibiendo actualmente; – Motivos para seleccionar este Centro (distancia, costo, costumbre, instalaciones, personal, etc.; explorar el contraste con los establecimientos privados); – Disposición a recibir atención en el mismo Centro en un embarazo futuro; – Preferencias en cuanto a otros servicios que les interesaría recibir en el mismo Centro. 2. Percepción y satisfacción con los servicios de salud materno-infantil recibidos en el Centro (NOTA: en el caso de referirse a un servicio específico, como el Programa de Madres Canguro, orientar las preguntas a este servicio). – Facilidades para recibir los servicios (distancia, procedimientos, tiempo de espera, etc.); – Gastos en los que tenían que incurrir para recibir la atención (pasaje, pago servicios, etc.); – Limpieza y comodidad de instalaciones (salas de espera, internamiento, baños, agua corriente, etc.); – Disponibilidad de equipos, medicamentos y otros materiales; 112 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 – Preparación y experiencia del personal que provee la atención (especificar servicio y tipo de personal); – Trato del personal (especificar servicio y tipo de personal); – Confianza con los/as médicos (explicación de servicios, empatía, posibilidades de hacer preguntas, consentimiento informado en diferentes procedimientos, etc.); – Privacidad y confidencialidad de la información suministrada; – Principales obstáculos para recibir servicios en el Centro / posibles formas de superarlos. 3. Experiencias específicas en el proceso de recibir servicios de salud materno￾infantil – Cuánto tiempo después de parto le acercaron a su niño. ¿Qué le pareció eso? – Cómo manejaron el tema de lactancia materna después del parto. ¿Qué le pareció eso? – Antes de salir del hospital, ¿le explicaron cuándo volver? Le dieron todas las vacunas que necesitaba su niño? ¿Le hablaron sobre planificación familiar? – ¿Alguna ha tenido algún problema en su último embarazo, parto o con su bebe recién nacido? ¿Alguna amiga / conocida suya lo ha tenido? ¿Qué tipo de problema? ¿Cómo lo manejaron? ¿Qué le pareció eso? – Si tiene alguna pregunta o duda sobre cómo funciona algún servicio en el Hospital, ¿a dónde se dirige? ¿Recibe siempre la información que necesita? – Si tiene alguna queja relativa a los servicios de este Hospital, ¿sabe a quién dirigirla? Conoce a alguien que lo ha hecho? ¿Cuál ha sido la reacción del personal de salud frente a su reclamo? ¿Cómo describiría esa experiencia? 4. Informaciones recibidas en el Centro en el contexto de servicios de salud materno-infantil – Cuidados prenatales (explorar signos de alarma, alimentación, preparación al parto, involucramiento de la pareja, etc.) – Cuidados post-parto (explorar signos de alarma, visitas post-natales, espaciamiento embarazos, etc.) – Cuidados infantiles (explorar lactancia materna, alimentación infantil, estimulación temprana, etc.); – Métodos de planificación familiar; – Infecciones de transmisión sexual / VIH y SIDA; – Otros temas de relevancia / interés personal. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 113 5. Participación comunitaria – ¿Conoce algún Comité en su comunidad que se integra al trabajo de este Hospital? ¿De otro Hospital? – ¿Cómo se vincula ese Comité al trabajo del Hospital? ¿Qué hace? – ¿Participa alguna de ustedes en algún Comité de este tipo? ¿Cuáles son sus funciones? – ¿Le han explicado alguna vez sobre las condiciones del hospital, sus programas y estado financiero? ¿Quién, cuándo, dónde? Si la respuesta es no, ¿le gustaría que alguien lo haga? ¿Por qué? 6. Evaluación de la experiencia / sugerencias relativas a servicios de salud materno￾infantil en el Centro – Luces y sombras de los servicios recibidos en este Centro de salud. – Servicios que le gustaron más / menos. ¿Por qué? – Si tiene más de dos años acudiendo a este Centro, ¿ha notado alguna diferencia en los servicios provistos? ¿En qué sentido? ¿A qué atribuye las diferencias observadas? – ¿Recomendaría estos servicios de atención prenatal / parto a una amiga o familiar? ¿Por qué? – Sugerencias con respecto a cómo mejorar los servicios recibidos. FASE III. Cierre de la discusión Ofrecer la oportunidad de agregar comentarios sobre temas adicionales a las participantes. Asegurar con la co-facilitadora que no haya preguntas pendientes. Resumen de la discusión / agradecimiento. 114 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 115 ANNEX 12. GUÍA DE OBSERVACIÓN CUALITATIVA EVALUACIÓN DE MEDIO TERMINO DEL RESULTADO 4 DE USAID REPÚBLICA DOMINICANA: INCREMENTAR EL ACCESO A SERVICIOS MATERNO-INFANTILES DE CALIDAD EN ÁREAS SELECCIONADAS Guía de observación cualitativa (Reportada por cada centro de salud visitado, con referencia a espacios dedicados a cuidados prenatales, atención al parto, post-parto, post-aborto) Tópico Aspecto Descripción 01. Impresión a. General Detalles más prominentes / primera impresión / ambiente general del Centro. 02. Instalaciones físicas a. Higiene Impresión visual, olores, insectos, prácticas / frecuencia de limpieza. b. Adecuación Condiciones de amplitud, ventilación, temperatura, colores, iluminación, ruido, etc., tomando en cuenta el uso del espacio. c. Servicios básicos Instalaciones y disponibilidad de agua y luz, manejo de desechos, según las funciones del espacio. d. Equipamiento Muebles / equipos / materiales requeridos en cada espacio (presencia, cantidad, condiciones, funcionalidad, etc.). e. Centro de Capacitaciones Existencia, espacio físico, utilización, acceso del personal, equipamiento, etc. f. Privacidad Posibilidades de aislamiento acústico / visual, manejo del personal de salud (con usuarias y entre sí), manejo de registros médicos. 03. Entrega de servicios a. Procedimientos Condiciones establecidas para recibir servicios, claridad de procedimientos para las usuarias, costos, flujograma, tiempo de espera, tiempo de consulta, etc. b. Personal Horarios, disponibilidad, preparación, experiencia, etc. c. Material educativo Presencia, disponibilidad, calidad y uso de afiches, brochures u otro material educativo en los diferentes espacios. d. Bioseguridad Recursos disponibles, prácticas del personal, demandas de usuarias/as, etc. e. Confidencialidad Reglamentos institucionales y manejo de la información confidencial por parte del personal / usuarias, manejo de registros médicos, etc. 04. Usuarias de servicios a. Perfil general Apariencia, estado afectivo, procedencia, acompañantes, estimado del nivel educativo y socio-económico, comportamiento en los diferentes espacios, etc. 116 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Tópico Aspecto Descripción b. Interacción con el personal de salud Actitud general, contacto visual, gestos, tono de voz, empoderamiento personal, demandas y reclamos expresados, etc. c. Interacción entre usuarias Actitud general, contacto visual, gestos, tono de voz, temas de conversación entabladas, patrones de asociación, niveles de confianza, identificación con el grupo, apoyo mutuo, etc. 05. Personal de salud a. Perfil general Apariencia, estado afectivo, preparación para las funciones desempeñadas, actitud en el trabajo, etc. b. Interacción con las usuarias Atención brindada, vocación de servicio, cordialidad, orientaciones provistas, gestos, contacto visual, etc. c. Interacción entre el personal de salud Actitud general, contacto visual, gestos, tono de voz, temas de conversación entabladas, patrones de asociación, niveles de confianza, identificación con el grupo, apoyo mutuo, etc. d. Trato de personas y grupos estigmatizados Actitudes, procedimientos y trato especial de personas de procedencia haitiana, PVVS, personas pobres, usuarias por complicaciones de aborto, etc. 06. Comentarios espontáneos a. Instalaciones físicas Comentarios relativos a tópicos y aspectos señalados en temas 01–05, emitidos por usuarias de servicios, acompañantes, personal de salud, etc. b. Entrega de servicios c. Usuarias de servicios d. Personal de salud e. Otros temas relevantes Planificación familiar, anticoncepción, VIH y SIDA, etc. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 117 ANNEX 13. GUIDELINES FOR OBSERVER TEAMS 2011 RTF COLLABORATIVE SBA PROJECT LAC Region Background1 The Regional Task Force (RTF) on maternal-newborn health is undertaking a rapid assessment of skilled birth attendants (SBA) in selected countries in Latin America and the Caribbean region. You have been selected as an expert clinician (obstetrician or midwife) to observe SBAs in a variety of settings and countries. Five different Observation Tools have been prepared to guide your work on this project over the next few weeks. The five (5) tools are: 1. Demographic Form 2. Antenatal First Visit Form 3. Antenatal Repeat Visit Form 4. Labor Observation Form 5. Birth/Immediate Postpartum & Newborn Form Directions for the use of each are as follows. The completion of these tools will come once you have discussed the project and selected the most likely health worker to observe who is considered an SBA in that setting. Demographic Form Begin by reading through the entire form and ask methodology trainer any questions that arise. The more familiar you are with the form, the easier it will be to complete. Fill in date, province, district (if appropriate), and country. Check the number of the facility type that most closely fits the description of the facility where you are observing. Each individual health worker you observe requires a participant ID number. You will create the ID number using the first two boxes for the country code as noted on the attached sheet and the next three numbers for the facility code (leave blank as they will be filled in by Joyce Thompson once completed forms are received by her). The next three boxes are filled in by you, beginning with 001, 002, 003 depending on how many SBAs you observe in a specific setting. Each individual health worker you observe needs a completed demographic form by the time you complete your observation. The first part of the form includes permission to observe and assignment of Participant ID that needs to be completed before you start observing them. You will complete the demographic form with the health worker after the observation is finished. 1 These draft questionnaires, March 2012 version, were used with the kind permission of the LAC Regional Maternal Mortality Reduction Task Force. 118 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Place the Participant ID number on the Demographic Form and each page of the observation tool you are using with that health worker. Fill in your name under Observer Name. If two are doing same observation, write down both names. Read through the first page of Instructions to observer/evaluator and be very familiar with the paragraph soliciting verbal informed consent. Note that instructions for how to fill in a particular item on the demographic form are written in the columns corresponding to that item. Some instructions require circling the response(s) that apply, others require filling in blanks. Item 009 on the demographic form requires you to read the introduction as written, and then ask each item 01-09 individually, recording a Yes or No response in each column. Before thanking the health worker, review the demographic form and make sure each item has been completed. Antenatal First Visit Form Begin by reading through the entire form and ask methodology trainer any questions that arise. The more familiar you are with the form, the easier it will be to complete. Write the Participant ID number from the Demographic Form just completed for this observation on the top of each page of this Observation Tool. You already obtained verbal consent to observe from the health worker when completing the Demographic Form. Read through Instructions to Observer(s) and note that you are being asked to be selective in what you record from your observation of this first antenatal visit. You will also review the clinical record at the end of the observation visit and then offer your expert judgment with rationale on the quality of care given and the competence of the health worker observed. You may wish to review an antenatal record used in an individual setting in advance to become familiar with it or to determine if the CLAP clinical record content is being used. Remember that you have selected one health worker with ID number (no names, please) to observe and are only required to list any other category of worker who may have been incidentally involved in the antenatal visit; e.g., person who takes blood pressure or weighs the patient. The criteria for recording your observations are listed next as Evaluation of performance. Note that you will need to be familiar with WHO standards as well as country standards, if they exist, in order to judge whether the health worker completed the task or not in keeping with clinical standards. The N/O response is reserved for when you notice that the health worker did not perform an expected task or activity. If a specific item is not applicable to a given situation, use the N/A (not applicable) response. Record the time the observation started using the 24-hour clock; i.e., 0800 or 1400. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 119 Write down brief notes by a specific item that you want to follow up on with the health worker at the end of the observation. Item 112: Using your notes on specific items, proceed to question the health worker and write down responses. Item 113: This is a continuation of questioning with particular emphasis on “Was there anything in the facility that kept you from doing Y?” Item 114: This is time for the health worker to ask you anything about the observation. Remember you are NOT there to provide clinical consultation, so need to refer those questions to a senior member of staff if available. Record time that antenatal visit ended to determine how long the health worker spent with the pregnant woman. Record the time the observation ended after you thanked the health worker for allowing the observation and completed the demographic form. Request the clinical record before the health worker leaves your area so that you may confirm the accuracy and completeness of the health worker’s charting under Item 111. This clinical record review may also give you additional insight for your judgment of health worker competence and the quality of care given. Items 115-116: Circle the best response to OBSERVER RATINGS. Item 117: Explain your justification (rationale) for each rating given in items 115-116. Note that “Needs improvement” under health worker competency implies that the person is competent in what you have observed, but could do better or be more timely in their caring. Item 118: There are often unexpected events that could interfere with providing antenatal care in a given setting. Please record anything that may have happened while you were there. Note: This is not related to lack of equipment or medications as this information should have been noted by the health worker in Item 113. Review the form for completeness before going on to the next observation. Antenatal Repeat Visit Form Begin by reading through the entire form and ask methodology trainer any questions that arise. The more familiar you are with the form, the easier it will be to complete. Write the Participant ID number from the Demographic Form just completed for this observation on the top of each page of this Observation Tool. You already obtained verbal consent to observe from the health worker when completing the Demographic Form. Read through Instructions to Observer(s) and note that you are being asked to be selective in what you record from your observation of this repeat antenatal visit. You will also review the clinical record at the end of the observation visit and then offer your expert judgment with rationale on the quality of care given and the competence of the health worker 120 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 observed. You may wish to review an antenatal record used in an individual setting in advance to become familiar with it or to determine if the CLAP clinical record content is being used. Remember that you have selected one health worker with ID number (no names, please) to observe and are only required to list any other category of worker who may have been incidentally involved in the antenatal visit; e.g., person who takes blood pressure or weighs the patient. The criteria for recording your observations are listed next as Evaluation of performance. Note that you will need to be familiar with WHO standards as well as country standards, if they exist, in order to judge whether the health worker completed the task or not in keeping with clinical standards. The N/O response is reserved for when you notice that the health worker did not perform an expected task or activity. If a specific item is not applicable to a given situation, mark the N/A column. Record the time the observation started using the 24-hour clock; i.e., 0800 or 1400. Write down brief notes by a specific item that you want to follow-up on with the health worker at the end of the observation. Item 138: Pre-referral treatments are marked N/A if none of these items 01-05 happen while you are observing. However, if the complications do arise, fill in accordingly or if in a referral setting, fill in both Items 138 and 139. Item 139: Fill in only if observing in a referral setting and the conditions are observed. Item 141: Using your notes on specific items, proceed to question the health worker and write down responses. Item 142: This is a continuation of questioning with particular emphasis on “Was there anything in the facility that kept you from doing Y?” Item 143: This is the time for the health worker to ask you anything about the observation. Remember you are NOT there to provide clinical consultation, so need to refer those questions to a senior member of staff if available. Record the time that the antenatal visit ended to determine how long the health worker spent with the pregnant woman. Record the time the observation ended after you thank the health worker for allowing the observation and completing the demographic form. Request the clinical record before the health worker leaves your area so that you may confirm the accuracy and completeness of the health worker’s charting under Item 140. This clinical record review may also give you additional insight for your judgment of health worker competence and the quality of care given. Items 144-145: Circle the best response to OBSERVER RATINGS. Item 146: Explain your justification (rationale) for each rating given in items 144-145. Note that “Needs improvement” under health worker competency implies that the person is USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 121 competent in what you have observed, but could do better or be more timely in their caring. Item 147: There are often unexpected events that could interfere with providing antenatal care in a given setting. Please record anything that may have happened while you were there. Note: This is not related to lack of equipment or medications as this information should have been noted by the health worker in Item 143. Review the form for completeness before moving on to the next observation. Labor Observation Form Begin by reading through the entire form and ask methodology trainer any questions that arise. The more familiar you are with the form, the easier it will be to complete. Write the Participant ID number from the Demographic Form just completed for this observation on the top of each page of this Observation Tool. You already obtained verbal consent to observe from the health worker when completing the first section of the Demographic Form. Read through Instructions to Observer(s) and note that you are being asked to be selective in what you record from your observation of this laboring woman (women). You will also review the clinical record at the end of the observation visit and then offer your expert judgment with rationale on the quality of care given and the competence of the health worker observed. You may wish to review a labor record (partograph form) used in an individual setting in advance to become familiar with it or to determine if the CLAP clinical record content is being used. Remember that you have selected one health worker who is responsible for managing laboring women with ID number (no names, please) to observe and are only required to list any other category of worker who may have been incidentally involved in the care of the laboring woman visit; e.g., person who takes blood pressure or prepares pain medications. See Observer note about how to record observations when the health worker is responsible for managing the labor of more than one woman at the same time. Use the ‘a,b,c’ designation to reflect more than one laboring woman cared for by the health worker you are observing. The criteria for recording your observations are listed next as Evaluation of performance. Note that you will need to be familiar with WHO standards as well as country standards, if they exist, in order to judge whether the health worker completed the task or not in keeping with clinical standards. The N/O response is reserved for when you notice that the health worker did not perform an expected task or activity. If a specific item is not applicable to a given situation, mark the N/A column. Record the time the observation started using the 24-hour clock; i.e., 0800 or 1400. Write down brief notes by a specific item that you want to follow up on with the health worker at the end of the observation. 122 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Item 210: Space is left for your notes of any actions that needed to be taken and were taken by the health worker. Item 211: This is where you record information once the observation is over and you are reviewing the contents of the clinical record that the health worker charted. Item 212: Using your notes on specific items, proceed to question the health worker and write down responses. Item 213: This is a continuation of questioning with particular emphasis on “Was there anything in the facility that kept you from doing Y?” Item 214: This is the time for the health worker to ask you anything about the observation. Remember you are NOT there to provide clinical consultation, so need to refer those questions to a senior member of staff if available. Record the time the observation ended after you thank the health worker for allowing the observation. Request the clinical record before the health worker leaves your area so that you may confirm the accuracy and completeness of the health worker’s charting under Item 211. This clinical record review may also give you additional insight for your judgment of health worker competence and the quality of care given. Items 215-216: Circle the best response to OBSERVER RATINGS. Item 217: Explain your justification (rationale) for each rating given in items 216-217. Note that “Needs improvement” under health worker competency implies that the person is competent in what you have observed, but could do better or be more timely in their caring. Item 218: There are often unexpected events that could interfere with providing antenatal care in a given setting. Please record anything that may have happened while you were there. Note: This is not related to lack of equipment or medications as this information should have been noted by the health worker in Item 213. Review the form for completeness before moving on to the next observation. Birth/Immediate Postpartum and Newborn Observation Form Begin by reading through the entire form and ask methodology trainer any questions that arise. The more familiar you are with the form, the easier it will be to complete. Write the Participant ID number from the Demographic Form just completed for this observation on the top of each page of this Observation Tool. If this observation is a continuation to birth with the same health worker that managed the labor of the woman, use the same ID number. If the health worker being observed is a new person, you will need to go through the Demographic Form process first. You already obtained verbal consent to observe from the health worker when completing the first part of the Demographic Form. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 123 Read through Instructions to Observer(s) and note that you are being asked to be selective in what you record from your observation of this birth and immediate care of mother and newborn. You will also review the clinical record at the end of the observation visit and then offer your expert judgment with rationale on the quality of care given and the competence of the health worker observed. You may wish to review a labor and birth record (partograph form) used in an individual setting in advance to become familiar with it or to determine if the CLAP clinical record content is being used. Remember that you have selected one health worker who is responsible for attending the birth and immediate post-birth care with ID number (no names, please) to observe and are only required to list any other category of worker who may have been incidentally involved in the care of the birthing woman and her newborn; e.g., person who prepares oxytoxic or provides newborn and maternal care after 15 minutes. See Note about how to record observations when the health worker who attends the birth does NOT take care of newborn or postpartum woman during the first hour after birth. You can list the other health worker and care provided on the front of the Observation Tool, and use designation of ‘a,b,c’ on the actual items the other person performed. The criteria for recording your observations are listed next as Evaluation of performance. Note that you will need to be familiar with WHO standards as well as country standards, if they exist, in order to judge whether the health worker completed the task or not in keeping with clinical standards. The N/O response is reserved for when you notice that the health worker did not perform an expected task or activity. If a specific item is not applicable to a given situation, mark the N/A column. Record the time the observation started using the 24-hour clock; i.e., 0800 or 1400. Write down brief notes by a specific item that you want to follow up on with the health worker at the end of the observation. Item 232: Space is left for you to write down the time action line reached on the partograph as well as the time the health worker took action. If the action line was NOT reached, leave response columns blank. Item 233: This item will require N/A response if no need for action. Item 238: This item has three actions that should not be taken. If the health worker does one of these items you need to mark the No column. Items 241-243: These items reflect globally accepted detailed procedure for active management of the third stage of labor (AMTSL) and each step included. Items 249 and 253-254 may not occur during the observation so you will use N/A response if mother and baby are healthy. Item 255: This is where you record information once the observation is over and you are reviewing the contents of the clinical record that the health worker charted. Item 256: Using your notes on specific items, proceed to question the health worker and write down responses. 124 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Item 257: This is a continuation of questioning with particular emphasis on “Was there anything in the facility that kept you from doing Y?” Item 258: This is the time for the health worker to ask you anything about the observation. Remember you are NOT there to provide clinical consultation, so need to refer those questions to a senior member of staff if available. Record the time the observation ended after you thank the health worker for allowing the observation. Return to complete the rest of the Demographic Form with the health worker. Request the clinical record before the health worker leaves your area so that you may confirm the accuracy and completeness of the health worker’s charting under Item 255. This clinical record review may also give you additional insight for your judgment of health worker competence and the quality of care given. Items 259-260: Circle the best response to OBSERVER RATINGS. Item 261: Explain your justification (rationale) for each rating given in items 259-260. Note that “Needs improvement” under health worker competency implies that the person is competent in what you have observed, but could do better or be more timely in their caring. Item 262: There are often unexpected events that could interfere with providing antenatal care in a given setting. Please record anything that may have happened while you were there. Note: This is not related to lack of equipment or medications as this information should have been noted by the health worker in Item 257. Review the form for completeness before moving on to the next observation. Note: You will have already filled out the first part of the Demographic Form for this observation and need to make sure the participant number filled in on that form matches the participant number on this form. Assign a new participant number with a new Demographic Form completed if the health worker attending the birth is NOT the same person observed during the labor assessment. Instructions to observer(s): As an experienced clinician, you will take in many aspects of both health worker behaviors/actions as well as critical aspects of the environment of care. However, you will record for this rapid assessment only the observations detailed below. In addition to completing the Demographic Form and specific observations below, please do the following: 1. Write down questions related to the specific observations you make to guide your discussion of care given with the health worker at the end of the observation period. If there is no time to talk with the health worker, include your questions on the Observation Form without responses with notes as to why you wanted to ask such questions. 2. At the end of the observation, review the birthing woman’s clinical record (chart) related to this observation to determine the completeness of care given, the accuracy of recorded findings, and the appropriateness of the plan for follow-up care. [Fill in question # 255 related to review of clinical record.] USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 125 3. After discussion of the observation with the health worker and review of the birthing woman’s record, record your clinical judgment on both the quality of care given and the competency demonstrated by the health worker in this situation. 4. Note whether the setting is using the CLAP perinatal clinical record and the new neonatal record. Yes ___________ No _____________ 5. If appropriate, ask why the setting is NOT using the CLAP clinical records. 6. Remember that you are recording only the selected observations on this tool that are a subset of what the health worker(s) will be doing. BIRTH AND IMMEDIATE POSTPARTUM-NEWBORN PERIODS [Note: Some of tasks may be performed simultaneously or by more than one health worker in a setting, If this happens, note the category of health worker performing the specific task on the observation tool at the end of the tool. Record only the actions of the primary health worker attending the birth of the baby.] If other health workers did a specific task during this visit, such as giving oxytocin or newborn care, please list that category of health worker and what they did directly on the observation form. Evaluation of Performance: Mark the corresponding box with an “X” according to the following definitions Yes Task or activity is performed satisfactorily in compliance with country and/or WHO standards No Task or activity is performed unsatisfactorily – not in compliance with country and/or WHO standards N/O Did not perform expected task or activity during the observation N/A Not applicable during this observation Time observation started: __________________ Use 24-hour clock to record time Record day of week observation took place ___________________ No. Knowledge, Skills, Behavior (task) Yes No N/O N/A 231 Treat the mother and newborn with respect at all times 232 Birth imminent Yes No N/O N/A 01-if action line reached, note time ________ 02-no partograph filled out 233 What definitive action was taken? Yes No N/O N/A 01- prepared for normal birth 02- consulted with specialist if needed 126 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 No. Knowledge, Skills, Behavior (task) Yes No N/O N/A 03-prepared for assisted delivery 04-prepared for C-section 05-other action – explain 234 Puts on clean, protective clothing in preparation for birth that protects face, hands, and body from contact with body fluids Yes No N/O N/A 235 Prepares uterotonic drug to use for active management of third stage of labor (AMTSL) Health worker: _________________ Yes No N/O N/A 236 Wears high-level disinfected or sterile gloves for vaginal examinations and delivery Yes No N/O N/A 237 Prepares for immediate care of the newborn Yes No N/O N/A 01-suction bulb 02-dry/warm blankets 238 Safely attends birth Yes No N/O N/A 01-clean technique 02-supports infant body as delivered 03-used vacuum/forceps only on indication 04-DOES NOT use fundal pressure 05-DOES NOT cut episiotomy if not indicated 06-DOES NOT clear infant nose with head on perineum in absence of meconium fluid 239 Clears newborn airway once body delivered Yes No N/O N/A 240 Immediate newborn care Health worker category: ______ Yes No N/O N/A 01- checks newborn’s respirations and heart rate 02- dries infant & covers infant with warm, dry blanket 03- Places infant on mother’s chest, skin to skin, and covers both for warmth 04- takes newborn temperature before infant leaves the delivery area 241 Procedure: Active Management of the Third Stage of Delivery Yes No N/O N/A 01-palpates the abdomen and rules out presence of another fetus before continuing. 02-administers 10 units of IM oxytocin immediately after birth of infant. If oxytocin is not available, administers 0.2 mg of ergometrine (NOT in preeclamptic/eclamptic women) or prostaglandins p.o. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 127 No. Knowledge, Skills, Behavior (task) Yes No N/O N/A 03-clamps and cuts the umbilical cord (clamps near the perineum) when cord stops pulsating. 04-with one hand, maintains slight tension on the cord and waits for a uterine contraction if placenta not detached from uterine wall 05-applies controlled traction to the cord so as to deliver the placenta: Pulls gently, firmly, and uniformly while applying counter pressure above the pubis to prevent uterine inversion. 06- slowly delivers the placenta, supporting it with both hands. Extracts the membranes gently with lateral movements. 242 Checks to see if the placenta & membranes are whole and intact. Yes No N/O N/A 243 After inspection of the placenta and membranes, immediately massages the uterine fundus through the abdomen until the uterus contracts. Ceases to massage uterus after confirming that the uterus is firm (contracted). Yes No N/O N/A 244 Carefully examines the woman and repairs cervical or vaginal tears, or repairs the episiotomy. Yes No N/O N/A No. Immediate Care of PP Woman Health Worker Category __________ Yes No N/O N/A 245 Monitors maternal fundus/bleeding every 15 minutes during first 2 hours post-birth 246 Promotes maternal-infant bonding and immediate, exclusive breastfeeding 247 Monitors maternal and newborn status every 15 minutes for the first 2 hours post-birth 248 Performs eye prophylaxis for newborn 249 Manages severe postpartum bleeding if present Yes No N/O N/A 01-vigorous massage of uterus 02-IV fluids with uterotonic drug 03-bimanual compression of uterus 04-blood transfusion on indication 05-aortic compression on indication 250 Orders syphilis test and treats if indicated Yes No N/O N/A 251 Care of preterm/low birth weight infant without breathing problems 252 If in first level facility, referral of newborn with signs of severe illness, injury, or malformation 253 If in referral facility, manage care of newborn with severe problems Health Worker category __________ Yes No N/O N/A 01-preterm baby with breastfeeding problems 128 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 No. Knowledge, Skills, Behavior (task) Yes No N/O N/A 02-severe infection 03-severe birth asphyxia 254 If in referral facility Health worker category: _________ Yes No N/O N/A 01-perform C-section on indication 02-hysterectomy on indication 03-blood transfusion for severe anemia 255 Charting complete and accurate 01-noted time of birth of infant 02-noted time of birth of placenta 03-noted Apgar scores or other methods of determining immediate transition of newborn to extrauterine life 04-noted any treatment given to mother, including medications 05-noted any treatment given to newborn, including medications 06-noted any complications and treatment given Open-ended Questions to Discuss with Health Care Worker As Time Allows: 256. [Observer note: write the questions you asked and the responses of the health worker. Use back of paper as needed.] 257. Sample questions: May wish to ask such questions as: Please tell us/me why you did X (fill in from observation)? Was there anything in the facility that kept you from doing Y (fill in from observation)? 258. Do you have any questions for us? (Observer note: Record questions asked) [Observer note: Return to demographic form to complete it with the health worker. End observation session by thanking the health worker for their time and cooperation. Remember to ask for the clinical record so that you can fill in question related to record review.] Time observation ended ______________________ Use 24-hour clock OBSERVER RATINGS (clinical judgment) Circle best response 259 Observed quality care Yes No 260 Level of health worker competency observed Competent Needs improvement 261. Observer: Please give a brief justification of your ratings on quality of care given and health worker competency. (Use back of paper if needed.) USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 129 262. Observer: Note any facility problem that may have interfered with the health worker’s ability to give care (e.g., power outage). OBSERVER NOTE: Please fill in a demographic form for each health worker observed. Assign consecutive participant ID numbers in each site visited (codes for each country and type of facility are on Observer Guidelines). Do not enter the name of the participant anywhere on this form. In order to match each observation to the participant ID number, make sure you record the participant ID number on each observation form. Date: Day Month Year Check (X) Facility Type: ____(1) Community facility without maternity beds ____(2) Community facility with maternity beds ____(3) Referral level facility : (list type) ____________________________________ ____(4) Urban ____(5) Rural ____(6) Private ____(7) Public Other (list) ______________________________________________________ Participant ID No. _ Country Facility Participant number Observer(s) Name(s): Instructions to the Observer/Evaluator Find a health worker involved in delivery of services in this setting who is considered a skilled attendant by the facility and proceed to obtain verbal consent for the observation. You should also obtain consent from the client when possible. Make sure the health worker understands that you are there to observe and not to serve as ‘experts’ to be consulted during the session. In addition, please tell the provider that you cannot talk with him/her during the observation period. Please read the following statements aloud following as close to this script as possible, without adding or leaving out anything. This will avoid bias and provide each participant with the same orientation. “Good morning/afternoon Dr, Mr., Ms. _____, 130 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 “Welcome and thank you for agreeing to meet with us today. My name is __________ and my colleague’s name is ___________. We are part of a project interested in observing maternal and newborn care in various countries in the LAC region and have received permission to do this from the health authorities in your country. We would like your permission to observe your consultation with/care of pregnant women for the next hour or so as you attend women or newborns during [antenatal, intrapartum, postpartum] care. Any information gathered today will be combined with information gathered from other similar settings and countries. Your name or the name of the pregnant women will not be recorded. All information gathered will maintain the confidentiality of this site and yourself. We will not interrupt your care, but will ask questions at the end of our time with you for clarification of anything we observed. You will also have time to ask any questions of us. Please note that we cannot talk with you or the pregnant woman while we are observing. Your participation is voluntary. Are you willing to give oral permission to talk with us and to allow us to observe your practice and talk with you at the end of the observation?” [If the respondent says ‘No’, thank him/her for their time and locate another health worker to observe in the same setting. If the respondent answers ‘Yes’ please proceed to request permission of the pregnant woman and then proceed with the observation.] Please proceed with the observation and then return to this form to complete the information. *********************************************************************************** Observer note: Please circle the indicated response or fill in blanks as requested. PROVIDER DETAILS (for use in all settings) No. Question/Observation Please Circle Response Number 001 What is your professional background? (Observer note: May need to read the categories and ask individual which ones correspond to them. Circle all that apply.) Midwife …………………………….….1 General Nurse ……….…………….…2 Maternal-Infant Nurse Specialist …..….3 Obstetrician ……………………….…..4 Pediatrician ……………………….…...5 Medical doctor (general practitioner).. 6 Obstetric resident physician…………...7 Auxiliary midwife ……………….….….8 Auxiliary nurse………………….….….9 Empirical/traditional midwife …….....10 Other (specify) ________________11 _________________________________ 002 How many years has it been since you received your first professional qualification? If you have a second professional qualification, how many years ago did that occur? Years:__________ Months:__________ Years:__________ Months:__________ 003 How many years of formal schooling have you completed, beginning grade one? Please fill in actual number below _____________ years 004 How many months/years of formal training did you have to provide prenatal care, attend births, Please circle response #1 or record actual USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 131 No. Question/Observation Please Circle Response Number and provide immediate newborn care? (Observer Note: may rephrase as “have in midwifery knowledge, skills, & professional behaviors?” The classification is in terms of ‘midwifery’ training even if the participant is not a midwife. You may find that an individual was prepared for providing prenatal or postpartum/newborn care but not to attend births. If so, please write in which area the preparation occurred.) number No midwifery training………………1 _________ years _________months 005 Do you attend births? Please tick actual response Yes _____ No _____ 006 If you attend births, how many births have you attended in the past 6 months? Record actual number ______________ 007 How many years have you been working as a health worker providing: (Observer note: circle all clinical areas that match the observation site.) Please record actual number of years Antenatal ___________ years Intrapartum ___________ years Postpartum ____________ years Newborn _____________ years 008 How long have you worked in this facility and setting? Record in months _______ and years ____________ Please continue on the next page. 009 Most likely specific emergency situations will not occur while I am observing your care of a pregnant woman. Please respond yes (Y) or no (N) to the following questions related to specific emergencies. For each one I will ask whether (1) you have learned this, (2) you have the skills to do this, (3) you are authorized to do this in your country, and (4) when was the last time you did this. 132 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Selected life-saving skills for emergency situations Learned this Have skills to do this Authorized to do this Last time I did this (date) 01—Start IV 02—Administer parenteral antibiotics for infections 03—Administer parenteral oxytoxics for vaginal bleeding, postpartum hemorrhage 04—Administer magnesium sulfate and calcium carbonate on indication for severe pre-eclampsia or eclampsia 05—Perform forceps- or vacuum-assisted delivery 06—Perform manual removal of placenta 07—Remove retained products of conception using manual vacuum extraction (MVA) 08—Perform bimanual uterine compression for postpartum hemorrhage 09—Perform newborn resuscitation for depressed baby Observers, please say: “Thank you for answering these questions.” USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 133 ANNEX 14. RESULTADOS DEL EQUIPO EVALUACIÓN EQUIPOS DE GESTIÓN EN HOSPITALES RESUMEN DE LAS OBSERVACIONES REALIZADAS Y LA APLICACIÓN DE LAS ENTREVISTAS EN 6 HOSPITALES DEL PROYECTO CENTROS DE EXCELENCIA MATERNO INFANTIL. PREPARADO POR EL EQUIPO No. 2 Dr. Rafael Montero De oleo Dr. Donatilo de los Santos Presentacion En el marco de la evaluación de medio término del proyecto centros de excelencia en materno infantil y la conformación de los equipos de trabajo, en donde se delegaron responsabilidades a los diferentes equipos conformados, los cuales tuvieron como base para la formación, las competencias y experiencias laborales. Se conformaron 4 equipos y cada uno de ellos se dirigió al campo en donde se ejecutan las acciones del proyecto, con el propósito de recabar las informaciones y datos pertinentes al proceso de ejecución. El informe o relatoría que a continuación se presenta es el resultado del trabajo realizado por el equipo dos, que durantes los días 5 al 9 de marzo, se desplazaron por los distintos establecimiento, previo contactos con los equipos de gestión y con ellos en el campo se realizaron una entrevistas, utilizando una guía dirigida, a la cual se le agregaban preguntas, según el interés y el nivel de respuesta de los entrevistados. El trabajo de levantamiento tuvo dos fase, una primera que fue la entrevista como proceso en si, para lo cual se entrevistaron a los informantes claves que eran los miembros del equipo de gestión del hospital y luego de concluida esa fase de pasaba a la verificación mediante observación de las distintas áreas intervenidas por el proyecto. Todos los equipos entrevistados dieron toda su colaboración y pusieron su empeño para que los evaluadores realizaran su trabajo, de tal suerte que los datos y opiniones vertidas en este informe son de la entera responsabilidad de los evaluadores y no necesiamente las opiniones descrita aquí, deberán coincidir con otras evaluaciones, aunque en sentido general si se espera que si. La sugerencias y comentarios hecha al proyecto, son apenas parte de las percepciones del equipo que esperamos estén contribuyendo al fortalecimiento del las acciones del sistema. Nuestro agradecimiento a cada uno de los participantes en las entrevistas, en este proceso. 1. Sistema de Gestión de Suministros En cuanto a las observaciones realizadas y las preguntas de la guía, en los 6 hospitales se observo que el mejoramiento del sistema de gestión de suministros ha sido implementado en el cien por ciento de los hospitales, en todos se verifico que existen las estanterías para la organización de los almacenes, señalizados, climatizados, materiales e insumos identificados, con relación a 134 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 medicamentos también se verifico que existe la distribución para hospitalización con el sistema de dosis única. Se verifico la instalación del software SIGHO el conocimiento y empoderamiento del uso del software, para la gestión de suministros En uno de los centros visitados, se identifico como un elemento de resistencia a los equipos de enfermería, que se resisten al sistema de dosis única, todavía prefieren despachos en volúmenes. En dos de los centros se verifico que no se han instalado los termómetros y se manifestó una contradicción en cuanto al manejo de la oxitoxina, refiriéndose que alguien había comentado que se podía mantener fuera de la nevera. 2. Gestión Clínica En cuanto a la medición de la gestión clínica, y las preguntas de la guía, se verifico que en todos los centros se han realizado capacitaciones en AMEU, COEM, EBR, BIOSEGURIDA, socialización de NORMAS Y GUIAS CLINICAS, sistemas de información entre otros que impactan positivamente la gestión clínica. Se verifico que existe una guía de Monitoreo y Evaluación en Emergencias Obstétricas y Neonatales, la cual ha sido aplicada solo por una de las tres DPS, que fueron visitadas, los informantes claves entrevistados hicieron referencia a que usan los indicadores de atención para la toma de decisión, en todos los centros visitados excepto uno, presentaron los indicadores señalados por el proyecto para el monitoreo y evaluación del desempeño de los centros de excelencia. Observamos que tienen los registros, en algunos centros más organizados que en otros, en el sentido de tenerlo identificados oportunamente. En todos los centros se verifico que se realizan las auditorias de las muertes maternas y tienen formado y funcionado el comité de morbilidad materna extrema y el uso del sistema de información perinatal está siendo implementado en todos los centros, con debilidades en cuanto al registro de las informaciones sobre el llenado correcto de la historia clínica perinatal, en el componente de hospitalización, en la mayoría de los centros el proceso de digitación se realiza en la oficina de Atencion al usuario y en uno de los centros se realiza en la misma área de hospitalización; Todos los centros tienen formado los comités de bioseguridad. Otras evidencias fueron observadas por el equipo de olga y pets. 3. Referencia Contrarreferencia y traslado El aspecto de referencia y contra referencia, medido por la evaluación, se indago sobre como hace el hospital la gestión del proceso de referencia y contra referencia y en todos los centros se observo que existe una gran debilidad, en cuanto al proceso, se han diseñada, funcionan precariamente para referencia, sin embargo la contra referencia ha funcionado muy poco, en una de las entrevistas refirieron, que la contraferencia solo ha funcionado en el 3% de los casos, en esta región se ha desarrollado, un sistema de digitalización de los instrumentos. En cuanto al proceso, los centros refirieron que cuando tienen necesidad de trasladar algún pacientes le hacen su historial para referirlos, hacen un contacto telefónico con el centro que será receptor del paciente, si tienen cama disponible se completa el proceso, que siempre se USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 135 envía usando regularmente una ambulancia propia o prestada y son enviados acompañados por un profesional (enfermera y/o medico) Los centros hicieron referencia que han recibido capacitación del proyecto en este aspecto, sin embargo todavía ser verifica una gran resistencia de parte del profesional médico, pues el instrumento requiere completarla con información médica, y que los profesionales se resisten a escribir en los expedientes. 4. Gestión por Procesos En cuanto a la gestión por procesos en todos los centros hicieron referencia a que están desarrollando esta iniciativa en estos momentos, en la capacitación y en la definición de los mismos, coincidimos los evaluadores en uno de los centros con la capitación de gestión por procesos. 5. Gestión de Riesgo y bioseguridad En cuanto a la gestión de Riesgos y Bioseguridad, la evaluación del proyecto consistió en la verificación de las iniciativas que se habían implementado en el nivel de los hospitales en ese sentido se verifico que todos los centros habían recibido capacitación para el tema de gestión de riesgos y bioseguridad. Las iniciativas estuvieron focalizada en mantener el control de todos los tipos de desechos que se producen en el centro, realizando una separación del material de riesgo del material biológico en ese sentido se verifico que en todos los centros, excepto uno se realizo un esfuerzo inicial de realizar una limpieza profunda en el centro en la cual participaron algunas instituciones locales como el cuerpo de bomberos y según refieren algunos centros se planificar su realización periódicamente. La implementación el plan de bioseguridad implico la adquisición de equipamientos, algunos de los cuales el proyecto, los suministro y en otros el hospital debía hacerlo, sobre todo lso zafacones, en tal sentido uno de los centro explico que no habían completado el proceso por falta de estos recursos. Todos los centros señalaron que fueron capacitados en bioseguridad pero presentan debilidades en la clasificación de los desechos hospitalario y en el destino final se ponen juntas todas da las fundas Se implementaron las rutas, se observaron los rótulos de áreas restringidas entre otros aspectos, sin embargo en cuanto a la disposición final de los desechos se observan todavía debilidades, pues probablemente el proyecto no contempla la rehabilitación de áreas para depósitos y los centros no han realizado estas inversiones, observamos que en uno de los centros visitados este proceso está en construcción y en otro ya realizaron las adecuaciones de los depósitos intermedios, sin embargo en todos centros todavía observamos la mezcla de fundas rojas y fundas negras. Lo que se evidencio es que en todos los centros excepto en uno, no estaban identificados los horarios de la ruta. Este proyecto implicaba una intervención de la gestión de estos desechos para lo cual era necesario hacer algún tipo de acuerdo con los gestores municipales, en este sentido se verifico que en los centros se habían realizado acuerdos y trabajado con el municipio, capacitación, 136 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 vacunación, entre otros. En este aspecto en los municipios menores, se verifico que estos hicieron las inversiones en la creación de la fosa especial para el depósito final de los desechos. Se evidencio que los hospitales se han reunidos y llegado a acuerdo con la municipalidad, pero falta dar seguimiento, comprobar que los acuerdos se están o tan ejecutados. Todos tienen comité de bioseguridad, pero por las debilidades encontradas, nos hacen sospechar, sin embargo que no están funcionando a plena capacidad. 6.Participación comunitaria. En cuanto a participación comunitaria, las actividades del proyecto evaluadas es como el hospital estaba realizando estas gestiones, en cuanto a este aspecto se verifico que se ha capacitado con énfasis en la constitución del consejo de administración hospitalaria, según referencia de los informantes claves, se han realizado las capacitaciones, se han constituido los consejos con la participación de instituciones de la sociedad civil, iglesias, etc. No verificamos ninguna acta del Consejo, lo que nos hace sospechar que se constituyeron, sin embargo no están funcionando a plena capacidad. No detectamos ninguna otra acción que haga referencia a participación comunitaria. 7. Sistema de Monitoreo de Emergencias y Desastres Una de las intervenciones del proyecto ha sido trabajar con planes de emergencia y desastres para las instituciones hospitalarias, asumiendo que estas deben ser las instituciones que mejor deben estar preparadas para hacer frente a cualquier contingencia derivada de una situación de emergencia o desastre. Como todas las intervenciones del proyecto, esta se ha realizado con la participación de la instancia del nivel central, regional y/o local, en caso de los planes de emergencia y desastres, se evidencio que ha existido, aparentemente dificultades de coordinación, celos institucionales, pues en los hospitales visitados, este componente se ha comportado de manera variada. En 4 de los 6 centros visitados existen los planes elaborados, en dos de ellos se han realizado, simulacros, en dos de ellos no se han realizados los planes de emergencia, los que tienen plan de emergencia y desastre tienen diseñada la ruta de evacuación y punto de encuentro( pero no señalizado ), algunos centros comentaron que se han presentado dificultades de coordinación, pues desde el área central del ministerios (atención a desastres y emergencias) han dicho que esa es su responsabilidad y que no podía hacer ninguna intervención sin la autorización de ellos. La iniciativa de emergencias y desastres, ha tenido dificultades de implementación, se ha quedado en la capacitación y elaboración del plan. 8. Sistema de Monitoreo & Evaluación Con relación al desarrollo del proceso de Monitoreo & evaluación, todos los centros hicieron referencia a la existencia de indicadores, sin embargo no lo han manejado como parte de la cultura institucional de análisis de los datos para la toma de decisiones, aunque de una u otra manera lo realizan. Están realizando de manera sistemática el levantamiento y registro de los indicadores de desempeño del proyecto sin embargo deberían tener establecido el sistema de monitoreo & evaluación, pero no lo tienen, dicen que requieren capacitación y el desarrollo de un sistema de USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 137 monitoreo & evaluación, como parte del proceso de medición permanente del sistema de garantía de calidad 9. Proyecto oficina atención al usuario. Uno de los puntos luminosos y de los cuales se sienten más orgullosos todos los centros es la llamada Oficina de Atencion al Usuario, en este sentido el proyecto apoyo en el desarrollo de toda la estrategia, aporto infraestructura, capacitación y equimiento para el funcionamiento de las oficina, el personal tiene alta motivación para el trabajo que están realizando. Se evidencio el nivel de organización de los expedientes clínicos, de los sistemas de registros, del software desarrollado (SIRS). Se observa el impacto del proyecto en cuanto a la imagen institucional, la eliminación de registros duplicadas, que en algunos casos se reportaron hasta 17, expedientes para una sola persona, todo esto ha sido superado, dicen los informantes claves da En dos hospitales se ha desarrollado la oficina de admisión, en uno de los hospitales están intentando implementar la oficina de admisión, pero en cuanto eso se produce, para mejorar el acceso de la información a los médicos, están dejando los records de las pacientes que están en consulta prenatal en el área de emergencia, al momento de cumplir las 36 semanas, como una forma de conseguir que cuando la paciente se ingrese, tenga su record. Con esto se evita el desconocimiento de las historias previas y la detección de un posible factor de riesgo. OTROS 11. Proceso De Selección, Induccion, Autoevaluacion Caf Y Preparacion El proyecto inicio con una nueva modalidad, en la cual se procedió a un proceso de selección interna, entre los centros del ministerio, para lo cual fueron convocados un numero de centros, este datos está disperso, cada informante clave da un dato de cuantos participaron, lo único que es sistemático, es que del primer proceso se preseleccionaron 27 y de estos 10, que son los que actualmente reciben los beneficios del proyecto Centros de excelencia materno Infantil. La información inicial que recibieron fue del la oficina de proyecto junto al ministerio, de que se iniciaría un proyecto que tendría una intervenciones, que no conocían, refirieron algunos informantes claves y que los que se interesaban, tenían que concursar, se les suministro una guía de preparación de la memoria para la presentación. Con esta memoria y el requisito mínimo de que tenían que tener área de maternidad, se seleccionaron los 10 hospitales. Una vez seleccionados, según la descripción de los informantes claves, los hospitales fueron capacitados para la conformación de los equipos de gestión del cambio y la aplicación de la herramienta CAF, que es el instrumento que utiliza el ministerio de la Administración Publica para otorgar el premio de buenas prácticas promisorias de calidad en la administración pública. Refieren algunos de los informantes claves que hubo mucha resistencia en los momentos iniciales, uno de ellos refiere que solo tres personas comenzaron y que luego que comenzaron a visualizar resultado se fueron incorporando. 138 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Cuando aplicaron el CAF, identificaron áreas de mejoras para la calidad, la mayoría de las cuales estaban referidas a debilidades gerenciales y de herramientas para gestión de información entre otras, la mayoría refieren que tuvieron puntuaciones muy bajas. Estas evaluaciones se realizaron en el 2009. En el 2011, todos los centros repitieron las evaluaciones usando el método CAF, alguno de ellos con vistas a la aplicación del premio a las prácticas promisorias de la calidad en la administración pública. Dos de ellos ganaron certificados de participación y uno gano medalla de plata. La mayoría de los centros tuvieron mejor desempeño en la segunda evaluación. Todos mostraron la documentación que registra la evidencia. Todos los informantes claves, estuvieron de acuerdo con el proceso de selección, dos centros hospitalarios señalaron que de las cosas que cambiarían en el proceso de concurso y selección, seria que se le debería dar una inducción por parte del ministerio para la preparación de la propuesta para el concurso y uno sugirió que para un próximo proceso y replica, se debería capacitar a los centros para que preparen sus memorias. 12. Proceso De Planificación Estratégica. El segundo proceso de la fase de preparación de los esfuerzos de implementación el proyecto de Centros de Excelencia, estuvo centrado el un esfuerzo de capacitación e instrumentación de los planes estratégicos para los 10 hospitales, fue un esfuerzo que duro unos dos meses, se realizaron varios talleres, acompañados por consultores internacionales y nacionales coordinados por la universidad CES de Colombia. De este proceso cada centro termino con su plan estratégico 2010/2015 el cual se elaboro en el 2010 con el concurso de todo el personal involucrado y con el plan operativo 2010 y luego todos los centros han trabajo el plan operativo 2011, ninguno a trabajado el plan operativo 2012 y no se evidencio el proceso de evaluación de los planes operativos. En todos los centros se verifico la evidencia y los documentos de planes estratégicos y operativos, sin embargo se observo todavía una cierta debilidad en el uso de la herramienta sobre todo en lo relativo al monitoreo y evaluación de las intervenciones del plan. Lo que coincidió con la evaluación de la parte de monitoreo & evaluación. En todos los centros se evidencio por parte de los entrevistados claves que no tienen sistematizado la evaluación de los planes estratégicos y operativos para mejorar las ejecuciones de las acciones propuestas. 13. Capacitación En lo relativo al proceso de capacitación, es evidente que el proyecto ha realizado muchas actividades de capacitación en herramientas gerenciales y del componente de gestión clínica, se han capacitado en el uso de la herramienta CAF, Planificación Estratégica, Bioseguridad, Manejo de Desechos Sólidos, definición de procesos, gestión del cambio, COEM, manejo de emergencias y desastres, entre otros. Un aspecto que contempla el proyecto es que los centros se conviertan en centro de capacitación como estrategia de sostenibilidad, y en ese sentido a facilitado y dotado de equipos algunos de los hospitales visitados, en cuanto a infraestructura y equipamiento. y formador de RRHH para la réplica. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 139 Sin embargo no tienen o muestran todavía los centros, fortalezas para el desarrollo de manera sistemática procesos de capacitación, metodologías educativas etc. Para poder decir que efectivamente se han convertido en centros de replica como centro de excelencia. Solo uno de los centros visitados no tiene espacio para el centro de capacitación. 14. Habilitacion La habilitación es uno de los ítems que el proyecto a de considerar para el proceso de certificación de los centros de excelencia, en tal sentido los hospitales han recibido capacitación para gestionar los procesos de habilitación, y han recibido algunos insumos y equipos para completar áreas que les permitan pasar las inspecciones que realiza el ministerio para el proceso de habilitación. De los centros visitados solo uno no ha sido habilitado, debido a que les falta un montacargas de pacientes o elevador y las filtraciones de los techos. Un centro ya estaba habilitado al momento de la llegado del proyecto. SUGERENCIAS DE LOS INFORMANTES CLAVES PARA LA SOSTENIBILIDAD DEL PROCESO AL FINALIZAR EL PROYECTO. Al finalizar las entrevistas a los informantes claves, se les cuestiono sobre su percepción de quien debería asumir las competencias y responsabilidades que hasta la fecha ha desarrollado el proyecto para la sostenibilidad y replica de este proceso en otros hospitales, Direcciones Regionales y Direcciones Provinciales. Las repuestas fueron en 5 de seis centros visualizan a la Dirección de Redes en el nivel central como la instancia natural para recibir la transferencia de competencias y las Direcciones Regionales para el proceso de gestión de las replicas en los hospitales, solo uno de los centros vinculo este proceso al vice ministerio de garantía de la Calidad, talvez porque este centro no se ve como parte de la red a la cual corresponde. LESIONES APRENDIDAS. En las palabras de los informantes claves, el proyecto ha aportado en: El mejor entendimiento del significado de trabajar por una cultura de la calidad. El mejor entendimiento del significado de trabajar en equipo. El valor del uso de la información y la generación de evidencias para la toma de decisión y para saber que hacemos, capacidad de análisis. El valor de la planificación estratégica, saber dónde estamos y para donde vamos o queremos ir. El valor de la supervisión, evaluación y acompañamiento de los procesos. El saber que podemos hacerlo, que somos capaces, que estamos empoderados. Reconocer que para impactar en el perfil epidemiológico, la focalización en las áreas que mayor aporte hacen a la frecuencia del problema es una buena alternativa. 140 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 SUGERENCIAS Y RECOMENDACIONES. 1. Completar el proceso del desarrollo del sistema de indicadores de evaluación & monitoreo o realizar un refrescamiento de este proceso para fortalecer los procesos de monitoreo y evaluación en los centros. 2. Diseñar capacitaciones de refrescamiento en algunas de las estrategias para la mejora de la gestión clínica, siempre capacitación en servicio, si posible en modalidad de pasantía, aprender haciendo. 3. Reforzamiento de las actividades de Bioseguridad y de manejo de desechos sólidos, con énfasis en la disposición intermedia y final. 4. Dar seguimientos al cumplimiento de los acuerdos con la municipalidad. 5. Diseñar un proceso de transferencia e instrumentación de los ejes de implementación de dentro de excelencia materno infantil de la Dirección de Gestión de Redes, hacia las Direcciones Regionales y de estas a los hospitales que serán certificados como centros de excelencia. El enfoque de procesos de esta transferencia, por ejemplo vamos a capacitar e instrumentalizar a la dirección de redes y la dirección regional para que evalué el proceso de bioseguridad y manejo de desechos de los hospitales, de manera que el proyecto, solo haga una supervisión del cumplimiento. Y no más responsabilidades. En este aspecto especifico, y así sucesivamente. Lo mismo para planificación estratégica, lo mismo para la herramienta CAF, igual con los ejes de implementación del proyecto etc. 6. Diseñar un proceso de transferencia de capacidades e instrumentación de monitoreo y supervisión a las direcciones provinciales de salud en los ejes de implementación de dentro de proyecto centros de excelencia materno infantil. 7. Diseñar un proceso de capacitación en metodologías y procesos de diseños de cursos y capacitaciones, para que los centros de capacitación adquieran esas competencias y habilidades, para ello se deberán seleccionar en los centros los RRHH con unos perfiles deseables para recibir estas capacitaciones y posterior desarrollo, esto se podría hacer con algún acuerdo con universidades nacionales o internacionales. 8. Que se valore la posibilidad de la creación de un joinventure para que en el pais se cree una agencia acreditara de centros de atención. Con participación publico/privada. Una especie de filial de alguna de las acreditadoras Norteamericanas, Europeas o latinoamericanas reconocidas. 9. Que el proyecto dedique parte de los esfuerzos del tiempo restante en realizar una determinación de brechas gerenciales en las Dirección de gestión de redes y sus diferentes instancias, en la subsecretaria de salud colectiva y las DPS, así como en el vice ministerio de garantía de la calidad, para que estas instancias puedan continuar con las iniciativas del proyecto en los centros de excelencia desarrollados y multiplicar la iniciativa con una visión más amplia y holistica 10. Que el proyecto dedique una parte del tiempo restante a instrumentalizar la Dirección de Redes, las Direcciones Regionales, Salud Colectiva y las DPS, así como el vice ministerio de garantía de la calidad, para la futura transferencia de las competencias; es decir elaborar una especie de manuales operativos para cada proceso de la A a la Z, definiendo responsabilidades, perfil necesario de los RRHH, tecnologías, presupuestos, etc, para las implementaciones de las diferentes acciones. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 141 11. Que el proyecto se dedique a escribir un manual de implementación de los procesos pasa a paso para ser centro de excelencia, de forma que se pueda transferir las competencias a las instancias que las autoridades decidan. 12. Hacer intervenciones para mejorar la participación comunitaria en los hospitales y hacer mas funcional el papel de los Consejos de Administración hospitalarios. 142 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 143 ANNEX 15: RESULTADOS DEL EQUIPO EVALUACIÓN CUALITATIVA CONTENIDOS 1. PRESENTACIÓN ...................................................................................................................... 145 2. OBJETIVOS DEL COMPONENTE CUALITATIVO ......................................................... 145 3. ASPECTOS METODOLÓGICOS ......................................................................................... 145 3.1. Diseño .................................................................................................................................. 145 3.2 Técnicas e instrumentos de recopilación de datos ..................................................... 146 3.3 Procedimientos .................................................................................................................... 147 3.4 Consideraciones éticas ...................................................................................................... 148 3.5 Limitaciones .......................................................................................................................... 148 4. PRESENTACIÓN Y DISCUSIÓN DE RESULTADOS ...................................................... 149 4.1. Gestión de cambio ............................................................................................................. 149 4.1.1. Oficinas de Atención al Usuario ............................................................................. 149 4.1.2. Reclamo de derechos del usuario ........................................................................... 149 4.1.3. Sistema de archivo y registro ................................................................................... 150 4.1.4. Recursos humanos ..................................................................................................... 151 4.2. Gestión Clínica ................................................................................................................... 151 4.2.1. Normas y protocolos ................................................................................................ 151 4.2.2. Integración entre diferentes programas y servicios ........................................... 152 4.2.3. Trato a la usuaria ........................................................................................................ 152 4.2.4. Parto humanizado, apego precoz y lactancia materna ....................................... 153 4.2.5. Servicios de emergencia y laboratorio .................................................................. 155 4.2.6. Intervenciones de información, educación y comunicación (IEC) ................... 155 4.3. Referencia y contra-referencia ........................................................................................ 156 4.4. Centro de capacitaciones ................................................................................................. 156 144 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 4.5. Infraestructura y suministros........................................................................................... 156 4.6. Gestión de riesgos y bioseguridad ................................................................................. 157 4.7. Participación comunitaria ................................................................................................. 157 4.8. Emergencias y desastres ................................................................................................... 158 4.9. Oportunidades.................................................................................................................... 158 5. CONCLUSIONES ..................................................................................................................... 159 6. RECOMENDACIONES ........................................................................................................... 161 ANEXO. Cuadro 1. Resumen de datos recopilados por Centro de Salud ...................... 163 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 145 1. PRESENTACIÓN El presente documento se concibe como un addendum al informe general de la Evaluación de medio término del acceso a servicios materno-infantiles de calidad en áreas seleccionadas de la República Dominicana (USAID). En este sentido, expone de manera detallada los objetivos, los aspectos metodológicos y los principales hallazgos del Componente Cualitativo de la evaluación, así como las conclusiones y las recomendaciones derivadas de dichos hallazgos, a la vez que comparte lo antecedentes expuestos en el informe general de esta iniciativa. 2. OBJETIVOS DEL COMPONENTE CUALITATIVO Los objetivos y las preguntas de investigación del Componente Cualitativo fueron enmarcados en los objetivos generales de la Evaluación, tal y como se presentan en el informe final de la misma. En este sentido, tratándose de una evaluación de término medio, este Componente se propuso identificar en el contexto de la entrega de los diferentes servicios de salud materno￾infantil, las dimensiones del proyecto que están funcionando bien o no, los factores subyacentes al nivel de funcionamiento descrito y, de manera prioritaria, las medidas y modificaciones que puedan incidir de manera positiva en el desempeño del proyecto en sus próximas fases. De manera específica, el Componente Cualitativo se propuso: Describir y analizar la percepción que tienen las usuarias y los/as proveedores de servicios con respecto a la calidad de atención prenatal, atención al parto y seguimiento postnatal en los Centros de Excelencia seleccionados de la República Dominicana. Contrastar la percepción que tienen las usuarias y los/as proveedores de servicios de salud materno-infantil con respecto a la calidad de los mismos, con las observaciones directas realizadas en los Centros de Excelencia seleccionados, tomando en cuenta el enfoque de derechos, el enfoque de género y la perspectiva de parto humanizado. Indagar la percepción de las usuarias y los/as proveedores de servicios de salud materno￾infantil sobre los cambios en la calidad de dichos servicios a lo largo de los últimos años, y su potencial relación con las intervenciones auspiciadas por el USAID en los Centros de Excelencia. Contrastar y complementar la información obtenida por otros Componentes de la evaluación con respecto a los aspectos programáticos del proyecto, desde el punto de vista de las usuarias y los/as proveedores directamente involucrados/as en la entrega de servicios de salud materno-infantil. Ofrecer recomendaciones puntuales para el diseño y la implementación de intervenciones futuras del Proyecto, ajustadas a las necesidades percibidas de la población que acude a servicios públicos de salud materno-infantil en el país. 3. ASPECTOS METODOLÓGICOS 3.1. Diseño El Componente Cualitativo de la Evaluación ha sido diseñado como parte integral del Proceso de Abordaje Rápido (RAP, por sus siglas en Ingles: Rapid Assessment Process), definido como la búsqueda intensiva de datos utilizando la triangulación de fuentes de información, el análisis iterativo, y la recolección adicional de datos para desarrollar rápidamente un entendimiento 146 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 preliminar de una situación en particular1. En este sentido, se tomaron en cuenta las percepciones de los/as proveedores de los diferentes servicios de salud materno-infantil, y de las usuarias de los mismos en los centros hospitalarios visitados, para caracterizar el contexto de la entrega de los servicios relevantes para el Resultado 4 de USAID, en torno a las preguntas de investigación planteadas para la presente Evaluación: 1. ¿Cuáles aspectos programáticos han sido los más / menos exitosos para asegurar los resultados del proyecto y por qué? 2. ¿Cuáles intervenciones técnicas han sido las más / menos exitosas para asegurar los resultados del proyecto y por qué? 3. ¿Cuáles aspectos programáticos aparentan ser los más / menos sostenibles y por qué? 4. ¿Cuáles intervenciones técnicas aparentan ser las más / menos sostenibles y por qué? 5. ¿Se ha enfocado el proyecto en las actividades más críticas y apropiadas? De no ser así, cuales fueron las oportunidades perdidas? Con el fin de responder a estas preguntas desde el contexto de la entrega de servicios, se visitaron cinco de los 10 centros de excelencia funcionando a nivel nacional, los cuales han sido seleccionados a partir de los insumos y las prioridades identificadas por el equipo de evaluación (ver Cuadro 1, ANEXO). Si bien, siguiendo la metodología cualitativa de este Componente, se procuró la mayor heterogeneidad viable de informantes claves, en términos de sexo, edad, perfil educativo y profesional de los/as participantes, los datos reunidos fueron sintetizados e integrados en torno a los bloques temáticos previstos para ser evaluados en cada centro de salud visitado. En este sentido, el informe de este Componente incluye una perspectiva integral de la información relevante para los ejes centrales de la evaluación en cada centro de salud, con el fin de facilitar la comparación y la síntesis de las observaciones conducidas en los diferentes contextos. 3.2 Técnicas e instrumentos de recopilación de datos Los datos primarios fueron reunidos a través de técnicas cualitativas, incluyendo los grupos de discusión, entrevistas cualitativas y observación directa conducida en los diferentes espacios de los centros de salud visitados. Asimismo, en el contexto de las observaciones directas, fueron realizadas entrevistas puntuales con informantes claves identificados en el proceso, utilizándose también la modalidad de entrevista grupal en función de lo requerido por las circunstancias específicas de cada Hospital. Se condujeron ocho (8) grupos de discusión en total, siendo cuatro de ellos integrados por las usuarias de servicios de salud materno-infantil y otros cuatro por los/as proveedores de estos servicios de diferentes perfiles profesionales (médicos residentes, médicos especialistas, enfermeras y personal de Emergencia). El Cuadro 1 (ver anexo) presenta de manera resumida las fuentes de datos primarios recopilados en cada uno de los Hospitales visitados. Con el fin de complementar la información reunida a través de los grupos de discusión, incluyendo las perspectivas de los diferentes actores claves en cada centro visitado, se condujeron entrevistas cualitativas complementarias, en torno a los contenidos equivalentes a 1 Beebe J (2001). Rapid Assessment Process: An Introduction. Walnut Creek, CA: Altamira Press. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 147 los abordados en los grupos de discusión. El Cuadro 1 (ver anexo) resume los perfiles de los informantes claves entrevistados/as en cada centro de salud. Las observaciones cualitativas directas (no estructuradas) fueron conducidas de manera paralela a la recopilación de datos primarios de otras fuentes. Estas observaciones fueron orientadas por una lista de aspectos a observar, y reportadas en el formato de diario de campo correspondiente a cada espacio observado. La realización de los grupos de discusión y las entrevistas cualitativas fue orientada por guías ajustadas a cada uno de los actores entrevistados. Estas guías fueron diseñadas en torno a 2 temarios macro, dirigidos, respectivamente a evaluar la percepción de: Usuarias de servicios de salud materno-infantil. Proveedores de servicios de salud materno-infantil. Las guías construidas fueron empleadas como formatos flexibles para la recopilación de datos vinculados a los objetivos establecidos, sin obstaculizar de manera alguna el libre desarrollo de las entrevistas y las discusiones en los grupos convocados. A la vez, estas guías partieron de propuestas temáticas abiertas, permitiendo que los participantes saquen a relucir espontáneamente las vivencias más relevantes en su contexto particular, evolucionando gradualmente hacia preguntas más dirigidas, para enfocar detenidamente aspectos juzgados como especialmente pertinentes en cada caso evaluado y aclarar cualquier aspecto que quedó insuficientemente abordado en las fases previas de la discusión. Todas las entrevistas y grupos de discusión fueron grabados en audio, como apoyo para el procesamiento y el análisis de datos, en adición a las notas y otros registros producidos en el contexto del trabajo de campo. Los datos fueron procesados y analizados utilizando el enfoque de análisis de contenido, construyendo las categorías de análisis a partir de los objetivos de la evaluación y el cuadro de marco de referencia elaborado para estos fines. El equipo de investigación se reunió al final de cada día de trabajo de campo para compartir las experiencias acumuladas en el centro de salud visitado, comparar las observaciones realizadas y extraer primeras inferencias para fines de análisis de datos. Se acudió a la triangulación de las diferentes técnicas con el fin de profundizar y consolidar la información obtenida a través de cada una de ellas. 3.3 Procedimientos En cada centro de salud a ser visitado fue designada, con el apoyo de las autoridades hospitalarias, una persona involucrada en la entrega de servicios de salud materno-infantil, quien sirvió de apoyo a la logística y otros aspectos operativos del proceso de recopilación de datos en el mismo. Esta persona acompañó al equipo en el proceso de recopilación de datos, facilitando el acceso a las áreas del hospital previstas para las observaciones cualitativas. Las actividades de recopilación de datos en los centros de salud visitados incluyeron, sin limitarse a éstas: Un recorrido inicial por la planta física del hospital, con conversaciones informales con los/as proveedores y los/as usuarios de servicios de salud presentes en las diferentes áreas de la misma. 148 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Realización de un grupo de discusión con las usuarias del perfil previamente definido (ver Cuadro 1, Anexo), previa obtención del consentimiento informado. Realización de entrevistas cualitativas con los actores claves del perfil pre-definido, procurando en cada caso complementar los puntos de vista y la perspectiva de los actores involucrados en los grupos de discusión. Realización de un grupo de discusión con los/as proveedores de servicios de salud del perfil pre-definido (ver Cuadro 1, Anexo). Un recorrido final de la planta física del hospital, observaciones y entrevistas complementarias estimadas como necesarias para obtener los datos requeridos para los objetivos de la evaluación. Reunión del equipo de recopilación de datos con el fin de compartir y contrastar las observaciones y las experiencias acumuladas en el transcurso del día. 3.4 Consideraciones éticas El Componente Cualitativo de la presente Evaluación no expuso a sus participantes a riesgos significativos de carácter psicológico, físico o social. Con el fin de prevenir y/o aliviar las posibles reacciones de incomodidad frente a tópicos de carácter personal, todo/a participante fue previamente informado/a de su derecho a rehusarse a responder a cualquier pregunta formulada durante la entrevista / grupo de discusión, o a retirarse de la misma cuando así lo desee, sin que le sea solicitado explicar su decisión al respecto, garantizándole a la vez la confidencialidad de la información compartida. A las personas que necesitaron trasladarse al centro de salud para poder participar en la Evaluación les fueron compensados los gastos en los que incurrieron por concepto de transporte. Se solicitó de manera escrita, el consentimiento informado de las usuarias de servicios que aceptaron participar en la evaluación. El formulario de consentimiento informado, elaborado para estos fines, fue presentado y explicado a cada participante en el momento de su reclutamiento, asegurando su plena comprensión de sus derechos y los compromisos asumidos con la Evaluación conducida. Antes de otorgar su consentimiento, los/as participantes dispusieron del tiempo suficiente para leer el formulario de manera detenida, o, de no poder hacerlo, éste les fue leído por el/la entrevistador/a y firmado en la presencia de un/a testigo. 3.5 Limitaciones Si bien el proceso de recopilación de datos para el Componente Cualitativo fue llevado a cabo procurando el mayor rigor posible, es importante tomar en cuenta algunas limitaciones metodológicas de este proceso en el momento de interpretar la información reunida. Se destacan, en este sentido, los siguientes aspectos: Las visitas de recopilación de datos fueron llevadas a cabo de manera conjunta con los otros equipos de evaluación, lo cual implicó una amplia expectativa y preparación previa de los Hospitales para la visita de evaluación por parte e USAID. Así, el esfuerzo del personal de salud por dejar una buena impresión en el equipo de evaluación, pudo haber interferido con la autenticidad de los datos reunidos en el componente cualitativo. Para reducir el impacto de esta limitación, se procuró contrastar las informaciones provistas por los/as proveedores de servicios de salud y las observaciones directas, con los reportes de las usuarias de estos servicios en los grupos de discusión. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 149 Algunos de los grupos de discusión de usuarias fueron convocados de manera anticipada por la Dirección de los Hospitales visitados, implicando este procedimiento una pre-selección de las participantes e integración de personas allegadas y/o familiares del personal de salud en estos grupos. Una vez fue identificado el sesgo que este procedimiento pudiera implicar para el proceso de recopilación de datos, se optó por reunir las usuarias presentes en los Centros de Salud en los días de visita, sin aviso previo a las autoridades hospitalarias. La metodología de Abordaje Rápido (RAP, por sus siglas en inglés) asumida en la presente Evaluación, partiendo del tiempo limitado disponible para el levantamiento de datos en el terreno, ha provisto una imagen general de la situación de servicios de salud materno-infantil en los Centros de salud visitados, así como de las fortalezas y los retos principales enfrentados por el Proyecto de Centros de Excelencia en cada uno de ellos. Cabe señalar, sin embargo, que esta metodología no permite un análisis detallado de los diferentes aspectos identificados en este proceso ni la inferencia de relaciones de casualidad entre las intervenciones del Proyecto y la situación de los servicios de salud materno-infantil en estos establecimientos, lo cual ameritaría de la aplicación de técnicas de investigación adicionales y, consecuentemente, de la dedicación de un tiempo más extenso al proceso de la Evaluación. 4. PRESENTACIÓN Y DISCUSIÓN DE RESULTADOS 4.1. Gestión de cambio 4.1.1. Oficinas de Atención al Usuario El impacto de las intervenciones del Proyecto de Centros de Excelencia en la organización y, particularmente, en la reducción de los tiempos de espera para acceder a los diferentes servicios, ha sido reconocido tanto por las autoridades hospitalarias y los proveedores de servicios de salud, como por las usuarias de los mismos. Las Oficinas de Atención al Usuario (OAU) están operando en todos los Hospitales visitados, siendo valoradas por las usuarias como uno de los cambios más positivos introducidos en el centro de salud en los últimos años: “Por ejemplo allá abajo cuando uno entra, eso cambio muchísimo, la sala de espera, está todo mas ordenado… Allá tu llegas, coges tu ticket, ves en la pantalla el turno que te toca… mucho mejor a como era antes” [GD1, Usuarias] En algunos Centros, las Oficinas de Atención al Usuario hasta la fecha no han logrado desempeñar todas las funciones previstas, incluyendo la asignación y el control de citas, si bien en todos los casos están en proceso para alcanzar esta meta. Por otro lado, las usuarias entrevistadas todavía no reconocen a estas Oficinas como el lugar indicado para obtener informaciones sobre los servicios provistos en el centro de salud, aclarar dudas y presentar las posibles quejas frente al funcionamiento de estos servicios, implicando esta falta de reconocimiento la necesidad de un proceso más amplio y prolongado de información a la usuaria. 4.1.2. Reclamo de derechos del usuario Si bien en todos los Centros de salud visitados fueron observados los buzones de sugerencias a través de los cuales los usuarios pueden expresar por escrito su percepción de la calidad de los servicios recibidos, en varios casos estos buzones estaban muy deteriorados, permitiendo al personal no autorizado extraer y leer sus contenidos. Asimismo, en la mayoría de los casos los proveedores de servicios desconocían los mecanismos establecidos para procesar las 150 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 sugerencias de los usuarios y las personas responsables de esta función, reportando no haber recibido ningún tipo de retroalimentación con respecto al desempeño de sus tareas, a raíz de las sugerencias depositadas en los buzones. Es importante observar, asimismo, que dado el bajo nivel educativo de la población que acude a los centros públicos de salud en la República Dominicana, los buzones de sugerencias no representan un mecanismo viable para la expresión de quejas y opiniones de las personas que no saben leer y escribir. De manera acorde, las usuarias de servicios de salud materno-infantil en los Centros visitados no tienen claros los mecanismos a través de los cuales pueden expresar y canalizar sus reclamos, optando, en la mayoría de los casos, por acudir a la Dirección del Hospital para estos fines. En algunos casos los procedimientos para el manejo de quejas de las usuarias no están claramente establecidos, dependiendo la respuesta a las mismas de la buena voluntad de la Dirección y el personal administrativo del Hospital. Esta falta de institucionalidad en el manejo de los reclamos interfiere con la respuesta oportuna a los mismos, contribuyendo a la percepción ampliamente observada en el discurso de las usuarias, de los servicios públicos de salud como un “favor” más bien que como un derecho constitucional, y a su preocupación frente a las potenciales represalias del personal de salud frente a sus quejas: “Si tu le dices algo te dejan para último, no te quieren atender… …Tu te quejas con alguien, por lo menos que tu sepas y ese te escucha y después sale por ahí y entonces lo sabe todo el mundo, y te tratan ya tu sabes…” [GD3, Usuarias] 4.1.3. Sistema de archivo y registro El personal administrativo y proveedores de servicios de salud de los hospitales visitados reportan como uno de los mayores impactos del Proyecto la organización del sistema de registro y archivo, así como la actualización del sistema informático para el acceso oportuno a los expedientes clínicos en sus centros de salud. A la vez, el personal del archivo muestra un empoderamiento notable frente al desempeño eficiente de sus funciones, siendo el mismo posiblemente atribuible al proceso de capacitación y entrenamiento recibido como parte del Proyecto. Los informantes claves de varios de los establecimientos visitados reportaron haber replicado el entrenamiento recibido en los centros de salud de menor nivel dentro de sus regiones. Uno de los beneficios más frecuentemente reportados de la actualización del sistema de archivos para la calidad de la atención en salud materno-infantil en estos establecimientos, ha sido la posibilidad de acceso permanente a la historia clínica prenatal de la usuaria desde los servicios de emergencia, a través de los cuales usualmente se realiza el ingreso de la mujer para la atención al parto. En varios centros este acceso se realiza da manera digital, mientras que en otros, en los cuales la digitalización de expedientes todavía está en proceso, están establecidos los procedimientos específicos para mantener los expedientes clínicos de las usuarias en los últimos meses del embarazo en las zonas de fácil acceso para el personal de emergencia. A pesar de los avances en el sistema de archivo y registro de los hospitales visitados, los reportes del personal de salud entrevistado sugieren una utilización muy limitada de los datos clínicos para la orientación y la retroalimentación de la planificación administrativa del hospital. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 151 4.1.4. Recursos humanos De manera general, las observaciones realizadas en los establecimientos visitados y las entrevistas conducidas, señalan el factor de recursos humanos y su motivación para lograr cambios positivos en el desempeño de sus funciones y, por ende, en la calidad de atención provista a las usuarias, como un componente crucial del impacto logrado por el Proyecto en estos Centros de salud. En este sentido, partiendo de las afirmaciones de las personas entrevistadas, las capacitaciones y, sobre todo, el acompañamiento provisto al personal de salud en el contexto del Proyecto, lograron impactar no solamente los conocimientos de este personal, sino también sus competencias y la motivación para el cambio. Asimismo, el estilo de gerencia de recursos humanos a lo interno de los hospitales y, particularmente, la clara asignación de funciones y responsabilidades entre los diferentes actores en el contexto hospitalario, parecerían influir en la magnitud de este impacto y el funcionamiento de las intervenciones introducidas por el Proyecto en cada establecimiento de salud. En este sentido, las experiencias más exitosas lucen ser aquellas que han estado a cargo del personal capacitado, competente y, sobre todo, altamente empoderado y motivado para el desempeño de sus funciones, con una clara delimitación de las jerarquías institucionales, responsabilidades individuales y canales de comunicación inter-institucional. Por el contrario, la gerencia menos eficiente de recursos humanos observada en algunos establecimientos, frecuentemente caracterizada por una delimitación de funciones más imprecisa y menos eficiencia en el acompañamiento y la supervisión del desempeño de los proveedores de servicios de salud, parecieron vincularse a un impacto más limitado y el menor desarrollo de las intervenciones introducidas por el Proyecto. 4.2. Gestión Clínica 4.2.1. Normas y protocolos En los centros de salud visitados se obtuvo muy poca evidencia del seguimiento de las normas nacionales establecidas para la atención en salud materno-infantil. El personal de salud entrevistado reporta, en su mayoría, conocer de la existencia de estas normas, si bien no siempre tiene la capacidad de referir sus tópicos ni contenidos específicos. De manera acorde, el personal de salud, incluyendo los médicos residentes de diferentes especialidades, no refieren haber recibido entrenamientos específicos en el manejo de las normas, si bien afirman haber oído hablar de las mismas en el contexto de las capacitaciones recibidas en torno a otros temas más generales (manejo de emergencias obstétricas, etc.). En ningún caso pudieron identificarse procedimientos claramente establecidos para la supervisión del cumplimiento de normas específicas, ni las sanciones previstas para los casos de no-cumplimiento de las mismas. Estos hallazgos coinciden con los estudios previos conducidos en el país sobre el cumplimiento de las normas nacionales en la atención a la salud materno￾infantil en el país2. 2 Pérez-Then E (Ed.) (2008). Calidad de la Atención de la Embarazada y del niño menor de un año en Centros de Primer Nivel de Atención de la Regiones III, IV, V VI de la República Dominicana. CENISMI. Serie de Reuniones Técnicas IV; Pérez-Then et al. (2010). Evaluación de la calidad de atención a la embarazada y al niño sano en centros de primer nivel de atención de las Regiones de Salud III, IV, V y VI de la República Dominicana: Resultados de la segunda fase. Boletín CENISMI, Centro Nacional de Investigaciones en Salud Materno Infantil, vol. 20, no 2. 152 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Por otro lado, el personal de varios hospitales visitados refirió las entregas de guardia de los diferentes servicios como un importante mecanismo para asegurar el cumplimiento de normas y protocolos, analizar el manejo de los casos y responsabilizar a proveedores específicos de darles seguimiento a las usuarias que lo ameritan. 4.2.2. Integración entre diferentes programas y servicios De manera general, en los centros de salud visitados fue observada poca integración y comunicación inter-departamental, incluyendo el manejo de los registros clínicos. Algunos programas, como el Programa Nacional de Prevención de la Transmisión Vertical del VIH (PNRTV), el Programa Nacional de Control de Tuberculosis (PNCT), y el Programa de Adolescentes, operan con más autonomía que otros y son vistos como “mundos aparte” por los proveedores y, en ocasiones, las autoridades hospitalarias. En este sentido, los proveedores de uno de los centros visitados hicieron referencia a la necesidad de una mayor integración e interacción inter-departamental del Hospital, con el fin de proveer atención más integral a la usuaria. Las observaciones cualitativas llevadas a cabo en los establecimientos visitados sugieren que los servicios de atención prenatal no se conciben de manera integral, desaprovechándose oportunidades para ofrecer servicios de salud complementarios que podrían requerir las embarazadas. En un caso específico, una usuaria en claro proceso de duelo por la muerte repentina de su pareja, en ningún momento fue abordada por el personal involucrado en la atención prenatal para ofrecerle un referimiento a servicios de salud mental, y tampoco sabía a dónde dirigirse para procurarlos ella misma. 4.2.3. Trato a la usuaria En varios Centros, en parte a raíz de las capacitaciones recibidas, tanto las usuarias como los proveedores de servicios de salud materno-infantil reportan una mejoría notable en el trato a la usuaria, enfocada de manera particular en menos “boches” recibidos en el contexto de estos servicios. No obstante, las observaciones cualitativas conducidas y los reportes de las usuarias en algunos de los establecimientos visitados, sugieren la persistencia de algunas formas más sutiles de malos tratos, incluyendo: El trato de “no persona”, con comentarios emitidos sobre la usuaria en su presencia como si no estuviera ahí. Manejo de informaciones potencialmente confidenciales de las usuarias, tales como los antecedentes de uso de drogas ilegales, en presencia de otras usuarias y proveedores en las salas de internamiento. Cabe señalar que las limitaciones de espacio físico, y la necesidad de realizar las consultas prenatales de manera grupal, también dificultan la confidencialidad de la información intercambiada en este contexto. Poca sensibilidad frente a las necesidades emocionales de las usuarias, incluyendo las recriminaciones de las expresiones de dolor durante el proceso de parto: “Yo le dije a la enfermera [durante la labor de parto] que no podía respirar del dolor, y me dijo ‘Aguanta ahora que tu aguantaste cuando lo estabas haciendo’.” [GD2, Usuarias] “Yo no me quejaba [del dolor] porque mayormente a ellas no les gusta… pero cuando uno se queja, ellas lo que le dicen a uno que aguante.” [E1, Usuaria] USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 153 Las usuarias que participaron en los grupos de discusión reportaron una mayor frecuencia de malos tratos en la atención al parto: “Con el mas chiquito que yo di a luz, ahí no había médicos, lo que había eran dos enfermeras, y eran dos perras… porque hay que decirlo asi… …por lo menos, está bien el dolor, uno sabe lo que es dolor, pero si usted se siente mal, verdad, usted tiene que decir ‘yo me siento esto’… y cuando yo les decía que sentía que no podía estar despierta, y cuando se lo decía, ellas lo que estaban era durmiendo.” [GD3, Usuarias] “Allá arriba en el área de parto tratan a uno como a un perro… los doctores le hablan malísimo a uno … si uno se levanta a buscar un poco de agua, a decir algo, te gritan ‘acuéstate, que yo no te mande a ti a parir’”. [GD4, Usuarias] No obstante, la mayoría de las usuarias entrevistadas perciben el mal trato recibido en los servicios de salud materno-infantil en estos Centros como propio de los servicios de salud que son gratuitos, asumiendo que no les corresponde exigir un mejor trato como parte de sus derechos ciudadanos: “En la clínica [privada] tratan a uno diferente… aquí atento a que es del gobierno, te tratan así” [GD4, Usuarias] Los proveedores de servicios de salud en salas de parto y post-parto frecuentemente necesitan manejar las situaciones de crisis emocional entre las usuarias, incluyendo los casos de abortos espontáneos y nacimientos de niños de sexo no esperado, u ofrecer apoyo psicológico a las mujeres en el proceso de parto. Esta necesidad es particularmente notable en el caso del personal de enfermería, quien pasa más tiempo con las usuarias en estos espacios. Al no contar con ningún tipo de entrenamiento formal para estos fines, y tampoco tener claros los mecanismos de referencia intra e interinstitucional para el manejo de este tipo de situaciones, el personal de enfermería tiende a proceder con la mejor intención a partir de su sentido común, el cual no necesariamente responde de manera adecuada a las necesidades de las usuarias: “[A las mujeres cuyo bebé se muere] uno les dice “Tu tienes más hijos? Mira, no te pongas triste, que Dios no quiso que este viviera, pero tú tienes otros hijos, y hay que darle gracias a Dios... … Papa Dios no quería que tu tuvieras ese bebe, hay que resignarse…” [GD2, Enfermeras]. “Yo le dije a una ‘Dale un beso a tu bebé, mira, tu bebé es varón’, y ella me salió con que ‘Ay no, yo estuve embarazada de hembra, la sonografía decía que era hembra’, y yo le dije ‘Mira, mi niña, tu hijo es varón y el Señor permitió de que fuera varón. Si el Señor hubiese querido que sea hembra, nace hembra. Y tengo para decirte que también tienes que darle el amor de varón, porque cuando vienes a ver tu tienes un homosexual en tu casa, si le das amor de hembra.” [GD2, Enfermeras]. 4.2.4. Parto humanizado, apego precoz y lactancia materna El discurso de los proveedores de servicios de salud y las autoridades hospitalarias en los establecimientos visitados no siempre refleja una plena comprensión de la idea de parto humanizado, limitándose este concepto a “no hablarle mal” a la usuaria, según lo descrito en el acápite anterior. Ninguno de los Centros utiliza la analgesia durante la labor de parto. Solamente en uno de ellos está disponible la analgesia para los casos que los proveedores consideran lo ameritan 154 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 (adolescentes que manejan muy mal el dolor, etc.), asumiendo la disponibilidad de anestesiólogos para administrarla. Las usuarias, por lo general, desconocen la existencia de este tipo de analgesia, resaltando el dolor del parto como algo positivo (mientras más dolor, más rápido avanza el parto). Por otro lado, el acompañamiento de la usuaria en el momento del parto es visto como deseable tanto por ellas como por los proveedores, aun cuando coinciden en que el acompañamiento por la pareja no es viable en el sector público, debido a las limitaciones de la infraestructura. Uno de los Hospitales visitados reporta haber tenido una excelente experiencia con líderes comunitarias entrenadas para realizar el papel de dulas. Las dulas acompañan a las mujeres a lo largo de su embarazo, las preparan para el momento del parto y, en el momento de parto les apoyan en el manejo del dolor, lo cual, según las autoridades hospitalarias y los informantes claves, ha contribuido de manera importante a la humanización del proceso de parto. Si bien, según las normas nacionales, el personal de enfermería no está autorizado para realizar partos, esta es una práctica común en los establecimientos visitados, siendo una de las mayores desventajas observadas en la misma su “clandestinidad”, que no permite registrar adecuadamente los partos realizados ni proveer el entrenamiento requerido al personal de enfermería para su realización. No obstante, en uno de los establecimientos visitados, que ha contado con el apoyo de otros proyectos para estos fines, el personal de enfermería ha recibido una amplia capacitación para la realización de partos, siendo su involucramiento en este proceso reconocido y respaldado por las autoridades hospitalarias, con muy buenos resultados para el funcionamiento de los servicios de atención al parto. Se reporta la práctica del manejo activo del tercer periodo del parto (MATPP) en todos los centros visitados, si bien no de manera consistente. Llama la atención el reconocimiento de esta práctica como una de las fortalezas de los servicios recibidos por parte de las usuarias de servicios en uno de los establecimientos, afirmando que las “limpian bien”, por lo cual sangran muy poco después del parto: “Me limpiaron tan bien que yo nada más dure con la menstruación dos días… Ni siquiera tuve que usar las toallas de maternidad” [GD1, Usuarias]. Por otro lado, en varios Centros visitados los proveedores reportan la práctica de colocar el recién nacido en el vientre de la madre hasta que deje de palpitar el cordón, según establecen las normas nacionales. No obstante, estos reportes no siempre se ven corroborados por los reportes de las usuarias en los mismos Centros. Aun así, las madres reportan que sus bebés les son entregados en un tiempo breve post-parto, normalmente en menos de dos horas. La lactancia materna es priorizada como parte de las actividades educativas en los Centros de salud visitados, tanto en las salas de espera como en las salas post-parto. Asimismo, es reforzada de manera práctica por el personal de enfermería en estas salas. Sin embargo, con excepción de un establecimiento con amplia experiencia de trabajo comunitario, los hospitales visitados no cuentan con la capacidad de ofrecer seguimiento comunitario a la lactancia materna una vez la usuaria abandona el hospital, lo cual interfiere de manera importante con el impacto de las intervenciones llevadas a cabo en el contexto intra-hospitalario. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 155 4.2.5. Servicios de emergencia y laboratorio Los reportes de las usuarias en los grupos de discusión conducidos en varios de los centros de salud visitados, señalan una percepción marcadamente negativa de los servicios de Emergencia, incluyendo la poca confianza depositada en su capacidad resolutiva. Entre otros aspectos las usuarias expresan su inconformidad con los largos tiempos de espera, pocos proveedores de servicios asignados a esta área, y el mal trato recibido por parte de ellos: “En Emergencia cuando tu llegas te atienden cuando a ellos les da la gana… Cuando yo vine el otro día, nunca aparecía el médico, y cuando fui a ver estaba el medico ahí charlando con dos mujeres, y yo dije en voz alta que a él no le pagan pa estar enamorándose…. Y el dijo ‘Cuál fue la fresca que dijo eso?’” [GF3, Usuarias]. En algunos Centros de salud las autoridades hospitalarias refirieron un trato privilegiado a las mujeres embarazadas en el acceso a los servicios de Emergencia con el fin de intervenir oportunamente en el caso de presentarse una emergencia obstétrica en este contexto, si bien este trato no fue observado en la práctica, ni corroborado por las usuarias de estos servicios. Cabe señalar que Por otro lado, en varios centros las usuarias manifestaron quejas con respecto a la poca eficiencia del servicio de laboratorio, incluyendo reportes de extravío de los resultados de sus analíticas. En uno de los establecimientos visitados se observó una marcada desorganización de archivos de laboratorio, corroborando este tipo de quejas. A la vez, en algunos de los Centros las pruebas de laboratorio tienen un costo para las mujeres que no cuentan con un seguro de salud, lo cual incide en que muchas usuarias opten por posponer su realización hasta que puedan reunir los fondos necesarios. En algunos casos, esto se prolonga hasta los finales del embarazo. Algunos hospitales ofrecen la prueba rápida de VIH, con la entrega de resultados el mismo día. En un hospital recién se inició la práctica de la prueba rápida de VDRL, en el contexto del Programa Nacional para la Prevención de la Transmisión Vertical (PNRTV). En otros casos, no obstante, la usuaria tiene que volver una o dos veces más para retirar los resultados de sus analíticas de laboratorio, lo cual incide en un alto porcentaje de mujeres que no reciben el resultado de estas pruebas en el momento oportuno. 4.2.6. Intervenciones de información, educación y comunicación (IEC) Las actividades de IEC se realizan en todos los hospitales visitados, si bien por lo general tienen un carácter poco estructurado y contingente. Se abordan sobre todo temas de lactancia materna, VIH/ITS y planificación familiar. En la mayoría de los Centros este rol lo realizan las enfermeras quienes, con frecuencia, no cuentan con el entrenamiento necesario para el cumplimiento de estas funciones. Por otro lado, el espacio físico en el cual se realizan las actividades de IEC (usualmente las salas de espera y las salas de internamiento) no es el óptimo para promover la atención de las usuarias y su capacidad de escucha. Con excepción de un Hospital, con una amplia experiencia de integración comunitaria, no se observó una coordinación estructurada Hospital – Comunidad en la realización de las actividades de IEC. Asimismo, los médicos y otro personal directamente involucrado en la provisión de servicios no recibe entrenamiento para el manejo de las estrategias de IEC ni las habilidades de comunicación con las usuarias, lo cual dificulta la entrega de informaciones oportunas en el momento de la consulta prenatal y otros servicios de salud materno-infantil. Este tipo de actividades, con 156 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 frecuencia, tampoco es percibido como parte de sus responsabilidades en el contexto institucional. 4.3. Referencia y contra-referencia De manera general, los procedimientos de referencia y contra-referencia en los hospitales visitados están muy vagamente definidos y poco funcionales. En algunos casos los Centros de mayor nivel reciben referencias de los Centros de menor nivel, pero luego no existen mecanismos de contra-referencia ni se retroalimenta el proveedor original con respecto a los resultados de la atención provista al usuario referido. Más aún, en algunos hospitales de tercer nivel fue observada la actitud de “apropiarse” de la usuaria una vez la reciben en la institución, y no “devolverla” a la atención primaria: “Una paciente cuando llega aquí ya es definitivo, ya se le da la atención aquí…. Le damos la orientación de que cualquier situación tiene que venir aquí y no ir a su centro de atención primaria porque fue aquí que se le dio la atención... …Si llegó aquí ya no la orientamos a que vaya a otro sitio, ya ella es de aquí.” [GD1, proveedores] Aun así, en algunos hospitales fueron observadas experiencias novedosas de referencia y contra￾referencia utilizando los teléfonos móviles para la comunicación entre los establecimientos de diferentes niveles de atención, las cuales fueron descritas como muy favorables. Cabe señalar, no obstante, que en gran parte estas experiencias descansan sobre las relaciones primarias entre los proveedores de los diferentes establecimientos, más que sobre los procedimientos claramente establecidos e institucionalizados para estos fines. Por otro lado, algunos de los establecimientos visitados no cuentan con una ambulancia para el traslado de los usuarios a Centros de salud de más capacidad resolutiva. Aun aquellos que si cuentan con ella, en ocasiones carecen de combustible, por lo cual las usuarias han de cubrir los costos del mismo en el caso de necesitarse. 4.4. Centro de capacitaciones Todos los Hospitales visitados cuentan con salas de capacitación renovadas y equipadas a través del proyecto de USAID. Su utilización, no obstante, tiende a limitarse a capacitaciones puntuales provistas al personal del establecimiento, o a actividades docentes cotidianas impartidas a los residentes de las diferentes especialidades, identificándose como una oportunidad perdida la posibilidad de aprovechar estos espacios para un proceso de gestión de conocimientos en el contexto hospitalario. Entre las capacitaciones recibidas por el personal de salud entrevistado como parte del Proyecto de USAID, se destacan el manejo de emergencias obstétricas (COEm) y las capacitaciones impartidas en torno a la la bioseguridad y la ruta sanitaria. En este último caso parece ser particularmente relevante el empoderamiento logrado entre el personal de limpieza y el cambio de actitudes hacia el desempeño de sus funciones en el mismo. 4.5. Infraestructura y suministros Entre los principales cambios observados y destacados por las usuarias y los proveedores de servicios de salud como uno de los principales productos del Proyecto de USAID en los establecimientos visitados, se reporta una mejora radical en la adecuación y la limpieza de las instalaciones. Varios informantes entrevistados atribuyeron este cambio a una actitud de empoderamiento por parte del personal de limpieza, fomentada a través de las capacitaciones USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 157 impartidas. De manera acorde, el personal de limpieza entrevistado expresó sentir satisfacción frente a su inclusión en las capacitaciones, la orientación recibida, y la protección que están recibiendo en términos de bioseguridad. A la vez, los informantes claves entrevistados en los Centros de salud visitados afirmaron una importante mejora en términos de servicios básicos en sus establecimientos, incluyendo el suministro de agua y la energía eléctrica, entre las principales áreas de impacto del Proyecto. Así, como parte del mismo, varios Centros de salud pudieron ser equipados con tinacos e inversores que ayudan a manejar el problema de energía eléctrica y la escasez de agua en los mismos. En términos de suministros, las autoridades hospitalarias reportaron una mejora en los últimos años, afirmando, no obstante, que la misma tiene un límite, ya que se necesitan mejoras a nivel central de Promese para responder a pedidos de las farmacias en los hospitales de una manera más eficiente. 4.6. Gestión de riesgos y bioseguridad La ruta sanitaria y los procedimientos de manejo de desechos fue establecida en varios de los Centros visitados como parte de las actividades del Proyecto de Centros de Excelencia, si bien las observaciones directas realizadas en algunos de ellos indican que estos procedimientos no necesariamente se cumplen a cabalidad. Aun así, en todos los casos el personal de limpieza y el personal involucrado en el manejo de desechos utiliza la vestimenta adecuada y ha recibido las vacunas de antitetánica y Hepatitis B. La mayoría de los establecimientos visitados ha firmado acuerdos específicos con los Ayuntamientos locales para la recogida de los derechos hospitalarios, o está en proceso de su negociación. En un caso se reportó la ampliación de estos acuerdos entre el Ayuntamiento y las clínicas privadas del municipio, incidiendo de esta manera el Proyecto a mejorar la salubridad de la zona en el sentido más amplio. 4.7. Participación comunitaria Las autoridades de la mayoría de los centros visitados reportan, como parte de las actividades del Proyecto de USAID, la integración de un miembro de la comunidad en el consejo hospitalario. No obstante, las entrevistas cualitativas y las observaciones directas realizadas sugieren que se trata del cumplimiento de una formalidad, más bien que de una alianza funcional entre el Hospital y los actores comunitarios específicos. Con excepción de un hospital, caracterizado por una amplia experiencia previa en el trabajo comunitario en el contexto de otro proyecto implementado en el mismo, los proveedores de servicios de salud y las autoridades hospitalarias en los Centros visitados muestran poca apertura hacia este tipo de integración, saliendo a relucir una actitud de “Ellos (usuarios) vs. Nosotros (proveedores)” en varios de los establecimientos visitados. Cabe destacar la excepción que en este sentido representa el Hospital San Vicente de Paul (San Francisco de Macorís), en el cual se reportan experiencias muy exitosas de integración comunitaria, incluyendo el entrenamiento de dulas, comités comunitarios y de voluntariado integrados al trabajo del Hospital, así como la realización de investigaciones comunitarias protagonizadas por los líderes locales entrenados. De manera general, el funcionamiento de servicios de salud materno-infantil en este hospital parece estar más enfocado en la usuaria y el parto humanizado, en contraste con otros centros de salud visitados, involucrándose el personal 158 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 voluntario de las comunidades a las actividades cotidianas y la provisión de servicios en este establecimiento. 4.8. Emergencias y desastres Si bien las rutas de evacuación fueron establecidas en la mayoría de los Centros visitados como parte del plan de respuesta a emergencias y desastres, los simulacros correspondientes no se han realizado en todos los establecimientos, estando el proceso de preparación frente a emergencias y desastres en proceso en la mayoría de ellos. Las autoridades hospitalarias y los proveedores de servicios de salud reconocen el impacto del Proyecto de USAID en el diseño y la implementación de este proceso. 4.9. Oportunidades En adición a las observaciones y las entrevistas realizadas en torno a los componentes y los objetivos específicos del Proyecto de Centros de Excelencia, las visitas a los establecimientos de salud permitieron identificar iniciativas e intervenciones exitosas llevadas a cabo en el ámbito de la salud materno-infantil con otras fuentes de financiamiento, que pudieran ser tomadas en cuenta y, potencialmente, aplicadas y ampliadas en el contexto de este Proyecto. Se destacan en este sentido: Psicoprofilaxis de parto (Maternidad San Lorenzo de Los Mina). Dirigido y ofrecido gratuitamente a todas las embarazadas primigestas que acuden a servicios de atención prenatal, incluyendo ejercicios y preparación para el manejo de dolor en el proceso de parto, contribuyendo de manera importante a la humanización del mismo. Madres tutelares (Maternidad San Lorenzo de Los Mina). Dirigido a embarazadas adolescentes, implica el entrenamiento y la asignación de una líder comunitaria al acompañamiento y seguimiento de las embarazadas y las madres adolescentes desde el contexto comunitario hasta la asistencia al establecimiento de salud para los controles de embarazo, parto, post-parto, vacunación y consultas regulares de niño sano. Madre canguro (Hospital San Vicente de Paul, Maternidad San Lorenzo de Los Mina). Intervención dirigida a recién nacidos prematuros o de bajo peso, implicando la promoción de contacto piel a piel de 24 horas con la madre u otro familiar, y el seguimiento longitudinal para monitorear y fomentar las metas de desarrollo de los bebés durante el primer año de vida. Programa de dulas (Hospital San Vicente de Paul) entrenadas por el Voluntariado entre las líderes comunitarias vinculadas al trabajo del hospital, quienes acompañan a las embarazadas desde el contexto comunitario en la atención prenatal, hasta el momento de parto y post-parto, contribuyendo a la humanización de estos servicios y el manejo del dolor durante la labor de parto. Entrenamiento de enfermeras para la realización de partos (Hospital San Vicente de Paul) ha sido descrito como una práctica altamente positiva para los resultados de la atención a la salud materno-infantil en el Hospital, tanto por las enfermeras mismas como por los médicos especialistas entrevistados en el contexto de la presente Evaluación. La amplia capacitación y el empoderamiento del personal de enfermería para la realización autorizada y supervisada de partos fisiológicos, alcanzada a través de las intervenciones sostenidas por la Universidad de Emory en este Centro de salud, así como las asignaciones USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 159 claras y transparentes de sus responsabilidades en el proceso, parecen ser factores claves en este sentido. Integración comunitaria (Hospital San Vicente de Paul) promovida a través del trabajo del Voluntariado en el Hospital, incluyendo el apoyo de líderes comunitarios en la provisión de servicios intra-hospitalarios, el seguimiento post-parto a través de visitas domiciliarias en el contexto comunitario, el entrenamiento de dulas, y la investigación comunitaria participativa. Integración de la prueba rápida y gratuita de VDRL en el contexto del PNRTV (Hospital Antonio Musa), con la entrega de los resultados de la prueba de VIH y la prueba de Sífilis en el mismo día de la primera consulta prenatal, si bien reciente, representa una oportunidad importante para la detección temprana y oportuna de ambas condiciones de salud entre las embarazadas, incidiendo, entre otros aspectos, en la prevención y la reducción de la sífilis congénita. 5. CONCLUSIONES – Se observan niveles desiguales en el desarrollo de las actividades del Proyecto y su impacto en los diferentes Centros de salud visitados, así como entre los diferentes ejes temáticos del mismo. Las condiciones específicas de los diferentes establecimientos, así como el estilo de gerencia en cada uno de ellos, parecen estar vinculados a la magnitud de impacto y el desarrollo de las intervenciones introducidas por el Proyecto de Centros de Excelencia en su contexto. De manera particular, la gerencia de recursos humanos con una clara asignación de funciones, responsabilidades y jerarquías institucionales, así como los conocimientos, competencias y la motivación del personal para lograr cambios positivos en el desempeño de sus funciones, lucen ser determinantes para el impacto logrado por el Proyecto en cada Centro de salud. – Las actividades desarrolladas en los Centros de Capacitaciones en los establecimientos visitados tienden a limitarse a capacitaciones puntuales provistas al personal del establecimiento, o a actividades docentes cotidianas impartidas a los residentes de las diferentes especialidades, identificándose como una oportunidad perdida la posibilidad de aprovechar estos espacios para un proceso de gestión de conocimientos en el contexto hospitalario. – De manera general, los reportes de los proveedores y las usuarias, así como las observaciones directas llevadas a cabo en los establecimientos visitados indican un impacto altamente positivo del Proyecto en: Reparaciones y adaptaciones de la infraestructura, incluyendo su limpieza. Gestión del cambio (organización de registros y archivos, Oficinas de Atención al Usuario, materiales y suministros). Capacitaciones impartidas en el contexto del Proyecto, particularmente en el ámbito de bioseguridad (personal de limpieza) y Cuidados Obstétricos de Emergencia (COEm). Empoderamiento y motivación de los recursos humanos para el desempeño de sus funciones específicas en los centros de salud. 160 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Procedimientos de bioseguridad y manejo de desechos. Preparación frente a emergencias y desastres. – La Gestión Clínica constituye el área que amerita más reforzamiento por parte del Proyecto en estos momentos, dadas las múltiples debilidades identificadas en el ámbito de: Capacitación y supervisión del personal en el cumplimiento de normas y protocolos de atención; Integración entre diferentes programas y servicios, como garantía de una atención integral a la embarazada. Trato a la usuaria en los diferentes servicios, particularmente en la labor de parto y post-parto. Promoción y supervisión de los diferentes componentes del parto humanizado, según lo establecido en las normas nacionales, particularmente en lo relativo a las necesidades emocionales de las usuarias. Fortalecimiento de servicios de emergencia y laboratorio. Intervenciones de información, educación y comunicación a la usuaria. – Se observaron debilidades en la atención y la respuesta estructurada a las necesidades emocionales y psicológicas de las usuarias de servicios de salud materno-infantil en los establecimientos visitados. Entre otros aspectos, las enfermeras y otros proveedores de servicios de salud en salas de parto y post-parto no cuentan con el entrenamiento requerido para responder oportunamente a las potenciales crisis emocionales de las usuarias, incluyendo el desconocimiento de los mecanismos más indicados para el apoyo psicológico en los casos de abortos espontáneos, muertes neonatales y otros posibles desencadenantes de crisis en este contexto. Tampoco están claramente establecidos los mecanismos de referencia intra e interinstitucional de los cuales pudieran auxiliarse en el manejo de estas situaciones. – Los procedimientos de referencia y, particularmente, contra-referencia en los hospitales visitados están muy vagamente definidos y poco funcionales, si bien han podido observarse esfuerzos aislados para superar estas limitaciones, frecuentemente con base en las relaciones primarias entre los proveedores de servicios individuales en los diferentes niveles de atención. – La integración de miembros de la comunidad en los Consejos Hospitalarios se cumple en la mayoría de los Centros visitados como una formalidad, si bien, con excepción de un hospital, caracterizado por una amplia experiencia previa en el trabajo comunitario en el contexto de otro proyecto implementado en el mismo, no se observa una integración funcional de la Comunidad con el Hospital ni la realización de actividades conjuntas estructuradas en beneficio de la salud de la población intervenida. – Los Centros de salud incluidos en el Proyecto de Centros de Excelencia desarrollan iniciativas e intervenciones exitosas en el ámbito de la salud materno-infantil con el financiamiento parcial o total de otras fuentes, que pudieran ser tomadas en cuenta y, USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 161 potencialmente, aplicadas, ampliadas y replicadas en las fases subsecuentes del mismo. Se destacan en este sentido: Los programas de Psicoprofilaxis de Parto y de Madres Tutelares desarrollados en la Maternidad San Lorenzo de Los Mina. El programa de dulas, la integración e investigación comunitaria, el entrenamiento del personal de enfermería para la realización de partos fisiológicos, y el programa de Madre Canguro en el Hospital San Vicente de Paul (este último iniciado recientemente también en la Maternidad San Lorenzo de Los Mina). La integración de la prueba rápida y gratuita de VDRL en el contexto del PNRTV, llevada a cabo en el Hospital Antonio Musa. 6. RECOMENDACIONES De los hallazgos del Componente Cualitativo de la presente Evaluación se desprenden varias pautas para agilizar el desarrollo de las fases subsecuentes del Proyecto de Centros de Excelencia y optimizar su impacto en los establecimientos de salud intervenidos. En este sentido, se sugiere: Fortalecer la gestión de recursos humanos en los centros de salud intervenidos, incidiendo de manera directa en la descripción de puestos, asignación de funciones y responsabilidades, sistema de incentivos y la supervisión del desempeño individual y de equipos de salud. Fomentar la transformación progresiva de los Centros de Capacitación en unidades intrahospitalarias de gestión de conocimientos, más allá de entrenamientos puntuales del personal de salud, incluyendo: – Actividades regulares de formación continua que promuevan la actitud proactiva en la búsqueda y el análisis crítico de la información actualizada en el ámbito de la salud materno-infantil y la medicina basada en evidencia (realización journal club, foros de discusión, etc.). – Iniciativas de producción de conocimientos, tomando ventaja, entre otros aspectos, de las posibilidades de análisis de datos generados por los diferentes servicios del Hospital, realización de investigaciones clínicas y comunitarias, preparación de artículos y otras publicaciones científicas, etc. – Posibilidades de reproducción de experiencias exitosas en la atención a la salud materno-infantil de Centro a Centro, en función de las prioridades y las áreas de especialización de cada establecimiento. Completar la ejecución y consolidar las intervenciones desarrolladas hasta la fecha en los establecimientos incluidos en el Proyecto de Centros de Excelencia en los ámbitos de gestión de cambio, adaptaciones de la infraestructura, bioseguridad y preparación frente a emergencias y desastres, incentivando su replicación en otros establecimientos de salud de las regiones correspondientes. Priorizar las intervenciones dirigidas a mejorar la gestión clínica y la calidad de la atención provista a las usuarias de servicios de salud materno-infantil, con énfasis en la divulgación, implementación y supervisión del cumplimiento de las normas nacionales establecidas para 162 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 estos fines y, particularmente, la humanización de estos servicios desde una perspectiva integral. Fomentar la comunicación y la interconsulta entre los diferentes programas que operan en el contexto intrahospitalario, con el fin de reducir las oportunidades perdidas y promover una atención integral en el ámbito de la salud materno-infantil que responda de manera oportuna a las necesidades de salud de mujeres embarazadas, madres y sus hijos/as. Proveer capacitación estructurada al personal de enfermería y otros proveedores involucrados de manera directa en la atención al embarazo, parto y puerperio, para responder a las necesidades psicológicas de las usuarias de estos servicios e intervenir de manera oportuna en el caso de potenciales crisis emocionales en este contexto. Precisar los mecanismos de referencia y contra-referencia entre los establecimientos de salud de diferentes niveles de atención, explorando las posibilidades de ampliar y generalizar las iniciativas implementadas con éxito y la disponibilidad de nuevas tecnologías para fomentar la comunicación directa entre los diferentes actores del sistema a través de la utilización de la telefonía móvil. Fortalecer, de manera prioritaria, la integración funcional de la gestión hospitalaria con las organizaciones y líderes comunitarios, reproduciendo las experiencias exitosas observadas en algunos de los establecimientos intervenidos, y tomando ventaja de las múltiples oportunidades que este tipo de integración implica para la calidad de los servicios de salud materno-infantil en este contexto, incluyendo: – El fomento del empoderamiento de los actores comunitarios en el conocimiento y el reclamo efectivo de sus derechos a recibir servicios de salud con calidad y calidez. – Integración de líderes comunitarios como voluntarios en la entrega de diferentes servicios provistos en el ámbito hospitalario, promoviendo su identificación con las fortalezas, las necesidades y los retos presentados en este proceso, así como el uso de recursos comunitarios para la superación oportuna de los mismos. – Involucramiento de líderes comunitarias en el acompañamiento de las mujeres en el proceso de embarazo, parto y puerperio, contribuyendo de esta manera a la humanización de servicios de salud materno-infantil en el ámbito hospitalario, y estableciendo canales permanentes para el seguimiento post-parto en el contexto comunitario. – Establecimiento de canales directos de comunicación entre los hospitales y la comunidad, facilitando el diseño y la implementación de intervenciones y actividades puntuales de IEC de manera que responda oportunamente a las necesidades sentidas en las comunidades especificas. – Realización de investigaciones comunitarias participativas, como un mecanismo para la transferencia de capacidades y competencias de gestión de conocimientos a miembros de la comunidad, empoderando a las comunidades como actores claves en el análisis crítico y la respuesta a sus propias necesidades de salud. Analizar la factibilidad de alianzas estratégicas con otras iniciativas y programas exitosos llevados a cabo en los Hospitales intervenidos con el fin de mejorar la calidad de servicios USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 163 de salud materno-infantil en los mismos, fomentando la transferencia de conocimientos y buenas prácticas entre los diferentes establecimientos en función de sus intereses y experiencias específicas. 7. Cuadro 1. Resumen de datos recopilados por Centro de Salud D Localidad / Hospital Técnicas de recopilación de datos Observaciones / Grupos de discusión comentarios Entrevistas cualitativas puntuales Observación cualitativa (áreas de atención) Lu 5 Santo Domingo Este Maternidad San Lorenzo de Los Mina Usuarias (n=6): Mujeres que han dado a luz en los últimos 3 años, 19-26 años. Una de ellas en el programa de adolescentes. Tres embarazos de alto riesgo. Proveedores (n=4): Residentes de II, III y IV año de Gineco￾Obstetricia, en áreas de Cirugía Ginecológica, Post￾quirúrgico, Post￾parto. Usuaria adolescente que estuvo en cuidados intensivos durante 2 semanas por eclampsia severa en el parto. Bebé sano de 4 meses. Asistente de la Dirección del Hospital (acompañando las observaciones realizadas). Equipo administrativo del Hospital (a manera de entrevista grupal). Programa Psicoprofilaxis de Parto (área equipada por USAID) Laboratorio Ruta sanitaria Áreas nuevas del hospital (anexo) Salas de espera de consulta externa, incluyendo consulta de adolescentes. Participantes en el grupo de usuarias fueron seleccionadas y convocadas por la Dirección del Hospital. Aun así, no parece haber intervenido una selección dirigida de casos en el proceso, incluyéndose una madre con historia de muerte neonatal / eclampsia en el mismo hospital. Ma 6 Cotuí Hospital Inmaculada Concepción Usuarias (n=14): Mujeres que han dado a luz en el último año, 16-33 años. NOTA: Convocadas por la Dirección del Hospital. Proveedores (n=9): Enfermeras de Maternidad, áreas de pre y post parto, con 4 a 28 años de trabajo en el mismo Hospital. Equipo administrativo del Hospital (a manera de entrevista grupal). Jefa de enfermería (acompañando las observaciones realizadas). Usuaria de 19 años, interna en la sala de alto Salas de internamiento (alto riesgo, post-parto). Manejo de una usuaria en sospecha de post-aborto espontaneo (resultó no estar embarazada) Áreas de consulta externa. Participantes en el grupo de usuarias fueron seleccionadas y convocadas de manera dirigida por la Dirección del Hospital. En varios casos se trataba de sus familiares y otros/as allegados. 164 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 D Localidad / Hospital Técnicas de recopilación de datos Observaciones / Grupos de discusión comentarios Entrevistas cualitativas puntuales Observación cualitativa (áreas de atención) riesgo por amenaza de aborto. Pareja de una mujer de 33 años, quien tuvo una cesárea (gemelos) en el hospital hace 5 meses. Mi 7 San Francisco de Macorís Hospital San Vicente de Paul Proveedores (n=7): Médicos involucrados en consulta externa. NOTA: Cada tres meses se turnan por diferentes áreas. Encargada del Programa Canguro. Presidenta del Voluntariado del Hospital. Equipo de gineco￾obstetricia: jefe de servicio, enfermeras, dulas, investigación comunitaria (a manera de entrevista grupal). Madre canguro en la sala de prematuros, 19 años / su familiar en la sala de usuarias. Programa Canguro (consulta de seguimiento / sala de prematuros). Salas de internamiento (pre y post parto). Consulta externa. Espacio del voluntariado del Hospital. No se pudo convocar el grupo de madres canguro graduadas, debido a la programación de la huelga general en el municipio en el día de la visita. Se optó por observar los servicios de este programa y conducir entrevistas informales con las usuarias presentes en las consultas de seguimiento. Ju 8 San Juan de la Maguana Hospital Alejandro Cabral Usuarias (n=6): Mujeres embarazadas en consulta prenatal, 20-27 años, con por lo menos un parto previo en el mismo Hospital. Proveedores (n=4): Personal de emergencia de Jefa de enfermería (acompañando las observaciones realizadas). Usuaria que tuvo un parto fisiológico en el Hospital hace 8 Unidad de Atención a la Usuaria Ruta sanitaria Consulta externa Emergencia Salas de internamiento Se realizó una entrevista cualitativa con una usuaria que tuvo un parto fisiológico en el Hospital hace 8 meses, para contrastar la información obtenida en el grupo de USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 165 D Localidad / Hospital Técnicas de recopilación de datos Observaciones / Grupos de discusión comentarios Entrevistas cualitativas puntuales Observación cualitativa (áreas de atención) diferentes perfiles (Residente de Medicina Familiar, Enfermera, personal de limpieza, orientadora). meses. discusión con usuarias. Vi 9 San Pedro de Macorís Hospital Antonio Musa Usuarias (n=9): Mujeres embarazadas en consulta prenatal, 21-34 años, algunas con partos previos en el mismo Hospital. Encargada de la Oficina de Atención al Usuario (acompañando las observaciones realizadas). Encargado del Departamento de Educación (actividades de IEC en salas de espera de consulta externa / salas de internamiento) Personal de laboratorio Unidad de Atención al Usuario Archivo Consulta externa Área de educación Salas de internamiento Emergencia Laboratorio Imágenes Estaba previsto un grupo de discusión con el personal involucrado en las actividades de IEC en el Hospital. Sin embargo, al contar este departamento con solo 2 proveedores, se optó por realizar una entrevista cualitativa con el encargado del departamento. 166 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 167 ANNEX 16. MID-TERM EVALUATION OF USAID’S INTERMEDIATE RESULT 4 OBSERVATION RESULTS AND RECOMENDATIONS Dr. Olga Arroyo, Coordinadora Nacional Programa Perinatal. Ministerio de Salud de la Republica Dominicana Gestión clínica Elementos a avaluar: Atención consulta prenatal Atención embarazo y parto Estrategia Mama Canguro Maternidad Centrada en la familia Prevención Sepsis neonatal Ayudando a los Bebes a respirar Metodología: Se realizó observación directa a la atención ofrecida por prestadores de salud a las embarazadas y a la parturientas, se revisaron los expedientes que contenían las historia clínica de estas pacientes, se hicieron preguntas a los prestadores vinculado a la atención y a las destrezas y habilidades que presentan, y al final se hizo un juicio clínico por parte del observador en relación a la atención brindada como a la competencia demostrada por el prestador. Se hizo observación del área física de las unidades de neonatología y sala de partos relacionadas con la higiene e insumos necesarios para mantenerla. La eleccion de los consultorios para la evaluación de la consulta prenatal de primera vez o subsecuente se hizo al azar. La recolección de la información la obtuvimos a través de varios formularios y apuntes. Hallazgos: Las areas visitadas para la observación, no son las mismas en todos los hospitales. En la Maternidad de Los Mina observamos sala de preparto, parto, puerperio y unidad de Neonatología; en el Hospital Alejandro Cabral de San Juan sala de preparto, puerperio, consulta, Unidad de neonatología; en el Inmaculada Concepción de Cotui sala de partos, preparto, puerperio, consulta y unidad de neonatología; San Pedro sala de parto, preparto, cirugía, consulta prenatal y unidad de neonatología. 1. Atención parturienta en Sala de partos y prepartos, tres centros de salud : Se observo que: – Las salas de parto de los hospitales evaluados son áreas rotuladas y restringidas, se observaron limpias, con pisos, paredes y techos lavables, y disponen de lavamanos con agua y jabón. – Trato personalizado con respeto a la parturienta, por ese momento. 168 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 – Indaga sobre número de partos anteriores y hace pregunta para calcular edad gestacional. – Realizó evaluación abdominal, signos vitales. – En todos los centros de salud el prestador vestía con vestimenta limpia adecuada al área. – Todos los prestadores se colocaron guantes estériles antes de la evaluación a la parturienta. – En todos los centros habían equipos limpios para la atención de la parturienta y el recién nacido. – En ningún centro de salud el proveedor de salud se lavó las manos antes ni después de las evaluaciones a la parturienta. En dos centros de salud se observaron los siguientes elementos: – No había historia clínica perinatal en el expediente de la paciente ni carnet perinatal. – El Partograma se llenó de manera incompleta. – hubo ausencia del resultado de las pruebas de laboratorio en el expediente. – EL prestador a cargo de la atención al parto desconocía los antecedentes maternos. – El prestador interrogado manifestó que tenía habilidades y destrezas para reanimar a un recién nacido. (cabe destacar que estos prestadores son médicos obstetras). En un centro de salud se observaron las siguientes acciones: – No revisaron la placenta después su extracción, ni se realizó pinzamiento tardío del cordón. – El proveedor de salud se mantenía alejado de la parturienta durante la labor de parto. – Ofrecieron los cuidados básicos inmediatos durante el nacimiento a los recién nacidos: secado de la piel, la lactancia materna durante el primer minuto, colocación piel a piel a su madre, profilaxis ocular . – Se observaron peritas rehusadas. – Hubo necesidad de campos estériles. 2. Área de consulta prenatal, tres centros de salud: – El prestador de salud saludó y trató con respeto a la embarazada durante toda la visita. – Indaga sobre número de partos anteriores y hace pregunta para calcular edad gestacional. – Realizó evaluación abdominal, signos vitales y peso. – Indicó las analíticas correspondientes a la embarazada – Prescribió tratamiento con hierro y ácido fólico. – Llenó una historia clínica a la paciente USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 169 – No se lavó las manos antes ni después de la atención ofrecida a las embarazadas. – No indagó sobre antecedentes de infección de vías urinarias ni examina mucosas y piel buscando signos de anemia. En dos centros de salud: – El prestador de salud registra en el expediente los datos de la parturienta – Se mantuvo la privacidad y confidencialidad de la paciente – No se llenó la Historia clínica perinatal base. – Llenó carnet perinatal – Contaban con lavamanos y agua corriente, en ausencia de jabón y toallas desechables y un consultorio sin lavamanos, agua, jabón ni toallas desechables. En un centro de salud – El prestador no dejó constancia en el expediente de la atención ofrecida a la embarazada. – No disponía de esfigmomanómetro. 3. Atención al recién nacido, área de recepción en un centro de salud: En un hospital observamos la atención a dos recién nacidos que se le brindaron los cuidados básicos inmediatos de manera adecuada: Secado de la piel Lactancia materna durante el primer minuto Cubre el cuerpo del recién nacido con paño seco Colocación piel a piel a su madre Profilaxis ocular . Técnica limpia Observación Unidad de neonatología cuatro hospitales:. Un hospital ofreciendo atención al recién nacido bajo peso y al prematuro a través de la Estrategia Mama Canguro con todas las intervenciones requeridas. Un hospital ofreciendo atención al recién nacido bajo peso y al prematuro a través de la Estrategia Mama Canguro con algunas intervenciones. Un hospital ofreciendo atención a la madre y al recién nacido bajo peso y al prematuro a través de la Estrategia Maternidad Centrada en la Familia con escasas intervenciones. Las unidades de neonatología y sala de partos de los centros evaluados todas son áreas restringidas, limpias e iluminadas y cuentan con lavamanos con agua y jabón dentro de la Unidad excepto en un hospital que no disponía de agua ni jabón para el lavado de las manos. 170 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Cuentan con mensajes educativos alusivos al lavado de manos y a la lactancia materna. Tres hospitales observados con atención humanizada al recién nacido Conclusiones: En una línea de base realizada por el Proyecto Centro de Excelencia en el año 2010 , para la implementación del sistema de vigilancia, prevención y control de infecciones intrahospitalarias en los hospitales seleccionados se concluyó que la mayoría de los hospitales requerían de reparación y/o restauración de las estructuras físicas de las áreas evaluadas, mejoría en el manejo y disposición de los desechos sólidos, necesidad permanente de insumos para el lavado de las manos e insumos para la protección de los pacientes y del personal de salud, colocación de rótulos de restricción y materiales educativos alusivos al lavado de manos en las salas de preparto, parto y neonatología y capacitación al personal de salud. Por lo que concluimos que existen evidencias concretas de mejoría para la atención materna y del recién nacido, como son: Se brindó atención con calidad al recién nacido El prestador de salud observado en la atención a la embarazada y parturienta tiene las habilidades y destrezas para la atención al parto y el embarazo, aunque debe mejorarla. El prestador de salud no aplica en su totalidad las normas y guías de atención al embarazo parto y puerperio. Las intervenciones de la Estrategia Mama Canguro y Maternidad Centrada en la familia están incompletas. Las condiciones de higiene de la sala de partos y neonatología ha mejorado Los hospitales observados cuentan con sala de partos y neonatología aptos para brindar una atención con calidad. La mejoría en la estructura física de la sala de partos y unidad de neonatología de los hospitales observados contribuyen con la disminución de los factores de riesgo para la aparición de infecciones en la madre y el recién nacido, aunque las intervenciones para prevenirlas deben mejorar. La mayoría de los hospitales cuentan con los insumos necesarios para evitar los riesgos de infecciones nosocomiales. Recomendaciones Realizar supervisión sistemática a través de las guías de Supervisión Clínica, Para conocer la aplicación de las normas y guías de atención al embarazo parto y puerperio, y atención al recién nacido. Establecer coordinación entre el archivo y emergencia o sala de partos, para la recuperación rápida de los expedientes maternos. Promocionar y vigilar el cumplimiento del lavado de manos, y proporcionar los insumos necesarios en las áreas de atención a la madre y al recién nacido. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 171 Establecer los mecanismos necesarios para que los expedientes de las parturientas cuenten con las analíticas y estudios correspondientes durante la labor de parto, y los responsables de la atención conozcan los resultados. Recomendar al prestador de salud, hacer énfasis en las pruebas del VIH y Sifilis en el momento del parto. Promocionar el pinzamiento tardío o cuando deje de latir el cordón umbilical Crear las condiciones para el acompañamiento de la parturienta con un familiar durante la labor de parto. Recomendar completar e llenado del Partograma durante la labor de parto. Promocionar el apego precoz al primer minuto. Mejorar la promoción de la prevención del VIH /SIDA y sífilis congénita a nivel comunitario, con énfasis en la provincia San Pedro. Reiterar la necesidad del llenado de la historia clínica perinatal base y el Carnet perinatal y recomendar a la madre llevarlo al momento del parto. Mejorar la confidencialidad durante la consulta Solicitar a la parturienta durante la atención, el carnet perinatal base o en su defecto, cualquier instrumento que contenga los datos maternos. Monitorear la adherencia al lavado de manos y la limpieza de los centros. Disponer de los insumos necesarios para prevenir las enfermedades nosocomiales 172 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 173 ANNEX 17. CLINICAL CARE OBSERVATIONS Dr. Olga Arroyo and Dr. Peg Marshall observed clinical care in the labor, delivery, post-partum, and neonatal nursery areas, antenatal clinic, family planning clinic, and neonatal clinic. GENERAL OBSERVATIONS AT LOS MINAS: The visit started with reception by management and then moved to the labor and delivery area. We were able to observe two deliveries and watch the management of six laboring women. The physical area is extremely tight, allowing minimal privacy. There were 14 personnel in the area with the six women in addition to the two observers. There are approximately four physicians/residents per nurse. The role of the nurses was basically to give the pitocin, wash off the perineum pre- and post-birth, and clean the delivery room. The residents help women go to the delivery room, do the charting including the partogram, and write post-partum orders. Interpersonal care: In Los Minas women are not consistently called by their names, though this is something that has been worked on. Women are referred to as Dona (roughly madam). Personnel stand at the woman’s feet and talk to or over her from this position. No nurse or doctor took a patient’s hand, comforted her, or gave advice on how she could relax or breathe for pain relief. This was true for both labor and delivery rooms. There was an effort to provide privacy with curtains around roughly half the patients. One patient without a gown was given a see-through over-gown. One patient had a vaginal exam without any effort to pull the curtains and the woman was not communicated with before he put his hand in her vagina. Care at Cotui was observed in the new antenatal clinic and post-partum post-natal care. There was no sink in the exam room and the obstetrician completed multiple physical exams and charting never once washing her hands or using hand sanitizer. Care at Musa care was observed in the neonatal clinic, family planning clinic, and labor ward. There was a severely pre-eclamptic adolescent in active labor in a darkened ward by herself crying. In spite of 16 personnel standing within twenty feet of her, no one entered her room to give her emotional and physical support. There were 12 obstetricians and medical students clustered around a desk chatting. Three nurses were washing and wrapping instruments. When it was pointed out her anxious family were all standing in the hall outside, the head nurse stated there is not enough room for families in labor and delivery and that that is also why there are no stools in the labor room for anyone to sit at the patient’s side and give labor support. In fact there was plenty of room for stools at every bedside. On the same unit not more than 10 feet from the huge cluster of personnel two post-operative patients were out of bed, one helping the second get her IV bottle down so she could go to the bathroom. In the neonatal clinic there was a sink but the neonatologist never washed his hands. He laid each baby on the same blanket, undressed them, touched them including examination for descended testicles and rewrapped the babies. When no baby was in the room, he sat on the same blanket. These babies were all 7–25 days of age. 174 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Observations: Respectful Care/ Humanization of Care There is a widely variable understanding of what humanization of care is. You see it implemented in the most elementary way at Cotui, where the nurses think that referring to women as “my love” constitutes good care yet harbor vehemently anti-gay attitudes and don’t work to support breastfeeding in the post-partum unit. to Las Minas, where using the woman’s name or referring to her as Dona is seen as respectful to care, to San Pedro de Macoris, where care of women is much more textured. There problem-solving is part of good care, e.g., finding more comfortable types of chairs for breastfeeding women, including fathers in a more integral fashion, loosening visiting hours, etc. Humanization of care represents a rights and equity perspective and requires for implementation an empowered staff who see problem-solving as an integral and exciting part of their job. Barriers to overcoming this are A markedly skewed physician/nurse ratio. Lack of clear policies that empower nurses to perform normal deliveries, episiotomies, and other skills under their own authority and signing for their own work (e.g., birth certificates). Lack of functioning models of humanization of care from which to stimulate change in their own environments. Lack of ongoing continuing education for nurses. Racism against Haitians, which bubbles up in various contexts. Lack of cultural awareness/demand for more choice/control of health care options including preparation for childbirth classes, nonseparation of families in the hospital, knowledge of risks of nonemergency procedures, such as c/sections and long-term morbidities and disadvantages for both mother and child, advantages of breastfeeding for all infants, and so on. This is a complex list of societal factors, training factors, facility factors, and human good will which will require a long-term, multi-pronged approach. Almost to a person staff communicate with women standing at the foot of their bed. Units need to have stools at the bedsides so staff can provide friendlier, more intimate communication. Supportive labor care will not take place if staff cannot comfortably sit and support laboring women. Recommendations: Identify key change agents from nursing, medicine, and administration to constitute a core team to work on this. Send the core team on a study tour to see family-centered care in action of various models. Potential sites might include the midwifery maternity hospital at San Martin de Porres in Lima Peru, the perinatal poverty hospital volunteer model in Buenos Aires, Argentina (a team from Musa has been sent there), the Centering Pregnancy model of antenatal care in the US, some of the humanization of care shakers and movers in Brazil, etc. Provide periodic and ongoing continuing education to nurses on this topic using the key champions to (1) set up a working model in one of the DR institutions where others can USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 175 come to visit and learn, and (2) conduct ongoing education sessions for nurses via Illuminate or other methodologies whereby they can learn and share in an interactive format. Establish a formal set of preparation for childbirth and parenthood classes that have certified instructors and up-to-date evidence-based practices and information. Support this program with community education materials. Investigate the ability of Jennifer Foster from Emory University Midwifery Program and Irene de la Torre from the University of Puerto Rico Midwifery Program to help in this process. Observations: Information System Strengthening In spite of great investments in improving hospital records archives, they have not translated into consistent improvement of care for patients. There are a number of areas of concern: 1. The recently adopted SIP PAHO form is not used well. The font is so small many staff cannot read the chart and note they write the notes in the margins since they can’t see where it goes. Others complain that it is too long, requires too much information, and duplicates information written in other parts of the chart. Rarely did we find syphilis or HIV testing recorded on the record. The SIP form is not used as a patient management tool as data are not synthesized and no management plan is articulated so that the team knows the plans for an individual patient. 2. In a number of hospitals patient records are not routinely sent to labor and delivery when a patient is admitted; thus the delivery staffs are “flying blind” regarding the patient’s history and antenatal care. When asked why so few labor patients had their records, someone was sent for them and they were available. 3. In one institution (Cotui) antenatal records were filled out and given to the patient. A second unfilled-out copy was kept at the hospital “to be filled out later.” It is unlikely that filling in those copies after seeing 15-18 patients is of any value and might be quite dangerous. 4. Often women delivering at referral hospitals do not have/come with their prenatal records. This clearly impacts quality of care as laboratory and other data are unknown. Recommendations: 1. Institutions should consider reprinting the SIP forms on two pages instead of one in a larger font with enhanced space for management notes. 2. The issues of filling in forms when the patient is still present, records routinely being sent upon admission, and referral of records when patients deliver at other institutions need to be sorted out. 3. Institutions need to work out systems so that laboratory results are located and placed in the patient’s chart prior to her next visit. Records missing lab data need to be tagged so that the patient is sent at the next visit to complete her lab work. This function could be handled by admin staff, clinic nurses, records managers, etc. 4. Develop consensus as what the minimum common information is to be filled out so each individual practitioner does not develop their own standard. 5. Analyze the information and give feedback to the staff filling out the forms. Seeing their own practice gaps should help improve quality of care. 176 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Observations: Currency in Practice Issues Currency in practice varies widely from site to site and professional to professional. It was clear at Las Minas that AMSTL is widely practiced and correctly, from the two deliveries observed. The nurses at Cotui, from an oral description of what they do, do not know AMSTL. The nurses at San Pedro de Marcoris noted they have been trained and use it consistently but detailed description was not sought. Neither nursing nor medical staff at any site seem to be aware of the literature regarding excess morbidity and mortality of both mothers and neonates resulting from overuse of c/sections, labor inductions, neonatal blood gases, etc. The physicians seem to recognize that their rates are high (32 to 49 per cent); however, they did not have an articulated approach to reduction of these rates. Most of the nurses interviewed did not have a real sense of what is a reasonable rate and most did not link post-partum early formula feeding to excess surgical interventions and condition of the mother post-partum. Clearly medical students are not sitting with laboring women and learning what labor is all about. It is very easy to rush to surgery when the normal is not well understood. There is an overreliance on technology as clinical judgment and skills take on less importance. In antenatal clinic the obstetrician observed did not use last menstrual period or size of uterus to determine the estimated date of confinement (EDC). She stated the EDC would be determined from the sonogram. She sends patients for sonograms every trimester, which is not in sync with MOH established protocols. It is very concerning from the skilled provider interviews that common skills are not mastered/practiced. One obstetrician noted she cannot resuscitate a baby but doesn’t need to as the neonatologists are there. Nor could she manage a patient on magnesium sulfate. Knowledge of management of assisted deliveries is also clearly a dying skill and is likely a substantial contributor to excess c/sections. Nurses do not exhibit skills in providing supportive labor care from the observations seen. It is not clear what expectations are for them from supervisors. Most nurses have not been updated in AMTSL or HBB. Recommendations: 1. An assessment of individual provider skills is essential with tailored continuing education offerings on a regular basis. 2. Supervisors need to review patient management and discuss sloppy care that does not use MOH policy guidelines and identify providers in need of educational and mentoring support. Observations: Basic Hygiene There is great concern regarding implementing basic standards of hygiene. In Las Minas with a significant delivery load of births, there is not a sink in the delivery or labor units. Hand sanitizer is used pre-vaginal exams (and erratically post-exam). In Cotui there is erratic water supply, erratic soap supply, no sink in labor and delivery. In San Pedro there are water, soap, and sinks. In Las Minas, Musa, and San Pedro there is an area for fathers/families to wash up and dress for the nursery and easy access to soap and water for nursery personnel. In Cotui there is not a sink in the examination room nor was hand sanitizer available. The obstetrician filled charts, examined the patient physically including a vaginal exam, sat down and continued charting, touching her face, furniture, pen, etc., and repeated the scene over and over with each patient USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 177 Issues related to the two deliveries observed in Las Minas: The residents did a good job with AMTSL but neither resident examined the placenta for completeness or normalcy. This is important in prevention of delayed PPH and identification of newborn congenital problems. The resident who performed an episiotomy did not have a clean/sterile field so had to balance the suture in her hands and without adequate lighting repair the episiotomy. The norm is to perform medio-lateral episiotomies despite their disadvantages for the mother. In the two cases observed, the infant was placed on the mother’s chest immediately upon delivery but whisked away immediately upon cutting of the cord and showing the mother the sex of the infant. Given the poor delivery conditions and lack of physical space and security on the broken delivery beds, this is likely not such a bad thing. It will be important to improve these behaviors as they move into new spacious rooms with labor/delivery/post-partum beds that are state of the art. Recommendations: If Semmilweis were not already dead, he would die to see the lack of scrupulous attention to hygiene in these institutions. It is difficult to change the culture of hand washing when soap and water are not available as a routine. We are training a whole generations of practitioners to “make do” instead of loudly demanding these basics for safe care. 1. First, administration must see that soap and water are continuously available in all clinical areas. There is no excuse good enough to justify no sink in the labor and delivery rooms in a hospital doing 12,000 deliveries . . . or any other hospital. It is not acceptable to see multiple neonates or antenatal patients without washing hands between each patient. 2. Nosocomial infection rates should be graphed and posted in the clinical areas monthly. 3. There needs to be work on empowering epidemiologists and infection control nurses to demand provider compliance with hygiene standards with direct access to the hospital director to ensure enforcement. 4. Names of providers and their compliance with hand washing pre-vaginal exam and pre￾delivery should be listed on the wall monthly. Observations: Installing Kangaroo Mother Care Kangaroo care was initiated over 18 months ago at San Pedro de Macoris and one month ago at Los Minas after a team (neonatologist, nurse, and psychologist) was sent to the Kangaroo Foundation in Colombia for training. This program has undergone an amazing evolution over this time. Initially data were not recorded and hospital support was tepid. Now data are carefully maintained and the service has become the darling of the hospital, enjoying lots of publicity, earning a prize from the first lady, and creating an empowered, committed, problem￾solving staff. Due to a strike, mothers did not come in the day we visited (though I have observed this service previously) for follow-up monitoring and care. In Los Minas the project does not yet have dedicated space. In spite of space and equipment needs, the staff has very energetically begun, enrolling 33 babies the first month. They were delighted to report that no baby had needed to be re-hospitalized, in contrast to a usual rate of 20 percent. Enthusiasm is running very high and clearly there are several committed change agents working to make it happen. 178 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Staff in San Pedro is very concerned that the next step be providing continuing education to staff from all the referral institutions. They note deaths of neonates from lack of proper referral (e.g., a baby that was not kangarooed for transport and arrived with terminal hypothermia). Recommendations: 1. It is so clear that nursing staff at San Pedro in both neonatal and maternity services are much more empowered, self-confident, and professional in their practice than other hospitals observed. Inputs from the midwives from Emory University and KMC training have been extremely positive. Providing multiple opportunities for nurses from the other centers of excellence is key to achieving a tipping point for high quality individualized care. 2. Support for starting KMC in multiple hospitals could be hastened by getting equipment and supplies for several at the same time rather than waiting for each to figure out the supply chain (patient fajas, special examination desk, data software, retinascope, etc, etc). 3. Support for continuing education on baby transport is needed. Institutions should likewise not be allowed to refuse a transported infant. Multiple instances of deaths were described as no one had called ahead regarding transfer of a moribund infant. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 179 ANNEX 18. RESULTS OF KEY INFORMANT INTERVIEWS AT SRS, SDP AND HOSPITAL LEVELS EVALUACIÓN DE TERMINO MEDIO DEL PROYECTO CENTROS DE EXCELENCIA MATERNO INFANTIL Republica Dominicana. Abril-Marzo, 2012 Entrevista a Informantes Claves: HOSPITALES REGIONALES Y PROVINCIALES RESULTADOS Nombre del establecimiento: Los Mina, San Vincente de Paul, Alejandro Cabral Entrevistador: Derek Sedlacek Fecha: 3/11/2012 Conocimientos generales acerca del proyecto: Hallazgos: All of the staff with whom I interacted from each of the hospitals I visited knew about the project and had a positive persepective on what had been implemented. At the same time, the lower the level of the staff member, the less aware they were of the full scope of the interventions that were taking place. Conclusiones: While the discrepancy in information between levels is normal, there is critical information that is not reaching crtitical players (example: The full scope of the OAU has not been described to patients or providers). Selección de los Centros de Excelencia (CDEx) Planificación Hallazgos: The interventions in strategic planning, change management, and the postgraduate diploma in health and social security management were all identified as important contributors to the success of the project in the hospitals. Each of the hospitals has been able to develop its own operational plan and was supported in the creation of quality improvement plans and strategic plans. Conclusiones: This is perhaps the most sustainable process of the intervention, and is a critical step for any replication. Monitoreo, Evaluación y Sistemas de Información Hallazgos: While the information system has been improved with the implementation of SIGHO, monitoring and evaluation remains weak. Conclusiones: SIGHO has been a fundamental tool, but working with the DPS to systemitize M&E is critical for improvements in the quality of care. Gestión de Recursos Hallazgos: The hospitals have limited sources of income generation, but different hospitals are implementing diverse strategies to tap into the private insurance market. All of the hospitals are 180 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 struggling with claim rejections, which are limiting additional resouces. Furthermore, due to delayed payments and other reasons, several of the hospitals have acquired significant debt. Conclusiones: The hospitals need to diversify their sources of income generation and improve their claims processing. Recomendaciones: Perhaps a separate component on income generation is needed. Atención Humanizada Hallazgos: Heterogeneous results among the different hospitals, but overall the humanization of care remains very superficial. Conclusiones: While this is perhaps a substantial improvement compared to the starting point, there is still much to do in this area. Recomendaciones: Atención del Parto: Estadísticas del servicio Intervenciones Prioritarias Hallazgos: Extensive implementation of Emergency Obstetric Care and AMTSL. Limited implementation of priority neonatal interventions. Never heard about Helping Babies Breathe although there should be trained persons in each of the hospitals. Conclusiones: Emergency Obstetric Care and AMSTL are close to a level of sustainability, but training for these interventions needs to be integrated into pre-service training or very early in in-service training. Helping Babies Breathe and Kangaroo Mother Care have the potential to be more sustainable, but have been implemented on a very superficial level to date. MORTALIDAD MATERNA Mortalidad Infantil Control de infecciones Hallazgos: The infrastructure and systems are in place to sustain these interventions. The behavior component is still weak. Conclusiones: The structural interventions implemented to date have been important for the sustainability of this component, but the behavioral component needs continued follow-up. Recomendaciones: This component is ready for replication and will help reinforce the behavioral component in all hospitals. ARTICULACIÓN ENTRE LOS DIFERENTES NIVELES Habilitación Oficina de Atención al Usuario Hallazgos: One of the most referenced improvements by all of the stakeholders. The full breadth of the intervention is either yet to be realized or not known by all that need to be aware (examples: Complaint Department, Appointments, interaction with the community). USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 181 Conclusiones: Without all the stakeholders being informed, this component will not reach its full potential. Recomendaciones: Beyond full implementation, this component needs to be promoted to both providers and patients. Sistema de Gestión de Suministro Hallazgos: Heterogenous implementation in the hospitals. The SIGHO application was available in all of the sites that I visited. Conclusiones: There has been tremendous progress in this component. Recomendaciones: This component is ready for replication and will help reinforce continued improvements in all hospitals. Plan de Manejo de Emergencias y Desastres Hallazgos: All of the hospitals recognized this component as an important component of the project. At some point there was an issue with the central level, and implementation of this component essentially stopped. Conclusiones: There was a lack of coordination by the project with the central department in charge of emergencies and disasters that has threatened the performance of this component. Participación comunitaria Hallazgos: There is very little community participation being supported by the project. Conclusiones: Without strengthening this component, humanization of care will continue to be a challenge. Lecciones Aprendidas Entrevista a Informantes Claves: DIRECCIÓN PROVINCIAL DE SALUD RESULTADOS Nombre del establecimiento: DPS Sanchez Ramirez, San Juan, El Seibo Entrevistador: Derek Sedlacek Fecha: March 11, 2012 Conocimientos generales acerca del proyecto: Hallazgos: All three of the DPSs have been intimately involved with the project since its induction phase. Conclusiones: All three of the DPS are committed to the project and advocate for implementation of its interventions in other provinces. Capacitación en gestión Hallazgos: The interventions in strategic planning, change management, and the postgraduate diploma in health and social security management were all identified as important contributors to the success of the project in the DPS. Each of the regions has been able to develop its own operational plans and was supported in the creation of quality improvement plans and strategic 182 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 plans. Furthermore, each of the DPS felt empowered enough to apply for the National Award for Public Administration. Conclusiones: This is perhaps the most sustainable process of the intervention, and is a critical step for any replication. Monitoreo, Evaluación, Sistemas de Información, Vigilancia y Epidemiologia Hallazgos: Monitoring and evaluation and information systems continue to be major weaknesses throughout the country. One of the main challenges identified is the pathway for information either directly to the province or through the regional health directorate. On the other hand there has been many advances in the area of epidemiology and surveillance. For example, El Seibo has been the number one province in terms of completeness and timeliness of information reported to DIGEPI. Expanding trainings beyond the medical interns to the staff that stay on for more time as well as giving refresher trainings when new interns arrive at the Unidades de Atención Primaria (UNAPS) have helped with improving reporting. Conclusiones: The pathway of monitoring and evaluation data from hospitals/UNAPS to the DPS needs to be systematized. The emphasis on mandatory reporting has been effective and is improving the performance of the DPS in the area of epidemiology and surveillance. Gestión de Recursos Hallazgos: The DPSs have limited financial resources and few human resources. The DPS in El Seibo has put forth a proposal to strengthen the capacity of its own resources while simultaneously expanding interventions to other DPSs. The DPSs did not recognize the financial resources management component as an area where they have been supported by the project. Conclusiones: The limited financial resources and human resources make it extremely challenging for a DPS to perform its functions effectively, making it even more important to optimize the management of resources. Redes programáticas de Salud Colectiva Hallazgos: The DPS and SRS are going through a transition with the PAI program and there is a surprisingly small number of people responsible for implementing a diverse set of programs. Conclusiones: The DPSs have limited capacity to perform this function. The persons assigned to particular programs are responsible for both promotion and stewardship of programs that require very different skill sets. Recomendaciones: The project should look at the organization of the DPSs to ensure that there is an appropriate match of skills with position responsibilities. Plan de Manejo de Emergencias y Desastres Hallazgos: All of the DPSs recognized this as an important component of the project. At some point there was an issue with the central level and implementation of this component essentially stopped. On the other hand, the DPS in El Seibo, which has a strong relationship with the central level, have been able to continue working on this issue. Conclusiones: There was a lack of coordination by the project with the central department in charge of emergencies and disasters that has threatened the performance of this component. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 183 Proceso de Habilitación Hallazgos: The process remains extremely centralized, although some DPSs (e.g., San Juan) seemed to have more autonomy than others (e.g., El Seibo), especially when it comes to recognizing the private sector and UNAPs. Conclusiones: Greater autonomy has resulted in more centers being recognized as rehabilitated. Replication Hallazgos: Due to the limited number and capacity of human resources, there might not be enough people to implement replications as originally designed. That said, El seibo has submitted a replication plan in order to begin this crucial next step. Conclusiones: All of the DPSs expressed the desire to replicate interventions in other DPSs, but El Seibo was the only one that has thought through how this process would happen. Recomendaciones: Rather than one DPS being responsible for replicating activities in another DPS, I would look to mixing and matching members from all three of the DPSs in order to optimize the effectiveness of replication and mitigate the burden placed on any individual DPS. Lecciones Aprendidas Entrevista a Informantes Claves: DIRECCIÓN REGIONAL DE SALUD RESULTADOS Nombre del establecimiento: Regions V, VI, VIII Entrevistador: Derek Sedlacek Fecha: March 9, 2012 Conocimientos generales acerca del proyecto: Hallazgos: Regions VI and VIII have been intimately involved with the project since its induction phase. Region V underwent a change in leadership during the course of the project and has not been as intimately involved in implementation of the project. Conclusiones: Regions VI and VIII are committed to seeing the interventions replicated in the hospitals throughout their region. Region V does not have the same commitment to the sustainability of these interventions. Selección de los Centros de Excelencia (CDEx) Hallazgos: Region VI and Region VIII agree fully with the selection criteria used by the project. The Region VI Director was planning to replicate the San Juan experience in Los Matas de Farfan, but when the replication team showed up to have the first meeting with the team from Los Matas, no one was there. In response, the Regional Director told the hospital director that there would be no effort made by the replication team until the receiving hospital showed the commitment necessary to complete the process. The Region V director was not present when the selection took place and has the perception that the hospitals were simply chosen. Conclusiones: Region VI and VIII understand that the competition that led to the selection of the hospitals has been critical. Commitment will be an important factor for choosing sites to replicate. 184 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Capacitación en gestión Hallazgos: The interventions in strategic planning, change management, and the postgraduate diploma in health and social security management were all identified as important contributors to the success of the project in the region and at the hospitals. Each of the regions has been able to develop its own operational plans and was supported in the creation of quality improvement plans and strategic plans. Conclusiones: This is perhaps the most sustainable process of the intervention, and is a critical step for any replication. Monitoreo, Evaluación y Sistemas de Información Hallazgos: Monitoring and evaluation and information systems continue to be major weaknesses throughout the country. While the regions and hospitals have the tools to complete the job, it is not being done completely or on time. Conclusiones: The project has not achieved a sustainable change in attitude about strategic information. Gestión de Recursos Hallazgos: This process has yet to be decentralized; therefore the achievements have been on a small scale. Region VIII has documented its success in managing the human resources paid by the sale of services and specifically requested our advocacy with the central level. Region VIII is implementing innovative income generation strategies such as selling lab services to private insurers. Region V identified more resources and more sources (donations) than the other regions, but was the most concerned about the sustainability of their current activities. Conclusiones: Without true decentralization it is impossible to know the capacity of these entities to manage the full extent of their resources. Gestión de cartera de servicios Articulación entre los diferentes niveles Hallazgos: While primary referral continues to improve, the counter-referral still remains a challenge. Conclusiones: The pilot being implemented in the Region V is still not working. Participación comunitaria Hallazgos: There is very little community participation being supported by the project. Conclusiones: Without strengthening this component, the humanization of care will continue to be a challenge. Replication Hallazgos: Region VIII identified financial resources as the main barrier to replication, while Region VI identified human resources as the primary barrier. Both are committed to replicating the interventions if/when resources are available. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 185 Conclusiones: Without true decentralization, it is going to be a challenge for the regional directors to fully commit to replication. Currently, the desire exists along with limited financial resources and capable human resources, which will result in replication of activities in only a limited number of other hospitals. 186 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 187 ANNEX 19. PRINCIPLES OF FAMILY-CENTERED MATERNITY CARE1 1 Information from: Goldy Mazia, MD, MPH. Technical Advisor for Newborn Health Maternal and Child Health Integrated Program (MCHIP). 1776 Massachusetts Ave., NW. Suite 300. Washington, DC 20036 FCMC Principle 1: Childbirth is seen as wellness, not illness. Care is directed to maintaining labor, birth, postpartum, and newborn care as a normal life event involving dynamics, emotional, social, and physical change. FCMC Principle 2: Prenatal care is personalized according to the individual psychosocial, educational, physical, spiritual, and cultural needs of each woman and her family. FCMC Principle3: A comprehensive program of perinatal education prepares families for active participation throughout the evolving process of preconception, pregnancy, childbirth, and parenting. FCMC Principle 4: The hospital teams assist the family in making informed choices for their care during pregnancy, labor, birth, postpartum, and newborn care, and strives to provide them with the experience they desire. FCMC Principle 5: The father and/or other supportive person(s) of the mother´s choice are actively involved in the education process, labor, birth, postpartum, and newborn care. FCMC Principle 6: Whenever the mother wishes, family and friends are encouraged to be present during the entire hospital stay, including labor and birth. FCMC Principle 7: Each woman’s labor and birth care are provided in the same location unless a cesarean birth is necessary. When possible, postpartum and newborn care are also given in the same location and by the same caregivers. FCMC Principle 8: Mothers are the preferred care providers for their infants. When mothers are caring for their babies, the nursing role changes from performing direct patient care to facilitating the provision of care by the mother or family. FCMC Principle 9: When mother–baby care is implemented, the same person cares for the mother and baby couplet as a single family unit, even when they are briefly separated. FCMC Principle 10: Parents have access to their high-risk newborns at all times and are included in the care of their infants to the extent possible given the newborn’s condition. 188 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 189 ANNEX 20. SUSTAINABILITY ELEMENTS OF FAMILY-CENTERED MATERNITY1 Period Interventions Elements of Sustainability and Implementable Comments Prenatal control Accompany Health education Adequate outpatient office Day hospital Preparatory course on maternity Respectful and cordial interaction Improved prenatal control Respectful and cordial interaction Accompany Health education Other suggested elements for the future (more resources): Prenatal checkups with emphasis on the prevention of maternal and neonatal risks (prematurity, pre-eclampsia, infection, nutritional deficiency) To establish the prenatal checkup for the first time, integration of complementary services and joint consultation Labor and delivery Accompany Skin-to-skin contact Exclusive and early breastfeeding No separation of mother and infant Everything but accompany, because even though it has a great impact, it implies changes on complex paradigms. It will also be useful to measure the skin-to-skin intervention over the next months More than the physical space for the patient’s recovery (2 hours after delivery), health staff attitudes are important: correct assessment of the mother and the newborn for early detection of signs of risk Rooming-in 10 steps for successful breastfeeding: 1. Written breastfeeding policy that is communicated to the entire staff 2. Train health staff in the necessary skills to apply the policy 3. Inform pregnant women of the benefits of breastfeeding 4. Help mothers to breastfeed in the first hour after delivery 5. Teach mothers to initiate and maintain breastfeeding It can be done with the support of the Breastfeeding Unit of the MOH and the La Leche League. MCHIP has worked with hospitals on the promotion of breastfeeding during the first hour (to be measured during the next few months), and Kangaroo Mother and HSVP are integrated with the breastfeeding unit There is need to change the attitudes of health staff and hospital directors to create favorable policies (including not allowing bottle-feeding commercial vendors). Create physical spaces at the hospital for women from remote areas 1 Information from: Goldy Mazia, MD, MPH. Technical Advisor for Newborn Health Maternal and Child Health Integrated Program (MCHIP). 1776 Massachusetts Ave., NW. Suite 300. Washington, DC 20036 190 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 Period Interventions Elements of Sustainability and Implementable Comments 6. Do not offer any food or liquids to the newborns other than breast milk 7. Rooming-in for the first 24 hours 8. Promote breastfeeding on demand 9. Do not offer pacifiers nor bottle 10. Facilitate the establishment of support groups and refer mothers to these groups during outpatient consultation Neonatal ward Unrestricted access for both parents Skin-to-skin contact Schedule visits from grandparents and siblings Support for parents in crisis Communication and management of parents with infants in risk of death Follow-up of high risk infants Possible elements in the short term: unrestricted access for both parents and skin-to-skin contact during visits (if possible). It is paramount to improve the quality of the newborn at risk, with emphasis on the examination of the retina and neurological signs of the premature infant. The Kangaroo Mother intervention has clear guidelines. The other elements are ideal but may be too complex to implement It is necessary to establish a follow-up visit for the high-risk newborn, which needs to be open and integrated on specific days USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 191 ANNEX 21. DETAILED METHODOLOGY THE EVALUATION TEAM A multidisciplinary team was formed from MOH, USAID/DR, USAID/Washington, and four independent and external consultants, totaling 8 team members. Dr. Olga Arroyo and Dr. Donatilo Santos represented the Department of Maternal and Child Health of the MOH; Mr. Derek Sedlacek, Ms. Veronica Valdivieso, and Dr. Peg Marshall represented USAID/W and USAID/DR; and Dr. Eddy Perez-Then, Ms. Marija Miric, Dr. Rafael Montero, and Dr. Marcelo Castrillo were independent consultants. All team members had experience in the country, with USAID’s Maternal and Infant health interventions, and in evaluation methodologies. The Centers of Excellence processes developed and implemented by Abt Associates are unique, so for most team members this was their first exposure to it. However, the two MOH MTE members had worked on some specific training activities and reviewing some of the case management protocols for the projects, though they were not familiar with the entire process until now. The MTE team was then divided into four evaluation groups in charge of: (1) interviews of staff from Provincial Health Directorates and Regional Health Services; (2) interviews of staff from Hospital Management Teams; (3) group discussions with users of maternal-infant health services and health service providers; and (4) rapid observation of the maternal and neonatal care setting, services delivered, and interviews with health care providers of these wards. PURPOSE AND THEORETICAL BACKGROUND OF THE EVALUATION The MTE was designed and implemented at a time when the MCH Centers of Excellence project transitioned from the implementation to the consolidation phase, and USAID/DR resources to MCH were declining. The MTE objectives were to determine what components and project aspects are working well or not well and the reason for their outcome, and the sustainability prospects of project interventions. It was also intended that the MTE results would help guide the MCH Centers of Excellence project to make modifications and corrections, if necessary. The MTE would provide information and recommendations to inform USAID, its Implementing Partners, the GODR, and other development partners on what is being accomplished regarding health systems strengthening and improving maternal, newborn, and infant health. Finally, the MTE would provide further information to better understand the initial results of the project and where needed, re-focus and strengthen it (see Annex 1: Mid term Evaluation Scope of Work). The five key questions the MTE aims to respond to are: 1. What programmatic and health systems strengthening aspects have been the most/least successful in ensuring the achievement of the project results and why? 2. What technical interventions have been the most/least successful in ensuring the achievement of the project results and why? 3. What programmatic and health systems strengthening aspects appear to be the most/least sustainable and why? 4. What technical interventions appear to be the most/least sustainable and why? 5. Has the project targeted the most critical and appropriate activities? what other opportunities exist? 192 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 In order to identify the most appropriate evaluation strategy and related factors for the specific category of analysis, each one of the three results of the Maternal-Child Centers of Excellence project was stratified into a category, and the categories into different indicators (see Annexes 4 and 5). Also, the evaluation of each objective of the MCHIP Program was performed, including prevention and treatment of newborn sepsis; Family-Centered Maternity and Kangaroo Mother Care strategies; strengthening of the newborn health national work plan in line with the LAC Neonatal Alliance Regional Strategy and Action Plan; and implementation of the "Helping Babies Breathe" (HBB) curriculum for newborn resuscitation. The evaluation of the two complementary projects for quality improvement of the maternal￾child services, the MCH Centers of Excellence and the MCH Integrated Program (MCHIP), was conducted following Rapid Assessment Process principles. This process included an extensive database search, using triangulation of information sources (observation, interviews, and literature review); iterative analysis (critical discussions with each team member about the observed findings regarding the parallel manner of the data recompilation process); and additional data collection as necessary to develop a rapid and preliminary understanding of project implementation status. The rapid assessment methodologies applied were: Document Review and Examination of Key Indicators Data. The MTE reviewed documents regarding the project design, implementation processes, and manuals, among others. In addition, once on site, the MTE evaluation team requested provincial directorates and health facilities project indicator data to verify how the health staff members use data for strategic planning and defining MCH priorities. Stakeholder Interviews at Different Levels – National/Policy Level. These were qualitative interviews to obtain information as to how key senior staff perceive the Centers of Excellence; how much information they receive regarding project implementation; whether it responds to the national priorities, and if so, obtain recommendations to ensure its continuation. – Regional Health Directorates. Qualitative interviews focused on the hospitals’ selection process; assistance on strategic planning; monitoring, evaluation, and use of data; systems strengthening and management of change (gestión de cambio); maternal￾child health committees; and the certification process. – Provincial Health Directorates. Qualitative interviews focused on the use of case management protocols; health information systems; and human resources training. – Selected Hospitals. Interviews focused on the two main components of the project; management strengthening and maternal-infant strengthening interventions. Direct Observation. All teams made direct observations of hospital settings, such as the Customer Service Office; delivery and neonatal rooms; and Kangaroo Mother units. Group Discussions were held with users of the services and hospital staff. USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 193 DATA COLLECTION INSTRUMENTS The team developed the following set of tools for data collection (Annex 6-13): 1. Key Informant Interview Guide: DIRECCIÓN REGIONAL DE SALUD 2. Key Informant Interview Guide: DIRECCIÓN PROVINCIAL DE SALUD 3. Key Informant Interview Guide: HOSPITALES REGIONALES Y PROVINCIALES 4. Informed Consent for ENTREVISTAS A USUARIAS DE SERVICIOS MI Y PERSONAL DE SALUD 5. Group Discussion guide: PROVEEDORES DE SERVICIOS DE SALUD 6. Group Discussion guide: USUARIAS DE SERVICIOS DE SALUD 7. Guide for qualitative observation 8. Guidelines for Observer Teams SAMPLING The MTE team visited six out of the 10 project hospitals. The criteria to select those six were that they have the highest patient flow, were accessible given the MTE time frame, and had been recommended by the implementing partners. The MTE team also interviewed staff from all three target regions and DPS. The MTE team carried out two group discussions per hospital catchment area, one in the morning and one in early afternoon. The following table shows the hospitals and directorates visited by the MTE team: March 5 March 6 March 7 March 8 March 9 H. San Lorenzo de Los Mina SRS Región VIII H. Morillo King DPS Cotuí H. Inmaculada Concepción H. San Vicente de Paul en San Francisco de Macorís SRS Región VI DPS San Juan H. Alejandro Cabral de San Juan SRS Región V San Pedro de Macorís H. Antonio Musa DPS de El Seybo In addition, during the first and third weeks, the MTE team visited the implementing partners and MOH project counterparts, namely, the MCH Department, the Vice-Ministry of Quality Care, the National Directorate of Epidemiology, the Vice-Ministry of REDES, and the Department of Habilitation of the DR MOH. 194 USAID/DOMINICAN REPUBLIC: EVALUATION OF USAID’S INTERMEDIATE RESULT 4 DATA ANALYSIS AND OUTCOMES INTERPRETATION To classify the evaluation outcomes of the two project sets of interventions, a qualitative score—based on two variables, performance and sustainability—was developed. Scales and definitions of the qualitative score are as follows: Performance Sustainability Low: Performance was consistently below expectations in most essential areas of responsibility. Not sustainable: The intervention has low potential for sustainability, no ownership by individuals and teams, and no financial support to maintain the level of effort is recommended. Needs im￾provement: Performance did not consistently meet expectations in one or more essential areas of responsibility Partially sustainable: Some level of sustainability has been achieved, but still needs external assistance to maintain the level of effort. High: Performance consistently met expectations in all essential areas of responsibility, at times possibly exceeding expectations, and the quality of work overall was very good. Fully sustainable: Individuals and teams have taken ownership of development processes, including financing, and can maintain the results and outcomes beyond the duration of the project, and conditional on the DR MOH level of compromise. Annexes 14 through 18 include the individual findings presented by the MTE team members, in their original language. ASSESSMENT LIMITATIONS There is one limitation in the approach the MTE team used to assess the USAID Intermediate Result 4 outcomes, and consequently the funding mechanisms. The evaluation of the project processes and effects was qualitative in nature. The MTE team gathered information from various sources and levels, and made subjective judgments in rating specific project interventions as relevant, sustainable, and replicable. However, the approach was systematic; a set of issues and facts were explored through predetermined discussion guides and observation lists (Annexes 6 through 13). When the responses of various informants and levels were combined, they usually supported one another, and, in cases where there were different or contradictory points of view, the MTE team kept on exploring in depth to determine if they were true differences or overall perceptions. After data collection, the MTE team met for three consecutive days to process the information obtained and to present the final conclusions and recommendations. The team brought additional perspectives, outside the implementing partners and stakeholders’ views about the strengths and limitations of the interventions. For more information, please visit http://www.ghtechproject.com/resources GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com