Baseline Report 1 The Impact of USAID’s Integrated Health Program in the Democratic Republic of the Congo: Baseline Report November 2020 2 The Impact of USAID’s Integrated Health Program in the DRC Data for Impact University of North Carolina at Chapel Hill 123 West Franklin Street, Suite 330 Chapel Hill, NC 27516 USA Phone: 919-445-9350 | Fax: 919-445-9353 D4I@unc.edu http://www.data4impactproject.org This publication was produced at the request of the United States Agency for International Development. It was prepared independently by David Hotchkiss, Janna Wisniewski, Paul-Samson Lusamba-Dikassa, Lauren Blum, Matt Worges, Charles Stoecker, and Eva Silvestre of Data for Impact. This publication was produced with the support of the United States Agency for International Development (USAID) under the terms of the Data for Impact (D4I) associate award 7200AA18LA00008, which is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, in partnership with Palladium International, LLC; ICF Macro, Inc.; John Snow, Inc.; and Tulane University. The views expressed in this publication do not necessarily reflect the views of USAID or the United States government. TRE-20-28 D4I ISBN 978-1-64232-246-0 The Impact of USAID’s Integrated Health Program in the Democratic Republic of the Congo: Baseline Report David Hotchkiss, PhD Janna Wisniewski, PhD Paul-Samson Lusamba-Dikassa, MD, PhD Lauren Blum, PhD Matt Worges, MS Charles Stoecker, PhD Eva Silvestre, PhD November 2020 Baseline Report 3 Abstract Data for Impact (D4I), which is funded by the United States Agency for International Development (USAID), is conducting a performance and impact evaluation of the Programme de Santé Intégré de l'USAID en République Démocratique du Congo (PROSANI USAID), known in English as USAID’s Integrated Health Program (IHP) in the Democratic Republic of the Congo (DRC). The evaluation investigates the following issues: 1. Did the expected changes in outcomes and impacts occur? 2. If there were changes in healthy behaviors during the study period, to what extent were these attributable to USAID IHP? 3. Did the project contribute to gender equity in health services and in the health system? 4. What factors enabled or limited the success of USAID IHP? We report baseline data from three sources: (1) surveys of health facilities and households in USAID IHP￾supported areas; (2) routine data from the DRC’s routine health information system (RHIS) for USAID IHP￾supported areas and a matched control group; and (3) qualitative data collected from representatives of the government health system, implementing staff, and beneficiaries. To assess baseline health behaviors, we analyzed data from the DRC’s health information system (which is built on the District Health Information Software, version 2 [DHIS2]) for both USAID IHP provinces and provinces not receiving USAID support. Results suggest that the government health system faces multiple challenges to effectively plan, implement, and monitor services, including: lack of electricity, cellular communication, and Internet; wide variations by province in the availability of medicines and equipment; limited managerial autonomy; variations in the levels of external supervision; insufficient number of staff; and low levels of health worker satisfaction. Although many of the health system challenges identified through the study are outside the control of the project (and are the purview of the government), they are nonetheless critical constraints. Other challenges can be partially addressed through project-supported interventions. Acknowledgments The authors wish to thank the United States Agency for International Development (USAID) for supporting this work. We acknowledge our partners at the Kinshasa School of Public Health who led the health facility survey: Professor Patrick Kayembe, Professor Eric Mafuta, and Marc Bosonkie; and the provincial supervisors: Benito Kazenza, Moza Kiakese, Louise Kpanda, Gabriel Kyomba, Zarine Lutonamo, and Nickson Sonabo. We also thank the medical anthropologists who collected qualitative data: Anicet Yemweni, Christelle Kasiama, Gerard Mbengo, Nadine Kamwang Kapend, and Fulgence Mwenze Mpanga. Finally, we thank the knowledge management team of the Data for Impact (D4I) project for editorial, design, and production services. Cover The lack of adequate roads and transportation means that young people must travel long distances in difficult conditions to sell their agricultural products. 2013 in Kasai Oriental Province in the Democratic Republic of the Congo. Photo: Alpha Luyoyo/USAID, courtesy of Flickr Creative Commons. Suggested citation Data for Impact. (2020). The Impact of USAID’s Integrated Health Program in the Democratic Republic of the Congo: Baseline Report. Chapel Hill, NC, USA: Data for Impact, University of North Carolina Baseline Report 5 Contents Abstract.......................................................................................................................................................... 3 Acknowledgments .......................................................................................................................................... 4 Contents......................................................................................................................................................... 5 Figures ........................................................................................................................................................... 7 Tables............................................................................................................................................................. 9 Abbreviations ............................................................................................................................................... 11 Executive Summary...................................................................................................................................... 14 Evaluation Purpose and Evaluation Questions.......................................................................................... 14 Project Background .................................................................................................................................. 14 Evaluation Questions, Design, Methods, and Limitations.......................................................................... 15 Baseline Findings and Conclusions........................................................................................................... 15 Evaluation Purpose and Questions................................................................................................................ 19 Evaluation Purpose .................................................................................................................................. 19 Evaluation Questions................................................................................................................................ 20 Purpose of the Report ............................................................................................................................... 21 Project Background....................................................................................................................................... 22 Objective 1: Strengthen Health Systems, Governance, and Leadership at Provincial, Health Zone, and Facility Levels in Target Health Zones...................................................................................................... 22 Objective 2: Increase Access to Quality, Integrated Health Services in Target Health Zones ....................... 23 Objective 3: Increase the Adoption of Healthy Behaviors, including the Use of Health Services in Target Health Zones............................................................................................................................................ 23 Evaluation Methods and Limitations............................................................................................................. 25 Methods ................................................................................................................................................... 25 Limitations............................................................................................................................................... 26 Findings ....................................................................................................................................................... 28 Health Systems Governance and Leadership............................................................................................. 28 Access to Quality, Integrated Health Services............................................................................................ 49 Health Behaviors ...................................................................................................................................... 94 Key Informants’ Perspectives.................................................................................................................... 99 Qualitative Baseline Report, Health Zone of Bunkeya ..............................................................................121 Qualitative Baseline Report, Health Zone of Kasaji ..................................................................................154 Summary and Implications ..........................................................................................................................176 Summary ................................................................................................................................................176 Programmatic Implications......................................................................................................................178 Limitations..............................................................................................................................................179 6 The Impact of USAID’s Integrated Health Program in the DRC References ...................................................................................................................................................179 Appendices..................................................................................................................................................180 Appendix 1. Evaluation Protocol.............................................................................................................180 Appendix 2. Data Collection Instruments ................................................................................................225 Appendix 3. Evaluation Team Members ..................................................................................................380 Appendix 4. Signed Disclosures of Conflicts of Interest ............................................................................382 Baseline Report 7 Figures Figure 1.1 Electricity in health zone offices, by province ................................................................................ 28 Figure 1.2. Mean number of hours of functional electricity on a typical eight-hour day in health zone offices with a source of electricity, by province ......................................................................................................... 29 Figure 1.3. Cellular telephone and Internet connections at health zone offices, by province............................. 30 Figure 1.4. Mean number of hours of functional Internet on a typical eight-hour day in health zone offices with an Internet connection, by province............................................................................................................... 30 Figure 1.5. Health zone offices’ Participatory Institutional Capacity Assessment and Learning Index participation and score, by province .............................................................................................................. 32 Figure 1.6. Percentage of health workers who are not head of the facility who report being internally supervised in the past week ............................................................................................................................................ 33 Figure 1.7. Percentage of health centers that report having internal management meetings, community co￾management meetings, and meetings with CODESA presidents monthly or more often in the past 12 months, by province................................................................................................................................................... 37 Figure 1.8. Percentage of hospitals that report having internal management meetings, community co￾management meetings, and meetings with CODESA presidents monthly or more often in the past 12 months, by province................................................................................................................................................... 38 Figure 1.9. Percentage of health zone offices that send a representative to Comités de Gestion provincial meetings, by province ................................................................................................................................... 39 Figure 1.10 Data displayed at health centers, by type and location (n=328) .................................................... 40 Figure 1.11. Data displayed at hospitals, by type and location (n=110)........................................................... 41 Figure 1.12. Percentage of health centers that hold a monthly meeting to discuss SNIS data, and that made decisions based on SNIS data, by province .................................................................................................... 41 Figure 1.13. Percentage of hospitals that hold a monthly meeting to discuss SNIS data, and that make decisions based on SNIS data, by province................................................................................................................... 42 Figure 1.14. Percentage of health workers who reported that they have ever received their full government salary on time, by province ..................................................................................................................................... 46 Figure 1.15. Percentage distribution of health center workers by whether they were part of a performance-based financing scheme, by province ...................................................................................................................... 48 Figure 1.16. Percentage distribution of hospital workers by whether they were part of a performance-based financing scheme, by province ...................................................................................................................... 48 Figure 2.1. Percentage of hospitals with X-ray, ultrasound, and autoclave equipment, by province (MOH minimum package of complementary services).............................................................................................. 51 Figure 2.2. Percentage of health centers with electricity, by province .............................................................. 51 Figure 2.3. Percentage of hospitals with electricity, by province...................................................................... 52 Figure 2.4. Percentage of health facilities with improved sanitation, by province............................................. 53 Figure 2.5. Percentage of health facilities with a private delivery room, by province........................................ 53 Figure 2.6. Percentage of health facilities offering a package of comprehensive sexual- and gender-based violence services, by province........................................................................................................................ 57 Figure 2.7. Percentage of health facilities offering a long-acting or permanent method of FP, by province ....... 58 Figure 2.8. Percentage of health facilities with at least one health worker trained in youth-friendly FP services, by province................................................................................................................................................... 58 Figure 2.9. Percentage of health facilities with FP information and resources specific to youth, by province .... 59 Figure 2.10. Maternal delivery outcomes, among those for whom an outcome was recorded, by health facility type .............................................................................................................................................................. 66 Figure 2.11. Percentage of women who died during childbirth who experienced complications, by facility type .................................................................................................................................................................... 67 Figure 2.12. Complications recorded for women who did not die during childbirth, by facility type ................ 67 8 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.13. Lengths of stay in health centers after normal delivery, by province............................................. 68 Figure 2.14. Lengths of stay in hospitals after normal delivery, by province .................................................... 69 Figure 2.15. Percentage of health workers who asked about aspects of medical history, by facility type ........... 70 Figure 2.16. Percentage of health workers who asked about aspects of social history, by facility type............... 71 Figure 2.17. Percentage of health workers indicating specific physical examinations, by facility type............... 72 Figure 2.18. Percentage of health workers who correctly identified bacterial dysentery/shigellosis as a likely diagnosis, by facility type and province.......................................................................................................... 73 Figure 2.19. Percentage of health center-based health workers who correctly mentioned metronidazole, fluids, or treatment for fever, by province (n=707).................................................................................................... 75 Figure 2.20. Percentage of hospital-based health workers who correctly mentioned metronidazole, fluids, or treatment for fever, by province (n=254)........................................................................................................ 75 Figure 2.21. Percentage of health workers who gave the correct diagnosis test (direct stool microscopy), and treatment (metronidazole) for bacterial dysentery/shigellosis, by facility type and province ............................ 76 Figure 2.22. Percentage of health workers who mentioned specific messages for counseling, by facility type.... 77 Figure 2.23. Percentage of health workers who asked questions related to health history, by facility type......... 78 Figure 2.24. Percentage of health workers who asked questions related to social history, by facility type ......... 79 Figure 2.25. Percentage of health workers who indicated that they would perform various physical examinations, by facility type........................................................................................................................ 80 Figure 2.26. Percentage of health workers who mentioned various diagnostic tests, by facility type ................. 81 Figure 2.27. Percentage of health workers who gave various recommendations, by facility type ...................... 83 Figure 2.28. Percentage of health workers who mentioned questions about aspects of medical history, by facility type .............................................................................................................................................................. 84 Figure 2.29. Percentage of health workers who mentioned questions about aspects of social history, by facility type .............................................................................................................................................................. 85 Figure 2.30. Percentage of health workers who would counsel the woman in choosing a contraceptive method, by facility type and province.......................................................................................................................... 86 Figure 2.31. Among health workers who indicated that they would not offer a contraceptive method, reasons given, by facility type .................................................................................................................................... 86 Figure 2.32. Among health workers who stated that they would offer a contraceptive method, counseling given, by facility type .............................................................................................................................................. 87 Figure 2.33. Among health workers who stated that they would offer a contraceptive method, criteria for determining the method to prescribe, by facility type ..................................................................................... 88 Figure 2.34. Among health workers who stated that they would offer a contraceptive method, response to the preferred method being unavailable, by facility type....................................................................................... 89 Figure 2.35. Among health workers who stated that they would offer a contraceptive method and their response to a request for confidentiality, by facility type............................................................................................... 89 Figure 2.36. Percentage of health centers that posted a standard fee schedule, had an indigent fee schedule, and posted an indigent fee schedule, by province.................................................................................................. 90 Figure 2.37. Percentage of hospitals that posted a standard fee schedule, had an indigent fee schedule, and posted an indigent fee schedule, by province.................................................................................................. 91 Figure 2.38. Percentage of health centers that accepted payment after treatment of emergencies and labor and delivery, by province..................................................................................................................................... 92 Figure 2.39. Percentage of hospitals that accepted payment after treatment of emergencies and labor and delivery, by province..................................................................................................................................... 92 Figure 3.1. Percentage of caregivers who sought care the last time their child under five had a fever, by province .................................................................................................................................................................... 97 Figure 3.2. Percentage of HAs in which at least one woman reported that she used RECOs for care for herself or her children in the past three months............................................................................................................. 98 Figure 3.3. Percentage of women who participated in a community scorecard meeting in the past 12 months . 99 Baseline Report 9 Tables Table 1.1. Degree to which health center directors report having autonomy in management (n=328).............. 31 Table 1.2. Degree to which hospital directors report having autonomy in management (n=110)...................... 31 Table 1.3. Status and role of respondents to health zone office survey, by province ......................................... 32 Table 1.4. Percentage of offices and health facilities that were visited in a supervisory capacity by a higher-level authority in the prescribed timeframe, by province......................................................................................... 33 Table 1.5. CODESA participation in health center management during the past 90 days, by province............. 34 Table 1.6. Percentage of health centers that received feedback from health zone offices in the past 90 days, by province ....................................................................................................................................................... 36 Table 1.7. Percentage of hospitals that received feedback from health zone offices in the past 90 days, by province ....................................................................................................................................................... 36 Table 1.8. Health zone offices’ oversight of health mutuelles, by province ...................................................... 36 Table 1.9. Frequency of health zone offices’ communication with CODESAs, by province............................. 38 Table 1.10. Frequency of health zone offices’ communication with other health zone offices, by province....... 39 Table 1.11. Frequency of provincial health offices’ communication and attendance at technical meetings, by province ....................................................................................................................................................... 40 Table 1.12. Timing of health zones’ reporting their most recent MAPEPI DHIS2 case, by province................ 42 Table 1.13. Barriers to a health zone submitting an immediate or weekly MAPEPI report on time, by province .................................................................................................................................................................... 43 Table 1.14. Timing of provinces’ reporting their most recent MAPEPI DHIS2 case........................................ 43 Table 1.15. Barriers to a provincial health office submitting an immediate or weekly MAPEPI report on time, by province ....................................................................................................................................................... 44 Table 1.16. Percentage of health workers who reported attending a training on iHRIS, by facility type and province ....................................................................................................................................................... 44 Table 1.17. Percentage of health center workers who reported being generally satisfied with their job and with specific dimensions of their job, by province .................................................................................................. 45 Table 1.18. Percentage of hospital workers who reported being generally satisfied with their job and with specific dimensions of their job, by province .................................................................................................. 45 Table 1.19. Status of health workers’ salary payment for the last completed month, by province ..................... 46 Table 1.20. Percentage of health workers with sources of supplemental income, by province........................... 47 Table 2.1. Percentage of health centers that offered select MOH minimum package of preventive services, by province ....................................................................................................................................................... 49 Table 2.2. Percentage of health centers that offered select MOH minimum package of curative services, by province ....................................................................................................................................................... 49 Table 2.3. Percentage of hospitals with capacity to conduct specific laboratory tests on the day of the survey, by province (MOH complementary package of activities)................................................................................... 50 Table 2.4. Percentage of health centers that had selected tracer drugs in stock on the day of the survey, by province ....................................................................................................................................................... 54 Table 2.5. Percentage of hospitals that had selected tracer drugs in stock on the day of the survey, by province 54 Table 2.6. Percentage of health centers with all basic equipment on the day of the survey, by province............ 55 Table 2.7. Percentage of hospitals with all basic equipment on the day of the survey, by province ................... 55 Table 2.8. Percentage of health centers with adequate infection control equipment, by province...................... 56 Table 2.9. Percentage of hospitals with adequate infection control equipment, by province ............................. 56 Table 2.10. Percentage of health centers meeting minimum standards for essential staff, supplies, and equipment to support the provision of long-acting or permanent methods of contraception, by province .......................... 59 Table 2.11. Percentage of hospitals meeting minimum standards for essential supplies and equipment to support the provision of long-acting or permanent methods of contraception, by province .......................................... 61 10 The Impact of USAID’s Integrated Health Program in the DRC Table 2.12. Percentage of health centers with adequate staffing numbers and mix according to government guidelines,* by province............................................................................................................................... 63 Table 2.13. Percentage of health center workers who reported agreeing with statements about their knowledge and attitudes, by province ............................................................................................................................. 64 Table 2.14 Percentage of hospital workers who reported agreeing with statements about their knowledge and attitudes, by province ................................................................................................................................... 65 Table 2.15. Total number of health workers who responded to the child health vignette, by facility type and province ....................................................................................................................................................... 69 Table 2.16. Percentage of health center-based health workers who ordered the correct tests for bacillary dysentery/shigellosis, by province ................................................................................................................. 73 Table 2.17. Percentage of hospital-based health workers who ordered the correct tests for bacillary dysentery/shigellosis, by province ................................................................................................................ 74 Table 2.18. Total number of health workers who responded to the ANC vignette, by facility type and province .................................................................................................................................................................... 77 Table 2.19. Percentage of health center-based health workers who gave various assessments, by province ....... 81 Table 2.20. Percentage of hospital-based health workers who gave various assessments, by province............... 82 Table 2.21. Percentage of health center-based health workers who recommended various treatment options, by province ....................................................................................................................................................... 82 Table 2.22. Percentage of hospital-based health workers who recommended various treatment options, by province ....................................................................................................................................................... 82 Table 2.23. Total number of health workers who responded to the FP vignette, by facility type and province .. 84 Table 2.24. Health centers’ responses to patients who could not pay fees owed, by province ........................... 93 Table 2.25. Hospitals’ responses to patients who could not pay fees owed, by province................................... 93 Table 3.1. Pooled estimates from multiply imputed data ................................................................................ 95 Table 3.2. Percentage of women who sought care in a public facility who reported long wait times, by service and province................................................................................................................................................. 96 Table 3.3. Percentage of women who sought care in a public facility who reported that the services listed were unaffordable, by service and province............................................................................................................ 96 Table 3.4. Percentage distribution of reasons why care was not sought for fever (n=number of cases of fever for whom care was not sought)........................................................................................................................... 97 Table 3.5. Percentage of HAs in which at least one woman reported accessing the following services at a community care site...................................................................................................................................... 98 Table 3.6. Percentage of women who reported that they participated in a public or consultative meeting organized by the administrative/political authority or civil society at various levels in the past 12 months....... 99 Table 4.1. Background information collected from in-depth interviews and key informants in Bunkeya..........121 Table 4.2.Facility infrastructure, supplies, and medications for child health services based on observations in Bunkeya ......................................................................................................................................................123 Table 4.3. Results from observations of health provider-client interactions.....................................................139 Table 4.4. Background information collected from in-depth interviews and key informants in Kasaji .............154 Table 4.5. Facility infrastructure, supplies, and medications related to child care based on observations in Kasaji ...................................................................................................................................................................156 Table 4.6. Results from observations of health provider-client interactions.....................................................170 Baseline Report 11 Abbreviations AC animateur communautaire [community animator] ACT artemisinin-based combination therapy AG administrateur gestionnaire [health zone financial and administrative manager] ANC antenatal care ARI acute respiratory infection BCC behavior change communication BCZS Bureau Central de la Zone [central zonal office] CAC Cellules d’Animation Communautaire [community animation cells] CDR Centre de Distribution Régional [regional distribution center] CHW community health worker CODESA Comité de développement de l’aire de santé [health area development committee] COGE comité de gestion [management committee] COR contracting officer’s representative CPN prenatal care CPON postnatal care CPP Comité Provincial de Pilotage [Provincial Steering Committee] CPS well-baby visits CPS consultation préscolaire [preschool consultation] CSO civil society organization D4I Data for Impact DFSA Development Food Security Activities DHIS2 District Health Information Software, version 2 DPS Direction Provincial de la Santé [provincial directorate of health] DRC Democratic Republic of the Congo EU European Union FC Congolese franc FFP Food for Peace FP family planning HA health area HC health center HCD human-centered design HSS health systems strengthening 12 The Impact of USAID’s Integrated Health Program in the DRC iCCM integrated community case management IGA Integrated Governance Activity IHP Integrated Health Program IP implementing partner(s) IPS Inspection Provincial de la Santé [provincial health inspector] IRA Infection Respiratoire Aigue IS Infirmier Superviseur [zonal nurse supervisor] IT infirmière titulaire [head nurse] ITA assistant infirmière titulaire [assistant head nurse] IT information technology LiST Lives Saved Tool M&E monitoring and evaluation MAPEPI maladies à potentiel épidémique [diseases with epidemic potential] MCH maternal and child heath MCZ Médecin Chef de Zone [health zone chief physician] MOH Ministry of Health MUAC mid-upper arm circumference NGO nongovernmental organization ORS oral rehydration solution PAO Plan d’action opérationnel [operational action plan] PARSS Projet d'Appui au Renforcement du Système de Santé [Health System Support and Strengthening Project] PBF performance-based financing PDSS Projet de Développement du Système de Santé [the DRC’s performance-based financing project] PESS Programme des équipements pour les structures sanitaires [Equipment program for health facilities] PEV Programme National de Nutrition [National Nutrition Program] PICAL Participatory Institutional Capacity Assessment and Learning Index PNDS Plan National de Développement Sanitaire [national health development plan] PROSANI Programme de Santé Intégré [Integrated Health Program] RECO relais communautaire [community relay] RHIS routine health information system SBC social and behavior change SNIS Système national d'information sanitaire [national health information system] Baseline Report 13 SRO Solution orale de réhydratation TB tuberculosis TIP Trafficking in Persons UNICEF United Nations Children’s Fund USAID United States Agency for International Development USD United States Dollar VAD Visite à domicile VPO Vaccin antipoliomyélitique oral VSAT very small aperture terminal WASH water, sanitation, and hygiene WHO World Health Organization 14 The Impact of USAID’s Integrated Health Program in the DRC Executive Summary Evaluation Purpose and Evaluation Questions Data for Impact (D4I), which is funded by the United States Agency for International Development (USAID), is conducting a comprehensive evaluation of the Programme de Santé Intégré de l'USAID en République Démocratique du Congo (PROSANI USAID), known in English as USAID’s Integrated Health Program (USAID IHP) in the Democratic Republic of the Congo (DRC). The evaluation is investigating the following issues: • The progress of USAID IHP in achieving Objectives 1, 2, and 3: strengthened health systems, governance, and leadership at the provincial, health zone, and facility levels; increased access to quality, integrated services; and increased adoption of behaviors that improve health outcomes (referred to as “healthy behaviors”). • The impact of USAID IHP on several proxy indicators of healthy behaviors, including treatment for childhood illnesses, maternal healthcare use, and contraceptive method use. • The factors that enabled or limited the success of USAID IHP, including the design and scope of the project, implementation factors, and external and contextual factors. This evaluation helps fill previous knowledge gaps by supplying critical information about the impact of USAID IHP, a large, complex health systems strengthening (HSS) project. It contributes to the country’s evidence base of successful HSS activities, and those that are not successful. Experience from this evaluation can be used to help design future projects intended to deliver high-quality healthcare and family planning (FP) services that meet the needs of the Congolese people. This report documents the functioning of the health system and select health outcomes immediately before the implementation of USAID IHP-supported activities in the targeted provinces. We report baseline data from three sources: (1) surveys of health facilities and households in USAID IHP-supported areas; (2) routine data from the DRC’s RHIS for USAID IHP-supported areas and a matched control group; and (3) qualitative data collected from representatives from the government health system, implementing staff, and beneficiaries. The key audiences for the evaluation include officials and relevant personnel of the DRC’s Ministry of Health (MOH), USAID, and other international health organizations and agencies. We expect that the results of this evaluation will inform the design and implementation of future HSS activities in the DRC. Project Background As part of its strategy to improve health outcomes in the DRC, USAID funded the USAID IHP in 2018. The program began operations in July 2018 and is being implemented by Abt Associates and several partner organizations. The purpose of USAID IHP is to strengthen the capacity of Congolese institutions and communities to deliver quality, integrated health services to sustainably improve the health status of the Congolese population. The project focuses on the following specific health, population, and nutrition areas: maternal health; neonatal, infant, and child health; tuberculosis (TB); malaria; child nutrition; water, sanitation, and hygiene (WASH); and FP. USAID IHP seeks to reach its goal through the achievement of the following overall performance objectives: • Strengthen health systems, governance, and leadership at the provincial, health zone, and facility levels in target health zones (Objective 1). • Increase access to quality, integrated health services in target health zones (Objective 2). Baseline Report 15 • Increase the adoption of healthy behaviors, including the use of health services, in target health zones (Objective 3). USAID IHP will work in nine contextually diverse provinces in the regions of Eastern Congo, Katanga, and Kasai, and will include a wide array of interventions. Given the breadth and depth of the planned interventions, the USAID/DRC Mission requested that D4I conduct an independent third-party evaluation of the performance and impact of USAID IHP on key health systems-related outcomes: the uptake of FP and healthcare services; health systems functioning (i.e., improved disease surveillance, the availability of essential commodities, and health worker motivation); and the practice of key healthy behaviors. Evaluation Questions, Design, Methods, and Limitations The evaluation questions for the overall study, along with the methods and sources of data, are presented below. Evaluation Question Method Source of Data Did the expected changes in outcomes and impacts occur? Quantitative analysis of changes over time in USAID IHP health zones Baseline, midline, and endline surveys of provincial health offices, health zone offices, and health facilities. Baseline, midline, and endline surveys of households. RHIS data (DHIS2) If there were changes in healthy behaviors over the course of the study period, to what extent were these attributable to IHP? Doubly robust model that combines difference-in-differences with propensity score matching RHIS data (DHIS2) Did the project contribute to gender equity in health services and in the health system? Quantitative analysis of changes over time in USAID IHP health zones Baseline, midline, and endline surveys of provincial health offices, health zone offices, and health facilities Baseline, midline, and endline surveys of households What factors enabled or limited the success of IHP? Qualitative methods Baseline, midline, and endline qualitative data collection from key informant interviews, in-depth interviews, and focus group discussions This report includes a discussion of the limitations of the evaluation and threats to carrying out the evaluation as planned. Baseline Findings and Conclusions Summary of Empirical Results To assess baseline levels of health governance and leadership, service readiness, and service quality, we administered surveys of provincial health offices (n=6), health zone offices (n=113), health centers (n=328), 16 The Impact of USAID’s Integrated Health Program in the DRC hospitals (n=110), and health workers (n=1,213). The results suggest that the government-run health system faces myriad challenges to effectively plan, implement, and monitor services. These challenges include problems with electricity, cellular communication, and Internet connectivity; wide variations by province in the availability of medicines and equipment; limited managerial autonomy with respect to resource generation and budgeting practices; wide variations in the levels of external supervision of hospitals and health centers (HCs); insufficient number of staff at health facilities; and low levels of health worker satisfaction. The specific results for these and other types of indicators are presented in the report. The results are disaggregated by province. We assessed health workers’ level of essential knowledge to provide high-quality healthcare services (as a factor enabling USAID IHP’s success) by presenting a sample of health workers with three vignettes describing hypothetical cases of diarrhea, antenatal care (ANC), and FP. We then asked a series of questions about diagnosis, testing, and treatment. In response to the vignette on diarrhea, about half of the hospital health workers and 58 percent of the HC workers were unable to correctly diagnose the problem, and there were substantial numbers of personnel who did not prescribe standard tests and recommended treatments. Responses to the vignettes for ANC and FP also suggest that a significant proportion of health workers had problems identifying key elements of the standards of care. To assess baseline health behaviors, we analyzed District Health Information Software, version 2 (DHIS2) data for both USAID IHP provinces (the “treatment” group) and provinces not receiving USAID support (the “control” group). Randomization of USAID IHP interventions for HSS between control and treatment sites was not an option, owing to budgetary, operational, and ethical factors. We used propensity score weighting to select an appropriate pool of control sites against which to measure changes in the treatment group—a process that helps control for selection bias. In effect, the propensity weights adjust the control group so that its pretreatment covariate distributions match those of the treatment group. We used a common pool of DHIS2 data elements (e.g., live births, malnutrition, breastfeeding, ANC, vaccination, modern contraceptive method use, insecticide-treated net distribution, and case management of malaria, pneumonia, and diarrhea) for the propensity score weighting procedure, which produced a group of control sites that did not show an appreciable level of bias compared with the treatment sites. We also used data from Abt Associate’s baseline household survey to characterize the care seeking experience and level of participation in health services among women of reproductive age. Most (82.2%) children under five with a fever received care. Among the caregivers who reported that they did not seek care for a child with a fever, the reason most frequently cited was lack of money. Overall, more than one-half (56.7%) of the health areas (HAs) appeared to have functioning relais communautaires (community health workers). Rates of civic participation among women were low. Summary of Key Informant Results We interviewed 20 key informants at central and provincial levels. Key informants were MOH representatives (7), USAID IHP senior staff (6), USAID staff overseeing USAID IHP (3), and collaborating partners (4). The key informants had a wealth of experience in health service delivery program development and implementation, and HSS. Informants consistently described the national health system strategy as well￾conceived and organized, and stated that it encompassed key elements. However, there was universal agreement that many facets of the health system were not being implemented as planned. The primary obstacles mentioned related to underfinancing, mismanagement of resources, and poor governance, which informants reported contributed to the low use and low quality of health services. USAID IHP uses an integrated approach to implement six technical programs—malaria, maternal and child health (MCH), FP, nutrition, TB, and WASH, with a focus on HSS. Malaria programs will be executed across all 178 target health zones. Implementation of the other programs will vary according to the availability of resources, local needs, and ongoing activities of implementing partners (IPs). Specifically, USAID IHP will Baseline Report 17 partner with other organizations to attempt to piece together a cohesive approach that covers all six technical programs in the nine target provinces. There are also cross-cutting approaches (such as strengthening health systems, providing medications, improving information systems, developing finance systems, engaging in community development, accommodating youth, and provisions regarding gender) with separate funding allocated for each approach. The project will concentrate efforts on geographical areas to increase synergy with other USAID programs and improve overall project impact, which may cause inequities in the quality and consistency of health service delivery. USAID IHP is funded through a contractual agreement between USAID and Abt Associates, giving USAID ultimate decision-making power and technical oversight regarding project development and implementation, and the use of funds. Abt Associates is responsible for executing activities and ensuring that indicators are followed and results are achieved. Some informants raised concerns that the restrictive nature of the contract and USAID IHP’s approach will limit the ability of government personnel to lead project development and implementation and to take ownership of HSS. Informants described USAID IHP as a complex project comprising many interventions and actors, with several informants reporting that the project is overly ambitious in relation to the myriad challenges plaguing the DRC’s health system. Commonly cited problems include poor health information systems affecting data quality and reporting, irregular provision of quality medications, low motivation of health workers linked to poor remuneration and work conditions, and rampant corruption and mismanagement of funds that infiltrate all levels of the health sector. Summary of Qualitative Field Results We carried out an in-depth, qualitative investigation in two health zones in the Lualaba province. One health zone had received extensive support from the USAID IHP predecessor project, whereas the second health zone had received support for specific activities, such as integrated community case management (iCCM). In each health zone, we administered a mix of qualitative research methods, including key informant and in￾depth interviews, observations, and focus group discussions. Data collection involved in-depth interviews with HC nurses (4), members of the health development committee (4), community health workers (4), village chiefs or village chief representatives (4), reference hospital physicians (2), hospital or Bureau Central de la Zone de Santé (BZCS [central health zone office]); BZCS managers (2), and an acting health zone medical officer; key informant interviews with the Médecin Chef de Zone (MCZ [health zone chief physician]) (2); observations of health infrastructures (6) and health worker-client interactions (40); and focus group discussions with caregivers of children under five years of age (4). Findings show that MCZs are frequently physically absent from their posts. Staffing of health personnel does not meet government standards, with HCs often using untrained workers to provide medical care. HCs are often small, and facilities are frequently poorly constructed, in disrepair, and lack key equipment and supplies. Stockouts of essential medications are common, forcing health workers to restock often with unregulated drugs or to give patients prescriptions to purchase drugs from local pharmacies. There is a growing number of private pharmacies selling unregulated drugs, which allows people to self-treat before seeking professional care. Community members often seek treatment from traditional healers, who provide remedies that can interfere with medical care and cause harmful health effects. Health workers rely primarily on monthly facility revenue for compensation, although performance-based financing activities are providing important bonuses to facilities that meet standards. Despite low and irregular remuneration, health providers expressed gratitude for being employed. Informants described health workers as respectful of and courteous toward clients, which was confirmed during health worker-client observations. Observations indicated that health workers fail to follow components of treatment protocols during patient consultations, which often take place in unclean, noisy environments, where privacy is not maintained. 18 The Impact of USAID’s Integrated Health Program in the DRC According to the people interviewed, no community-based organizations involved in health existed in the zones studied. Community outreach activities are not functioning according to the national strategy. Community health workers (CHWs) require training, supervision, and supplies to their fulfill roles. The lack of oversight and supplies do not allow iCCM sites to operate as planned. Poor motivation causes high attrition of CHWs. CHWs are predominantly male, with men controlling positions of leadership. Programmatic Implications Although many of the health system challenges identified through the study—such as limited access to electricity and phone connectivity for health workers and problems with health worker remuneration—are outside the control of the project and in the hands of the government, they are critical constraints that must be considered when designing and implementing USAID IHP-supported interventions. Obvious examples are the lack of electrical power and limited phone and Internet connectivity and Internet credit, which can affect efforts to improve service readiness and data collection and reporting for the Système national d'information sanitaire (SNIS [national health data collection and reporting system]). However, other challenges identified can be partially addressed through project-supported strategies and interventions. The following are recommendations based on the study results: • The design of both facility- and community-based service delivery interventions should be based on formative research on community perceptions and needs. This recommendation reflects the qualitative research findings that highlight the importance of geographical and cultural differences in the DRC, and the need to design program approaches to better coincide with contextual conditions. More experimental interventions should also be tested to assess the role of cultural context on their effectiveness. • The program should explore alternative approaches to improve the remuneration of health workers— possibly through coordination with the MOH’s Projet de Développement du Système de Santé (PDSS [the DRC’s performance-based financing project])—to improve health worker motivation and, in turn, service quality and availability. This recommendation is based on both quantitative and qualitative research findings, suggesting that a large percentage of health workers do not receive a salary. We found that this affects health workers’ motivation and leads to a heavy reliance on income received from household out-of-pocket payments. • The program should place more emphasis on supervision of local health officers and health workers to improve (a) service quality and (b) the collection, availability, and use of routine data, beyond the data that are available in the DHIS2. This recommendation is based on survey findings indicating that external supervision is not carried out as frequently as MOH guidelines stipulate, and a finding that providers need support in adhering to provider practice guidelines. • More emphasis should be placed on continuous education programs to improve adherence to provider practice guidelines, which can improve adherence to standards of care. Again, this is based on the finding from the clinical vignettes, indicating that providers often did not have the knowledge to correctly diagnose and treat certain health problems. • Stockouts of essential medications plague health facilities, undermining health services and their use, and jeopardizing the revenue needed for health facilities to function. Stockouts force health workers to use unregulated drugs and provide prescriptions to sick patients. To ensure the provision of regular and quality care, efforts are needed to guarantee that medications meet drug orders and are delivered in a timely and routine fashion. • Community health activities are not functioning as described in the national community health strategy. CHWs require ongoing training, supervision, and materials to successfully carry out activities, Baseline Report 19 including their roles in the execution of iCCM activities. Community activities would benefit from oversight by and the support of community-based organizations and development committees. Mechanisms to motivate and retain CHWs should be explored. Evaluation Purpose and Questions Evaluation Purpose Despite the need for a more responsive health system, the evidence base for improved policy decision making in the Democratic Republic of the Congo (DRC) is weak, due to limited research capacity, lack of transportation and communications infrastructure, security challenges, and limited funding for health systems research. Beyond the DRC, there is a general paucity of information about low-income countries, especially fragile states, regarding the efficacy of interventions to address system-level gaps and capacity shortcomings. In a 2012 systematic literature review of evaluations of health system strengthening (HSS) programs, few evaluations were found to be comprehensive across multiple health system building blocks, and few included evaluation designs that considered the complex nature of the programs (Adam, Hsu, and de Savigny et al., 2012). This evaluation helps fill the previous knowledge gaps by supplying critical information about the impact of the Programme de Santé Intégré de l'USAID en République Démocratique du Congo (PROSANI USAID), known in English as USAID IHP, a large, complex HSS project. By contributing to the country’s evidence base of successful and unsuccessful HSS activities, this evaluation provides lessons to help design future projects intended to deliver high-quality healthcare and family planning (FP) services that meet the needs of the Congolese people. D4I designed this evaluation to investigate the following: • The progress of USAID IHP over time in achieving Objectives 1, 2, and 3, as follows: strengthened health systems, governance, and leadership at the provincial, health zone, and facility levels; increased access to quality, integrated services; and increased adoption of healthy behaviors. • The impact of USAID IHP on several proxy indicators of healthy behaviors, including treatment for childhood illnesses, maternal healthcare use, and contraceptive method use. • The factors that enabled or limited the success of USAID IHP, including the design and scope of the project, implementation factors, and external and contextual factors. Achieving these evaluation objectives requires both a performance evaluation and an impact evaluation. The performance evaluation incorporates several study components based on data from USAID IHP provinces only, including District Health Information Software, version 2 (DHIS2) data; population-based household survey data; health facility, health zone office, and provincial health office survey data; and qualitative data. This component of the evaluation investigates whether USAID IHP achieved targets for all three of its performance objectives—strengthened health systems, governance, and leadership; increased access to high￾quality, integrated services; and increased adoption of healthy behaviors. To the extent possible, we will triangulate empirical data with qualitative data to explore whether changes over time in the adoption of healthy behaviors are associated with changes in strengthened health systems and increased access to high￾quality, integrated services, and to determine why expected changes are or are not observed. Last, to explore the health impacts of the changes in intervention coverage, we used the Lives Saved Tool (LiST) to estimate how changes in the adoption of healthy behaviors translate to changes in the number of lives saved. The LiST 20 The Impact of USAID’s Integrated Health Program in the DRC analysis will be conducted at midline and endline; therefore, no methods or results related to that analysis are presented in this report. The impact evaluation component of the study compares DHIS2 data of several proxy indicators of healthy behaviors in project areas and non-project areas using a difference-in-differences with propensity score matching approach. Because it was not possible to administer the population-based household surveys and the health facility, health zone office, and provincial health office surveys outside of the nine USAID IHP provinces areas, it is not feasible to assess the impact of USAID IHP on population-based service coverage rates, or on indicators of strengthened health systems, leadership and governance, and increased access to high-quality, integrated services. The key audiences for this evaluation include officials and relevant personnel of the Ministry of Health (MOH), USAID, and other international health organizations and agencies. We expect that the results of the evaluation will be used to inform the design and implementation of future HSS activities in the DRC. Evaluation Questions 1. Did the expected changes in outcomes and impacts occur? a. Strengthen health systems, governance, and leadership at provincial, health zone, and facility levels in target health zones i. Outcome: Enhanced capacity (institutional and individual) of provincial health offices, health zones, and facilities to plan, implement, monitor, oversee, and supervise services ii. Outcome: Strengthened capacity of civil society organizations (CSOs) and community structures to provide health systems oversight b. Increase access to high-quality, integrated health services in target health zones i. Outcome: Increased availability of high-quality, integrated facility-based health services and commodities ii. Outcome: Increased availability of high-quality, integrated community-based health services iii. Outcome: Improved affordability of integrated health services c. Increase adoption of healthy behaviors, including health service use, in target health zones i. Impact: Proportion of children under five for whom treatment or advice was sought for acute respiratory infection (ARI), diarrhea, and fever ii. Impact: Proportion of children under five who slept under an insecticide-treated net iii. Impact: Proportion of married women using any modern method of contraception iv. Impact: Proportion of children who received all eight basic vaccinations v. Impact: Proportion of pregnant women who attended four antenatal care (ANC) visits 2. If there were changes in healthy behaviors over the course of the study period, to what extent were these attributable to USAID IHP? 3. Did the project contribute to gender equity in health services and in the health system? Baseline Report 21 4. What factors enabled or limited the success of USAID IHP? a. Design and scope b. Implementation and management c. External environment and contextual factors d. Government decentralization Purpose of the Report This report documents the functioning of the health system and select health outcomes immediately before the implementation of USAID IHP-supported activities in the targeted provinces. We report baseline data from three sources: surveys of health facilities and households in USAID IHP-supported areas; routine data from the DRC’s routine health information system (RHIS) for USAID IHP-supported areas and a matched control group; and qualitative data collected from representatives from the government health system, implementing staff, and beneficiaries. 22 The Impact of USAID’s Integrated Health Program in the DRC Project Background As part of its strategy to improve health outcomes in the DRC, USAID funded the USAID IHP program in 2018. The program began operations in July 2018 and is being implemented by Abt Associates and several partner organizations. The purpose of USAID IHP is to strengthen the capacity of Congolese institutions and communities to deliver high-quality, integrated health services to sustainably improve the health status of the Congolese population. The specific health, population, and nutrition areas of focus for the project are maternal health; neonatal, infant, and child health; tuberculosis (TB); malaria; child nutrition; water, sanitation, and hygiene (WASH), and FP. USAID IHP seeks to reach its goal through the achievement of the following overall performance objectives: • Strengthen health systems, governance, and leadership at the provincial, health zone, and facility levels in target health zones (Objective 1). • Increase access to high-quality, integrated health services in target health zones (Objective 2). • Increase the adoption of healthy behaviors, including the use of health services, in target health zones (Objective 3). USAID IHP will work in nine contextually diverse provinces in the regions of Eastern Congo, Katanga, and Kasai, and will include a wide array of interventions. Given the breadth and depth of the planned interventions, the USAID/DRC Mission requested Data for Impact (D4I) to conduct an independent, third-party evaluation of the performance and impact of USAID IHP on key health systems-related outcomes, including the uptake of FP and healthcare services; health systems functioning (i.e., improved disease surveillance, the availability of essential commodities, and health worker motivation); and the practice of key healthy behaviors. The project team works closely with government health officials at the central, provincial, zonal, and health facility levels to build government capacity and leadership, and to increase the sustainability and local ownership of the interventions. The USAID IHP’s components will address three program objectives, as follows. Objective 1: Strengthen Health Systems, Governance, and Leadership at Provincial, Health Zone, and Facility Levels in Target Health Zones The programmatic approaches related to Objective 1 aim to support provinces, health zones, and communities to become empowered stewards and effective managers of health system functions, via tailored needs-based interventions guided by results of Participatory Institutional Capacity Assessment and Learning Index (PICAL) evaluations and human-centered design (HCD) techniques. The PICAL tool will be applied at provincial and health zone levels to foster a culture of self-assessment, enhance institutional capacity building, and guide the development and implementation of performance improvement action plans to support improved governance, leadership, and accountability. The capacity￾building needs identified during PICAL assessments will also be used to facilitate targeted technical assistance, coaching, and leadership training in (1) public financial management; (2) analysis and use of data for improved disease surveillance and facility-level data reporting; (3) management of human resources for health, taking gender into consideration in the recruitment and deployment of staff; and (4) use of the performance dashboard tool to equip provincial and health zone managers with real-time, data-driven, decision-making capabilities. Moreover, USAID IHP will: optimize the use of existing methods, such as results-based financing; employ mobile phone-based surveillance technologies; and strengthen supply chain activities to support quantification, forecasting, and timely inventory replenishment. Baseline Report 23 At the community level, USAID IHP will use the recently developed MOH community dynamics strategy to improve stakeholder coordination and oversight functions. By facilitating collaboration among provincial, health zone, and community stakeholders, this strategy aims to strengthen the capacity of Comités de Développement de l’Aire de Santé (CODESAs [health area development committees]), CSOs, and community￾based organizations to be true partners in addressing social and behavior change (SBC), and mobilizing the demand for and uptake of improved health services. Activities to support community-level monitoring of health system performance will include streamlining community scorecard approaches; launching a toll-free fraud and complaints hotline number for reporting corruption, abuse, or similar allegations; and providing rights-based education to communities. Capacity-building of CODESAs, select CSOs, or community-based organizations will also take place through a Grant under Contract program. Together, this enhanced coordination capacity and multi-level collaboration will support more effective community stewardship of the health system, while demanding accountability of both local and provincial authorities. Objective 2: Increase Access to Quality, Integrated Health Services in Target Health Zones The programmatic approaches related to Objective 2 focus on increasing health service demand, access, and quality in the program regions. A primary component entails scaling up health facilities that can provide essential, integrated, and high-quality health services. Facility-based activities include renovating health infrastructures; equipping health facilities with drugs and medical supplies; and building knowledge and capacity among health workers so that health personnel can provide a package of integrated services for maternal, neonatal, and child health; nutrition; FP and reproductive health; WASH; malaria; and TB. The interventions will also focus on improving health worker attitudes and interpersonal communications. As part of this approach, the project will implement a fraud and complaints hotline and reporting system to enhance health worker accountability. Using a cluster model strategy, the project will first prioritize building capacity in a high-performing facility in a health zone, and once strengthened, use that health structure to provide support and outreach to facilities in the same health zone. The project aims to strengthen other facilities located in more remote locations over the course of the project. Community-based health activities are considered critical to increasing the use of facility services and improving the provision of essential health services, especially in remote localities. Interventions designed to strengthen community-based health services will include recruitment of new community health workers (CHWs), especially women; training CHWs on health promotion (with a focus on WASH) and integrated community case management (iCCM); and training facility-based health workers on community outreach and the provision of health services at the community level. Community activities will be scaled up over time, with an initial focus on remote communities with access to supported health facilities. Interventions will also involve strengthening referrals from community platforms and HCs to referral hospitals. A general emphasis will involve building collaboration with government health structures, the United States Government, and other donors by supporting and actively participating in central-level meetings during which learning experiences, needs, and priorities can be jointly identified and discussed, and policy influenced. Objective 3: Increase the Adoption of Healthy Behaviors, including the Use of Health Services in Target Health Zones The interventions related to Objective 3 are meant to increase the adoption of healthy behaviors and use of health services in the targeted provinces. The strategy aims to raise community awareness and knowledge of healthcare services and address barriers to optimal healthcare-seeking, and to strengthen community engagement and social support to enable healthy behaviors. Specific interventions will include a “healthy family” campaign composed of a multipronged educational program involving a family drama series focusing on common health problems and issues related to accessing facility and community-based health services, the care received, and satisfaction derived. Storylines disseminated through radio and text messaging will highlight 24 The Impact of USAID’s Integrated Health Program in the DRC sociocultural barriers that inhibit access to services and the practice of healthy behaviors, and ways these barriers can be overcome. Radio listening sessions will be organized to facilitate community discussions and reactions to scenarios presented during the drama series at the local level. The messages conveyed through the drama series will be complemented by interpersonal communication carried out by CHWs and CODESAs, and supported by women’s organizations and other community-based groups through mobilization events. Open houses will be held to showcase improvements in health facilities and encourage use. The Champion Community model will be implemented to prioritize HAs and target audiences, and develop workplans and monitor activities in the targeted areas. Mini campaigns focused on addressing health problems will also be carried out according to specific and immediate needs. Efforts will be made to share lessons learned, harmonize strategies, and improve approaches by collaborating and coordinating with other groups involved in SBC, including the following: key government institutions working on communications; government officials, implementing agencies, and other stakeholders participating in coordination meetings (clusters, Médecin Chef de Zone [MCZ], head nurse) at the central, provincial, and zonal levels; and USAID staff and partners. The project aims to share SBC activity results with international audiences during academic conferences and through peer-reviewed, scientific manuscripts. At the more local level, coordination of SBC approaches will be carried out in conjunction with the health zone office, CODESAs, and Cellules d’Animation Communautaire (CAC [community-level organizations that engage in health communication]), with assistance provided to health zones during the development of their operational action plans to ensure the overall goal of scalability of sound and effective messaging and activities that align with and contribute to the achievement of agreed on health goals. The project started in July 2018 and will be implemented over a four-year period, with the possibility of a three-year extension. The project is led by Abt Associates, with the International Rescue Committee and Pathfinder International as core partners. Seven niche partners with expertise in health programming, designing innovative approaches, and research in fragile states—including DRC—are participating. Baseline Report 25 Evaluation Methods and Limitations Methods D4I will carry out two types of evaluation components for this study: a performance evaluation and an impact evaluation. As defined by USAID Evaluation Policy, performance evaluations incorporate before and after comparisons, but generally lack a rigorously defined counterfactual to control for factors other than the project or intervention that might account for the observed change. Impact evaluations assess the extent to which changes in health outcomes or service use over time are attributable to an intervention. The performance evaluation aspect of the study will address: Research Question 1, which investigates changes over time in USAID IHP areas; Research Question 3, which examines the extent to which the project addressed issues of gender equity; and Research Question 4, which investigates factors that enabled or limited the success of the project. Data for this component of the study will come from multiple sources, including: the DRC’s RHIS (DHIS2); household surveys; surveys of healthcare facilities, health zone offices, and provincial health offices; and key informant and in-depth interviews, observations of patient-health worker interactions, and focus group discussions. The impact evaluation aspect of the study will address Research Question 2, which investigates the extent to which changes in healthy behaviors are attributable to USAID IHP. The impact of USAID IHP will be based on a difference-in-differences with propensity score matching model, a nonexperimental design, based on data from the DHIS2. Ethical approval for this work was given by the Institutional Review Boards of Tulane University and the Kinshasa School of Public Health. Analysis of Impact Using a Difference-in-Differences Model Model description: A doubly robust model that combines difference-in-differences with propensity score matching will be used to estimate the impact1 of USAID IHP on the provision of maternal and child healthcare services and FP services (as identified in Research Question 2). The unit of analysis will be at the facility level. The treatment arm includes facilities in health zones in USAID IHP provinces (based on data from both hospitals and HCs) in the pre-intervention (January 2018 to June 2019) and post-intervention (July 2019 to 2023) periods. The control arm includes facilities in comparable health zones without USAID IHP support. The data used for this analysis come from the health management information system. Analysis of Changes in Project Areas Facility-, health zone office-, and provincial health office-level changes. To evaluate the progress of USAID IHP-supported areas on service readiness, service quality, and service use, a separate analysis component will use data from provincial health offices, health zone offices, and health facilities (HCs and hospitals), which we will collect, plus data from baseline, midline, and endline facility and household surveys that USAID IHP is collecting independently. This is the performance evaluation component of the overall study. Because USAID IHP is operating in all health zones in its nine provinces, it is not feasible to identify and survey a comparable control group of health zones in these same provinces. Comparisons will be made in selected indicators between baseline (Year 1), midline (Year 4), and endline (Year 7). To conduct the comparisons, the absolute changes for each indicator value will be compared between survey waves using t-tests or Chi2 tests, both overall and for the three regional sub-groups. We will also stratify results by sex and age when applicable. 1 It should be noted that, in the health evaluation research literature, the term “impact” typically refers to the effects on health outcomes, such as lives saved, or disability-adjusted life years averted. In the HSS evaluation literature, “impact” is also often used to refer to the effects on service delivery or other aspects of health systems functioning (Adams et al., 2012). For the purposes of this evaluation, we use “impact” in the latter way, unless otherwise noted. 26 The Impact of USAID’s Integrated Health Program in the DRC Household-level changes. The population-based household surveys that USAID IHP administered at baseline included data on the healthcare services and behaviors that the project is supporting, including FP, ANC, skilled-delivery assistance, immunization, and treatment of childhood illnesses. Questions on exposure to the various interventions that will be supported by USAID IHP were also included in the household surveys. As with the facility- and health-worker data, changes will be analyzed for the entire sample and by the three regional sub-groups. Analysis of Enabling or Limiting Factors To assess the perceptions of various stakeholders on the implementation of USAID IHP-supported interventions, qualitative research involving a mix of methods will be carried out during the baseline, midline, and endline evaluations, with the aim of complementing the quantitative data analysis. During the baseline evaluation, we carried out research at the central level (Kinshasa) and in one province in the Katanga Region to assess the status of health systems and to understand details of the USAID IHP interventions and plans for implementation. Limitations There are several limitations of the evaluation and threats to carrying it out as planned. First, due to the data limitations described in the previous section, the impact evaluation component of the study will only investigate the impact of USAID IHP on proxy indicators related to service provision, including treatment of childhood illnesses, contraceptive use, vaccinations, and ANC. Because data on health outcomes, service quality, and health systems governance and leadership are not available from non-project areas, impacts on these aspects cannot be rigorously assessed. However, to descriptively explore these aspects, a performance evaluation is being carried out using both quantitative and qualitative data collected in the nine USAID IHP provinces2 to explore changes in proxy indicators for all three USAID IHP objectives—HSS, quality, integrated health services, and healthy behaviors—and the factors that enabled or limited the success of the project. Second, the impact evaluation of the study is based on routine data from the DHIS2. Although it is expected that using a research design based on DHIS2 data (i.e., numerous, repeated health zone observations over extended periods and the real-time indicators of service coverage) provide power and cost advantages over a research design based on intermittent population-based surveys, poor data quality remains a threat due to inaccurate data on counts of services provided (numerators) and the populations that are served (denominators). These disadvantages could lead to two consequences. First, poor data quality could add spurious variability to the dependent variable. If it is a random measurement error, it will add to the variation of the random error in the model, with the consequence of larger standard errors in the estimated coefficients increasing the chances of not finding significant effects when there is impact. This is a common problem in research studies based on data from management information systems, and there is little that the evaluator can do to address the issue. Second, the measurement error could potentially evolve over time as data quality improves with improvements made to data reporting systems. Because USAID IHP aims to improve DHIS2 data quality, these improvements could be different in treatment and comparison areas, which will create a type of endogeneity3 in the program variable of the model that varies over time, so it is not controlled by the fixed effects. This heteroskedasticity4 will be accounted for through the estimation of robust standard errors. Third, the D4I and USAID IHP surveys used as the source of baseline data were administered several months after USAID IHP started operations (in July 2018). This timing would be problematic if USAID IHP’s 2 Qualitative data will be collected in three provinces and Kinshasa. 3 A model in which the explanatory variable is correlated with the error term. 4 A case in which the standard errors of a variable are not constant over time. Baseline Report 27 implementation activities had already begun because it would bias the performance evaluation component of the study. However, based on previous meetings with USAID and the USAID IHP management teams, one of the key purposes of the USAID IHP surveys is to provide evidence that can be used to comprehensively assess the health systems needs in the nine provinces and, therefore, USAID IHP will withhold most of its interventions until after the survey data have been analyzed. Fourth, the DRC is an unstable environment and there is a possibility that both the implementation of USAID IHP, and D4I and USAID IHP survey activities, could be affected by political and social unrest during the project period. This is not likely to affect the impact evaluation component of the study, which relies on RHIS data, but it could affect future surveys if the data collection teams cannot safely travel to sampled provinces, health zones, and facilities affected by unrest, should that occur. 28 The Impact of USAID’s Integrated Health Program in the DRC Findings Health Systems Governance and Leadership Capacity to Plan, Implement, and Monitor Services Electricity enables efficient work and regular communication. Not all health zone offices had a source of electricity. Overall, 44.3 percent of health zone offices had any source of electricity, and slightly less (37.7 percent) had electricity functioning at the time of the survey (Figure 1.1). There was substantial variation among the provinces. The availability of electricity in health zone offices ranged from 6.3 percent in Kasai Oriental to 74.1 percent in Haut Katanga. Figure 1.1 Electricity in health zone offices, by province Figure 1.2 shows the mean number of hours of electricity during a typical eight-hour day in health zone offices that had a source of electricity. Sankuru had the lowest number of hours, at an average of 3.3 hours per day (among the 20% of offices that had any electricity). Although Kasai Oriental had the lowest coverage of electricity, the lone health zone office that had a source of electricity also reported the longest duration, with uninterrupted service during an eight-hour day. 50.0 40.0 6.3 20.0 74.1 50.0 44.3 42.3 20.0 6.3 20.0 66.7 41.7 37.7 0 20 40 60 80 100 Sud Kivu (n=26) Tanganyika (n=10) Kasai Oriental (n=16) Sankuru (n=15) Haut Katanga (n=27) Lualaba (n=12) Overall (n=106) Eastern Congo Kasai Katanga Facility has electricity Electricity currently functioning Baseline Report 29 Figure 1.2. Mean number of hours of functional electricity on a typical eight-hour day in health zone offices with a source of electricity, by province A reliable means of communication is critical for health zone offices to carry out their oversight and reporting functions. Figure 1.3 displays the percentage of health zone offices that had a cellular telephone or an Internet connection (either provided by the office or employees’ personal devices). Overall, there were cellular telephones at roughly one in four health zone offices and an Internet connection at slightly less than one-half. Health zone offices in Lualaba appeared the least connected, with 75 percent of the offices having no cellular telephone or Internet connection. 5.5 7.0 8.0 3.3 6.4 7.0 6.1 0 2 4 6 8 Sud Kivu (n=13) Tanganyika (n=4) Kasai Oriental (n=1) Sankuru (n=3) Haut Katanga (n=20) Lualaba (n=6) Eastern Congo Kasai Katanga Overall (n=47) 30 The Impact of USAID’s Integrated Health Program in the DRC Figure 1.3. Cellular telephone and Internet connections at health zone offices, by province Among health zone offices that had an Internet connection, the mean total connectivity time was slightly less than half a day (3.7 hours) (Figure 1.4). Sud Kivu experienced the longest cumulative interruptions, with only 2.6 hours of Internet connection per day. Connectivity was best in Sankuru, with a mean of 6 hours per day. Figure 1.4. Mean number of hours of functional Internet on a typical eight-hour day in health zone offices with an Internet connection, by province 30.8 60.0 18.8 33.3 14.8 0.0 24.5 53.9 50.0 50.0 13.3 63.0 25.0 46.2 42.3 20.0 43.8 53.3 33.3 75.0 43.4 0 20 40 60 80 100 Sud Kivu (n=26) Tanganyika (n=10) Kasai Oriental (n=16) Sankuru (n=15) Haut Katanga (n=27) Lualaba (n=12) Eastern Congo Kasai Katanga Overall (n=106) Cellular reception Internet connection Neither 2.6 3.4 4.3 6.0 4.2 4.7 3.7 0 2 4 6 8 Sud Kivu (n=14) Tanganyika (n=5) Kasai Oriental (n=8) Sankuru (n=2) Haut Katanga (n=17) Lualaba (n=3) Eastern Congo Kasai Katanga Overall (n=49) Baseline Report 31 HC and hospital directors were asked whether they agreed with five statements about their autonomy in managing their facility (Tables 1.1 and 1.2). Directors at both HCs and hospitals reported the highest levels of autonomy in staff assignments and staff activities, and the lowest levels of autonomy in obtaining resources and allocating their budgets. Table 1.1. Degree to which health center directors report having autonomy in management (n=328) Completely Somewhat Not at all Not applicable Staff assignments 61.6 23.2 13.7 1.5 Assign activities to staff 60.1 24.1 13.7 2.1 Services provided 54.6 29.0 14.3 2.1 Authority to obtain resources 37.2 22.0 36.9 4.0 Budget allocation 36.9 20.7 35.7 6.7 Table 1.2. Degree to which hospital directors report having autonomy in management (n=110) Completely Somewhat Not at all Not applicable Staff assignments 66.4 25.5 6.4 1.8 Assign activities to staff 62.7 27.3 8.2 1.8 Services provided 48.2 28.2 21.8 1.8 Budget allocation 35.5 28.2 30.9 5.5 Authority to obtain resources 30.0 25.5 40.9 3.6 One aim of USAID IHP is to administer the PICAL at all supported health zone offices. At the time of the baseline survey, a small percentage of health zone offices in the two provinces surveyed in the Katanga region (Haut Katanga and Lualaba) reported having completed a PICAL (Figure 1.5). Most facilities that completed a PICAL had done so in the past six months. In Haut Katanga, most offices had received a PICAL score, but none knew their score. 32 The Impact of USAID’s Integrated Health Program in the DRC Figure 1.5. Health zone offices’ Participatory Institutional Capacity Assessment and Learning Index participation and score, by province Improved Transparency and Oversight in Health Service Financing and Administration In health zone offices, data collectors administered the survey to the highest-ranking official present. Table 1.3 shows the status and role of the person interviewed. In nearly 60 percent of health zones, the data collector interviewed an official who identified him or herself as the head of the office. In slightly more than half of the cases, the Médecin Chef (chief physician) was present. Table 1.3. Status and role of respondents to health zone office survey, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Head of the office 53.9 100.0 37.5 66.7 63.0 50.0 59.4 Médecin chef 34.6 100.0 31.3 66.7 51.9 58.3 51.9 N 26 10 16 15 27 12 106 Health workers were asked to report the last time that someone at their facility interacted with them in a supervisory capacity. Figure 1.6 shows the percentage of health workers who had internal supervision in the week preceding the survey. Across all provinces, recent supervision was more prevalent in HCs than in hospitals. The levels of supervision were quite low overall, with less than one-quarter of health workers reporting recent supervision. 14.8 8.3 4.7 11.1 8.3 3.8 11.1 0 2.8 0 5 10 15 20 Sud Kivu (n=26) Tanganyika (n=10) Kasai Oriental (n=16) Sankuru (n=15) Haut Katanga (n=27) Lualaba (n=12) Eastern Congo Kasai Katanga Overall (n=106) Ever completed PICAL assessment PICAL within last six months Received PICAL score Knows PICAL score Baseline Report 33 Figure 1.6. Percentage of health workers who are not head of the facility who report being internally supervised in the past week The government-run health system in the DRC is designed to have a cascade of supervision: the national level supervises the provincial health offices, which in turn supervise the health zone offices. The health zone offices are primarily responsible for supervising the hospitals and HCs. Four of the six surveyed provincial health offices reported that they were visited by national-level authorities in the prescribed six completed calendar months before the survey (Table 1.4). Both provincial health offices that were not visited were in Katanga (Haut Katanga and Lualaba). Similarly, Table 1.4 shows the percentage of health zone offices that were visited by either national-level or provincial-level authorities. Overall, more than one-half of the health zone offices received supervision in the prescribed three-month timeframe. The rate was lowest in Sud Kivu at 38.5 percent. Hospitals are also supposed to receive in-person supervision at least once every three months. Overall, more than one-half of the hospitals were supervised by either the health zone or provincial health office. The percentage ranged from 25 percent in Sankuru to 80 percent in Tanganyika. By contrast, HCs are supposed to be supervised every month. Half of the HCs reported that they had been supervised by either the health zone or the provincial health office. Rates of supervision were highest in Lualaba (62.2%) and lowest in Sankuru (33.3%). Table 1.4. Percentage of offices and health facilities that were visited in a supervisory capacity by a higher-level authority in the prescribed timeframe, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Provincial health offices 1 1 1 1 1 1 6 National yes yes Yes yes no no 66.7 19.3 34.8 31.4 12.5 21.0 33.0 23.3 15.1 20.0 40.3 12.0 26.7 29.3 23.7 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health center (n=862) Hospital (n=351) 34 The Impact of USAID’s Integrated Health Program in the DRC Health zone offices 26 10 16 15 27 12 106 Province 34.6 50.0 62.5 40.0 55.6 50.0 48.1 National 3.9 30.0 25.0 6.7 11.1 8.3 12.3 Either 38.5 60.0 62.5 46.7 59.3 58.3 52.8 Hospitals 29 10 17 16 26 12 110 Health zone 34.5 70.0 41.2 18.8 50.0 41.7 40.9 Province 31.0 70.0 35.3 12.5 19.2 25.0 29.1 Either 55.2 80.0 52.9 25.0 53.9 50.0 51.8 Health centers 86 30 47 45 83 37 328 Health zone 41.9 56.7 53.2 28.9 54.2 62.2 48.5 Province 5.8 3.3 10.6 6.7 8.4 5.4 7.0 Either 43.0 56.7 55.3 33.3 55.4 62.2 50.0 CODESAs function as liaisons between health facilities and the community. A component of their role is to provide support to and oversight of health facility operations. Table 1.5 reports the percentage of HCs for which a CODESA performed various management and oversight tasks in the 90 days preceding the survey. Overall, the most common function performed was developing health messaging for the community (94.4%), followed by two functions related to medicines: being present when medicines arrived at the facility (84.6%) and assisting with the inventory of medicines (79%). Functions of financial oversight tended to be less common, with auditing financial records, planning for monthly expenses, assisting with the expense report, and examining the cash box being the least frequently performed tasks. Table 1.5. CODESA participation in health center management during the past 90 days, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Developed health messaging for community 95.4 100.0 97.8 100.0 85.0 97.3 94.4 Present when medications arrived 88.4 96.7 93.5 95.6 62.5 89.2 84.6 Assisted with inventory of medicines 89.5 93.3 93.5 86.7 51.3 75.7 79.0 Discussed fee schedule with management 81.4 83.3 78.3 97.8 52.5 91.9 77.5 Completed medicine acceptance report 84.9 96.7 91.3 93.3 47.5 70.3 77.2 Assisted in calculating health indicators 62.8 86.7 89.1 55.6 77.5 51.4 70.1 Completed inventory analysis report 80.2 93.3 73.9 80.0 47.5 46.0 68.5 Baseline Report 35 Ensured that cold chain was respected 69.8 83.3 71.7 48.9 58.8 56.8 64.2 Presented health indicators to community 68.6 90.0 76.1 42.2 58.8 40.5 62.4 Equipment inventory 69.8 93.3 67.4 68.9 36.3 59.5 62.0 Audited financial records 58.1 83.3 58.7 26.7 33.8 51.4 49.4 Assisted with plan for monthly expenses 51.2 83.3 50.0 13.3 32.5 46.0 43.5 Assisted with expense report 50.0 76.7 54.4 8.9 30.0 48.7 42.3 Examined cash box 48.8 80.0 43.5 15.6 33.8 29.7 40.4 N 86 30 47 45 83 37 328 Health facility directors were asked whether they had received formal feedback from the health zone office in the past 90 days and, if so, in what format (Tables 1.6 and 1.7). If the director reported that the health zone office had provided written feedback, the surveyor asked to see the report to verify. Overall, 63 percent of HCs and 45.4 percent of hospitals had received feedback. Sankuru was a negative outlier, with more than 93 percent of HCs and hospitals reporting that they had not received feedback in the past 90 days. 36 The Impact of USAID’s Integrated Health Program in the DRC Table 1.6. Percentage of health centers that received feedback from health zone offices in the past 90 days, by province Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Yes, verbal 36.1 23.3 27.7 4.4 35.4 8.1 26.0 Yes, written (verified) 20.9 33.3 25.5 2.2 35.4 32.4 25.1 Yes, written (not verified) 11.6 13.3 10.6 0.0 9.8 32.4 11.9 No 31.4 30.0 36.2 93.3 19.5 27.0 37.0 N 86 30 47 45 82 37 327 Table 1.7. Percentage of hospitals that received feedback from health zone offices in the past 90 days, by province Hospital Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Yes, verbal 24.1 0.0 23.5 6.3 25.0 0.0 16.7 Yes, written (verified) 17.2 50.0 5.9 0.0 29.2 25.0 19.4 Yes, written (not verified) 27.6 0.0 5.9 0.0 4.2 0.0 9.3 No 31.0 50.0 64.7 93.8 41.7 75.0 54.6 N 29 10 17 16 24 12 108 Health mutuelles are health financing schemes that operate in some communities in the DRC. Twenty health zone offices in three provinces reported that at least one health mutuelle was operating in their zone (Table 1.8). The majority (65%) kept a list of the health mutuelles, but only 15 percent kept a list of the mutuelle members. In 40 percent of health zones, the health zone office must authorize fee reductions for health mutuelle members, and in 40 percent, the health zone office reported that they made visits specifically to supervise the health mutuelles. Table 1.8. Health zone offices’ oversight of health mutuelles, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Health zone keeps list of health mutuelles 84.6 N/A 0.0 N/A 33.3 N/A 65.0 Health zone keeps list of members 15.4 N/A 0.0 N/A 16.7 N/A 15.0 Health zone must give permission for fee reductions 53.9 N/A 0.0 N/A 16.7 N/A 40.0 Health zone makes supervision visits 53.9 N/A 0.0 N/A 16.7 N/A 40.0 n 13 0 1 0 6 0 24 Baseline Report 37 Strengthened Coordination Directors of HCs and hospitals were asked about the frequency of their facilities holding internal management meetings, co-management meetings with the community, and meetings with a CODESA president in the past 12 months. Figures 1.7 and 1.8 show the percentage of facilities that reportedly had such meetings monthly or more often. Overall, more than 80 percent of HCs had each type of meeting monthly or more often. Rates were lowest among HCs in Haut Katanga. In half of the surveyed provinces, a higher percentage of HCs than hospitals reported having internal management meetings (Figure 1.7). The frequency of meetings with the community and the CODESA tended to be much lower in hospitals (Figure 1.8). Figure 1.7. Percentage of health centers that report having internal management meetings, community co-management meetings, and meetings with CODESA presidents monthly or more often in the past 12 months, by province 93 93 72 93 67 86 83 94 93 91 96 70 86 87 94 100 89 96 76 89 89 0 20 40 60 80 100 Sud Kivu (n=86) Tanganyika (n=30) Kasai Oriental (n=47) Sankuru (n=45) Haut Katanga (n=83) Lualaba (n=37) Eastern Congo Kasai Katanga Overall (n=328) Internal Community CODESA 38 The Impact of USAID’s Integrated Health Program in the DRC Figure 1.8. Percentage of hospitals that report having internal management meetings, community co-management meetings, and meetings with CODESA presidents monthly or more often in the past 12 months, by province Health zone offices may coordinate with various stakeholders, including CODESAs and other health zone offices, and may also attend province-level management committee meetings. Overall, health zone offices’ communication with CODESAs was high, with 88.7 percent communicating monthly (Table 1.9). Communication with other health zone offices occurred less frequently. Approximately 42 percent of health zone offices reported communicating with other health zone offices on a monthly basis, and one in four said that this communication happened only irregularly (Table 1.10). However, participation in provincial-level management committee meetings was very high, with more than 98 percent of health zone offices sending a representative (Figure 1.9). Table 1.9. Frequency of health zone offices’ communication with CODESAs, by province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Overall Monthly 92.3 100.0 87.5 66.7 92.6 91.7 88.7 Quarterly 3.9 0.0 0.0 0.0 7.4 8.3 3.8 Twice a year 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Annually 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Irregularly 3.9 0.0 12.5 33.3 0.0 0.0 7.6 n 26.0 10.0 16.0 15.0 27.0 12.0 106.0 90 80 76 88 77 92 84 38 20 35 38 42 25 35 25 100 50 0 55 25 48 0 20 40 60 80 100 Sud Kivu (n=29) Tanganyika (n=10) Kasai Oriental (n=17) Sankuru (n=16) Haut Katanga (n=26) Lualaba (n=12) Eastern Congo Kasai Katanga Overall (n=110) Internal Community CODESA Baseline Report 39 Table 1.10. Frequency of health zone offices’ communication with other health zone offices, by province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Overall Monthly 8.3 40.0 50.0 66.7 40.7 75.0 42.3 Quarterly 8.3 30.0 12.5 6.7 33.3 8.3 17.3 Twice a year 4.2 0.0 12.5 0.0 0.0 0.0 2.9 Annually 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Irregularly 33.3 30.0 18.8 26.7 22.2 16.7 25.0 Don't know 45.8 0.0 6.3 0.0 3.7 0.0 12.5 n 24 10 16 15 27 12 104 Figure 1.9. Percentage of health zone offices that send a representative to Comités de Gestion provincial meetings, by province Provincial health offices may also coordinate with their health zone offices and with other provincial health offices. Some also participate in technical meetings with the MOH or nongovernmental organizations (NGOs). Table 1.11 shows the frequency of provincial health offices reporting engagement in these activities. All provincial health offices communicated with health zone offices on a monthly basis. Communication with other provincial health offices was more mixed, ranging from irregularly (Sud Kivu) to monthly (Kasai Oriental, Haut Katanga, and Lualaba). All provincial health offices reported attending technical meetings at least annually. 100 100 93.75 100 96.3 100 98.1 0 20 40 60 80 100 Sud Kivu (n=26) Tanganyika (n=10) Kasai Oriental (n=16) Sankuru (n=15) Haut Katanga (n=27) Lualaba (n=12) Eastern Congo Kasai Katanga Overall (n=106) 40 The Impact of USAID’s Integrated Health Program in the DRC Table 1.11. Frequency of provincial health offices’ communication and attendance at technical meetings, by province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Health zone offices Monthly Monthly Monthly Monthly Monthly Monthly Other provincial meetings Irregularly Quarterly Monthly Quarterly Monthly Monthly Technical meetings Quarterly Annually Quarterly Quarterly Annually Quarterly d. Improved Disease Surveillance and Strategic Information Gathering and Use Data collectors were instructed to record whether data were displayed at HCs and hospitals, and if so, what types of data and where they were posted. Overall, 85.4 percent of HCs and 69.1 percent of hospitals displayed data, whether demographic data, service statistics, or disease surveillance statistics (data not shown). Figure 1.10 shows the percentage of HCs displaying data. Demographics were the most common data displayed, at 78.3 percent of HCs. Roughly one-half of the HCs posted service statistics and/or disease surveillance data. Data were more frequently posted in public areas where they could be viewed by both staff and patients. Figure 1.10 Data displayed at health centers, by type and location (n=328) Hospitals displayed demographic data less frequently compared with HCs but posted service statistics and disease surveillance data at roughly similar rates (Figure 1.11). Demographics and service statistics were more frequently posted in public areas, whereas disease surveillance data were more likely to be in staff-only areas of hospitals. 57.9 35.4 31.7 20.4 16.2 18.0 21.7 48.5 50.3 0 20 40 60 80 100 Demographics Service statistics Disease surveillance Public area Staff-only area None displayed Baseline Report 41 Figure 1.11. Data displayed at hospitals, by type and location (n=110) Theoretically, all health facilities have access to the Système national d'information sanitaire (SNIS [national health data collection and reporting system]), which displays current data at the facility, health zone, provincial, and national levels. The majority of HCs (95.4%) reported holding monthly meetings in which SNIS data were discussed (Figure 1.12). Slightly fewer (90.2%) HCs provided documentation that decisions were made based on SNIS data. This rate was notably low in Sankuru, at 73.3 percent. Figure 1.12. Percentage of health centers that hold a monthly meeting to discuss SNIS data, and that made decisions based on SNIS data, by province 23.6 28.2 20.9 15.5 24.6 26.4 60.9 47.3 52.7 0 20 40 60 80 100 Demographics Service statistics Disease surveillance Public area Staff-only area None displayed 96.5 100.0 100.0 97.8 88.0 97.3 94.2 95.4 93.3 97.9 73.3 88.0 94.6 90.2 0 20 40 60 80 100 Sud Kivu (n=86) Tanganyika (n=30) Kasai Oriental (n=47) Sankuru (n=45) Haut Katanga (n=83) Lualaba (n=37) Eastern Congo Kasai Katanga Overall (n=328) Monthly SNIS meeting Decisions based on SNIS data 42 The Impact of USAID’s Integrated Health Program in the DRC The percentage of hospitals that reported holding a monthly meeting in which SNIS data were discussed was also high, at 94.6 percent (Figure 1.13), but the percentage that used SNIS data in decision making was lower (87.3% overall). This rate was lowest in Kasai Oriental, at 70.6 percent. Figure 1.13. Percentage of hospitals that hold a monthly meeting to discuss SNIS data, and that make decisions based on SNIS data, by province When a disease of epidemic potential (maladie à potentiel épidémique [MAPEPI]) is detected in a health zone, the health zone office is supposed to report it to the provincial health office within 24 hours. Health zone offices were asked to recall their most recent case and the amount of time it took to report it (Table 1.12). Overall, 58.9 percent reported the case immediately, and an additional 15.9 percent reported it within the 24- hour window. Reporting was most timely in Sud Kivu (82.8% reported immediately or within 24 hours) and least timely in Tanganyika, where 55.5 percent reported on time and more than 40 percent took more than two days to report. Table 1.12. Timing of health zones’ reporting their most recent MAPEPI DHIS2 case, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Immediately 75.9 33.3 68.8 28.6 70.4 33.3 58.9 24 hours 6.9 22.2 12.5 28.6 11.1 33.3 15.9 Next day 10.3 0.0 12.5 14.3 3.7 16.7 9.4 > 2 days 6.9 44.4 0.0 28.6 7.4 0.0 11.2 Don't know 0.0 0.0 6.3 0.0 7.4 16.7 4.7 n* 29 9 16 14 27 12 107 *MAPEPI data were collected in a separate module; therefore, the n’s vary slightly from the other health zone-level analyses. Health zone offices were asked to list the reasons why a MAPEPI case may not be reported on time (Table 1.13). The most commonly cited barrier was lack of telephones (34.6%), followed by lack of Internet (32.7%). 93.1 100.0 100.0 93.8 88.5 100.0 94.6 100.0 100.0 70.6 81.3 76.9 100.0 87.3 0 20 40 60 80 100 Sud Kivu (n=29) Tanganyika (n=10) Kasai Oriental (n=17) Sankuru (n=16) Haut Katanga (n=26) Lualaba (n=12) Eastern Congo Kasai Katanga Overall (n=110) Monthly SNIS meeting Decisions based on SNIS data Baseline Report 43 Table 1.13. Barriers to a health zone submitting an immediate or weekly MAPEPI report on time, by province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Overall Kasai Oriental Sankuru Haut Katanga Lualaba Lack of telephones 41.4 66.7 12.5 42.9 33.3 16.7 34.6 Lack of Internet 41.4 0.0 25.0 42.9 33.3 33.3 32.7 Lack of transportation 13.8 33.3 50.0 42.9 22.2 25.0 28.0 Lack of service 10.3 11.1 0.0 50.0 3.7 25.0 14.0 Lack of electricity 6.9 0.0 25.0 7.1 14.8 0.0 10.3 Do not have correct form 6.9 0.0 12.5 7.1 11.1 16.7 9.4 Lack of credit 3.5 0.0 6.3 14.3 11.1 0.0 6.5 Terrain (distance, insecurity) 0.0 11.1 6.3 7.1 7.4 8.3 5.6 No time to complete reports 3.5 10.0 6.3 0.0 3.6 0.0 3.6 Unaware of submission deadline 0.0 0.0 0.0 7.1 3.7 0.0 1.9 No supervision visits 0.0 0.0 6.3 0.0 0.0 8.3 1.9 No staff to complete reports 0.0 11.1 0.0 0.0 0.0 0.0 0.9 n* 29 9 16 14 27 12 107 *MAPEPI data were collected in a separate module; therefore, the n’s vary slightly from the other health zone-level analyses. When they receive a MAPEPI report from a health zone office, provincial health offices are supposed to report to the MOH. Timeliness of reporting the most recent case varied among the provinces, with four provinces reporting immediately, one reporting the next day, and one (Haut Katanga) taking more than two days (Table 1.14). Table 1.14. Timing of provinces’ reporting their most recent MAPEPI DHIS2 case Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Immediately Immediately Next day Immediately >2 days Immediately Provincial health offices were also asked to list the barriers to timely MAPEPI case reporting (Table 1.15). Four of six provinces cited a lack of Internet. Lack of telephones, and lack of phone or Internet credit were also mentioned by two provinces each. 44 The Impact of USAID’s Integrated Health Program in the DRC Table 1.15. Barriers to a provincial health office submitting an immediate or weekly MAPEPI report on time, by province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Lack of Internet X X X X Lack of telephones X X Lack of credit X X Improved Management and Motivation of Human Resources for Health As part of its HSS approach, USAID IHP plans to provide support to improve the management and motivation of health workers in targeted provinces. This includes supporting the implementation of the iHRIS (IntraHealth International’s open-source human resources information software), which is designed to rationalize staffing. Thus far, the iHRIS has been rolled out in parts of Équateur and both of the Kasai provinces, and the World Bank has expressed interest in further expansion. Table 1.16 provides results on the percentage of health workers who reported ever attending an iHRIS training. Overall, only 1.3 percent of HC staff surveyed, and 1.4 percent of hospital staff surveyed received any training in the iHRIS. Surprisingly, no staff sampled in either Kasai province, where the iHRIS had been introduced, had received such a training. Table 1.16. Percentage of health workers who reported attending a training on iHRIS, by facility type and province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Health center 1.3 0.0 0.0 0.0 4.1 0.0 1.3 n 228 69 137 136 195 97 862 Hospital 2.2 0.0 0.0 0.0 2.7 2.4 1.4 n 93 30 62 50 75 41 351 To assess health worker satisfaction, the health worker module included several questions on whether workers were “satisfied with,” “not satisfied with,” or “neutral” to various dimensions of their job, including their workload and the availability of medicines and equipment. Table 1.17 reports the results among HC workers who were sampled. Overall, the majority of health workers (57%) reported being satisfied with their workload. However, a much lower percentage of workers reported being satisfied with the other dimensions of their work, including management (36.5%), the condition of the facility (26.3%), and the availability of medicines and equipment (20.4% and 12.2%, respectively). In terms of the province-specific results, Sankuru in the Kasai region had especially low levels of satisfaction for most dimensions, and Haut Katanga and Lualaba had relatively higher levels of satisfaction for most dimensions. Baseline Report 45 Table 1.17. Percentage of health center workers who reported being generally satisfied with their job and with specific dimensions of their job, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Workload 44.7 59.4 67.2 76.5 50.8 54.6 57.0 Management 23.7 58.0 35.8 9.6 48.2 67.0 36.5 Facility 25.9 27.5 27.7 11.8 32.8 32.0 26.3 Medicines 14.0 5.8 21.2 11.8 31.8 34.0 20.4 Equipment 10.5 7.3 10.2 2.9 21.0 17.5 12.2 Salary 5.7 13.0 2.9 0.0 6.2 1.0 4.5 Generally satisfied 37.3 58.0 28.5 33.1 36.9 41.2 37.2 n 228 69 137 136 195 97 862 Table 1.18 shows the levels of health worker satisfaction among hospital staff sampled. Among the 328 hospital workers sampled, the majority (53.6%) reported being satisfied with their workload, which was similar to the levels among HC workers. However, the percentage of workers who reported being satisfied with the availability of medicines (38.2%) and equipment (22.8%) was higher than the percentage reported among HC workers. In the province-specific results, the two provinces in the Kasai region—Kasai Oriental and Sankuru— exhibited low satisfaction levels for the availability of medicines and equipment relative to the levels among workers in the other sample provinces. Table 1.18. Percentage of hospital workers who reported being generally satisfied with their job and with specific dimensions of their job, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Workload 36.6 53.3 61.3 76.0 56.0 48.8 53.6 Management 21.5 40.0 22.6 16.0 36.0 53.7 29.3 Facility 51.6 23.3 30.7 22.0 36.0 65.9 39.6 Medicines 36.6 16.7 43.6 24.0 45.3 53.7 38.2 Equipment 32.3 10.0 17.7 8.0 26.7 29.3 22.8 Salary 3.2 10.0 0.0 0.0 4.0 4.9 3.1 Generally satisfied 49.5 66.7 21.0 34.0 38.7 46.3 41.0 n 93 30 62 50 75 41 351 Workers were also asked whether they were “generally satisfied” with their job. Overall, 37.2 percent of sampled HC workers reported being generally satisfied, which ranged from a low of 28.5 percent in Kasai Oriental to a high of 58 percent in Tanganyika (Table 1.17). Among hospital workers, 41 percent were “generally satisfied.” This ranged from a low of 21 percent in Kasai Oriental to a high of 66.7 percent in Tanganyika (Table 1.18). Health workers in HCs and hospitals were asked whether their salary for the last completed month had been paid and, if so, whether it had been paid on time (Table 1.19). Overall, 15.7 percent of health workers were 46 The Impact of USAID’s Integrated Health Program in the DRC paid on time. Notably, no health workers were paid on time in Tanganyika. Overall, 52 percent were still waiting for their salary. This percentage was highest in Kasai Oriental, where more than 72 percent had not yet been paid. A substantial number of health workers (25.3% overall) chose not to respond to the question. Table 1.19. Status of health workers’ salary payment for the last completed month, by province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Overall Paid on time 17.5 0.0 11.7 2.9 35.9 5.2 15.7 Paid late 4.4 4.4 6.6 6.6 14.4 2.1 7.1 Not yet paid 50.9 56.5 72.3 68.4 32.8 38.1 52.0 Don't know/No response 27.2 39.1 9.5 22.1 16.9 54.6 25.3 n 321 99 199 186 270 138 1,213 Health workers were also asked whether they had ever received their full government salary on time (Figure 1.14). Overall, 8.7 percent could recall being paid in full and on time. This ranged from none in Tanganyika to 21.9 percent in Haut Katanga. Figure 1.14. Percentage of health workers who reported that they have ever received their full government salary on time, by province In addition to their positions in government health facilities, more than 40 percent of health workers reported engaging in other activities to supplement their income (Table 1.20). Health workers were most likely to generate supplemental income in the Katanga region (Haut Katanga and Lualaba) and least likely in Eastern Congo (Sud Kivu and Tanganyika). The most common source of supplemental income was agriculture (23.7%), followed by trade or business (9.3%), and working in the private health sector (8.7%). 7.8 0.0 6.0 3.8 21.9 2.2 8.7 0 5 10 15 20 25 Sud Kivu (n=321) Tanganyika (n=99) Kasai Oriental (n=199) Sankuru (n=186) Haut Katanga (n=270) Lualaba (n=138) Eastern Congo Kasai Katanga Overall (n=1,213) Baseline Report 47 Table 1.20. Percentage of health workers with sources of supplemental income, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba No supplemental income 74.8 64.7 62.3 50.0 49.6 48.6 59.5 Agriculture 15.9 26.3 11.6 43.0 21.1 37.0 23.7 Trade/business 3.1 5.1 14.1 4.8 15.9 13.0 9.3 Private healthcare 5.6 6.1 9.1 7.5 15.2 5.8 8.7 Livestock 4.1 8.1 6.0 15.1 6.3 8.7 7.4 Rental income 1.3 1.0 1.5 0.0 3.0 2.2 1.6 Contributions from family 0.0 0.0 0.0 0.0 0.7 0.7 0.3 Other source 1.6 1.0 1.5 4.8 6.7 0.7 3.1 n 321 99 199 186 270 138 1,213 One mechanism that has been introduced in the DRC to improve health service quality and reach is performance-based financing (PBF), which is intended to improve health worker motivation at the facility, health zone, and provincial levels. With assistance from the World Bank, in 2016, the DRC introduced a major PBF project, Projet de Développement du Système de Santé (PDSS ), including in three provinces targeted by USAID IHP: Lualaba, Haut Katanga, and Haut Lomami. Figures 1.15 and 1.16 show the percentage of surveyed HC and hospital workers, respectively, who reported that they were part of a PBF scheme. Overall, 23.6 percent of HC workers and 23.8 percent of hospital workers had received performance-based payments, with Sud Kivu and Lualaba being the provinces with the highest coverage levels. 48 The Impact of USAID’s Integrated Health Program in the DRC Figure 1.15. Percentage distribution of health center workers by whether they were part of a performance-based financing scheme, by province Figure 1.16. Percentage distribution of hospital workers by whether they were part of a performance-based financing scheme, by province 59.6 97.1 85.0 89.8 75.1 48.8 73.6 4.6 2.9 3.8 3.7 2.8 35.9 15.0 10.2 21.1 47.6 23.6 0% 20% 40% 60% 80% 100% Sud Kivu (n=198) Tanganyika (n=69) Kasai Oriental (n=127) Sankuru (n=88) Haut Katanga (n=185) Lualaba (n=82) Eastern Congo Kasai Katanga Overall (n=749) Received PBF bonus in past month Supposed to receive PBF bonus but did not receive bonus in past month Not supposed to receive PBF bonus 66.7 100.0 82.3 84.4 68.6 38.9 71.8 7.8 7.1 5.6 4.4 25.6 17.7 15.6 24.3 55.6 23.8 0% 20% 40% 60% 80% 100% Sud Kivu (n=90) Tanganyika (n=29) Kasai Oriental (n=62) Sankuru (n=32) Haut Katanga (n=70) Lualaba (n=36) Eastern Congo Kasai Katanga Overall (n=319) Received PBF bonus in past month Supposed to receive PBF bonus but did not receive bonus in past month Not supposed to receive PBF bonus Baseline Report 49 Access to Quality, Integrated Health Services Service Readiness Surveyors assessed HCs to determine whether they offered eight select services that are part of the MOH’s minimum package of services. More than 90 percent of facilities offered prenatal consultations, postnatal consultations, malaria intermittent preventive treatment, vaccination, and growth monitoring. FP, mebendazole supplementation, and zinc supplementation were less common. Overall, 27.1 percent of HCs offered all eight services. Table 2.1. Percentage of health centers that offered select MOH minimum package of preventive services, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Prenatal consultations 100.0 96.8 100.0 100.0 96.4 100.0 98.8 Malaria intermittent preventative treatment 97.7 90.3 95.7 100.0 90.5 97.3 95.1 Postnatal consultations 85.9 93.6 95.7 97.7 91.7 100.0 92.7 FP 96.5 58.1 36.2 100.0 78.6 100.0 80.5 Vaccination 100.0 96.8 100.0 97.7 97.6 97.3 98.5 Growth monitoring 95.3 90.3 97.9 95.5 78.6 89.2 90.2 Zinc supplementation 43.5 32.3 42.6 6.8 51.2 54.1 40.6 Mebendazole supplementation 89.4 83.9 78.7 31.8 72.6 78.4 74.1 All select preventive services 32.9 19.4 17.0 4.6 33.3 46.0 27.1 n 86 30 47 45 83 37 328 Similarly, HCs were assessed for the degree to which they offered six services in the minimum package of curative services (Table 2.2). Although the majority of HCs (93.9%) reported that they performed normal deliveries, no more than one-half of the HCs offered each of the remaining services. Table 2.2. Percentage of health centers that offered select MOH minimum package of curative services, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba HIV testing 56.5 22.6 46.8 34.1 67.9 32.4 49.1 HIV treatment (post-exposure prophylaxis kit) 44.7 32.3 12.8 11.4 14.3 5.4 22.3 TB testing 12.9 22.6 29.8 20.5 15.5 18.9 18.6 TB treatment 18.8 83.9 55.3 36.4 42.9 37.8 40.9 Minor surgery 7.1 3.2 6.4 2.3 20.2 2.7 8.8 Normal deliveries 85.9 100.0 95.7 100.0 92.9 100.0 93.9 All curative services 0.0 0.0 2.1 0.0 2.4 0.0 0.9 n 86 30 47 45 83 37 328 50 The Impact of USAID’s Integrated Health Program in the DRC The MOH recommends that hospitals offer a minimum complementary package of services categorized into parasitology, hematology, bacteriology, and biochemical testing (Table 2.3). Overall, prevalence of these services was high, ranging from 99.1 percent (stool microscopic exam) to 56 percent (gram stain). Sankuru tended to have the lowest percentage of hospitals offering the complementary services. Table 2.3. Percentage of hospitals with capacity to conduct specific laboratory tests on the day of the survey, by province (MOH complementary package of activities) Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Parasitology Malaria microscopy 100.0 100.0 100.0 75.0 92.0 91.7 93.6 Stool direct microscopic exam 100.0 100.0 100.0 100.0 100.0 91.7 99.1 Hematology Hemoglobin testing 100.0 100.0 100.0 87.5 100.0 91.7 97.3 White blood cell count 96.6 80.0 94.1 62.5 88.0 66.7 84.4 Leukocyte formula 89.7 70.0 88.2 56.3 84.0 75.0 79.8 Sedimentation rate 86.2 90.0 94.1 68.8 92.0 83.3 86.2 Blood type crossmatch 100.0 100.0 100.0 93.8 100.0 91.7 98.2 Bacteriology Ziehl stain 96.6 100.0 94.1 93.8 92.0 91.7 94.5 Gram stain 65.5 80.0 23.5 6.3 84.0 66.7 56.0 Urine analysis 100.0 90.0 100.0 100.0 100.0 91.7 98.2 Biochemical Blood glucose 96.6 90.0 82.4 50.0 84.0 91.7 83.5 HIV testing 100.0 100.0 100.0 93.8 100.0 91.7 98.2 Syphilis testing 96.6 100.0 94.1 87.5 96.0 75.0 92.7 Pregnancy testing 96.6 100.0 100.0 93.8 96.0 91.7 96.3 Hepatitis testing 96.6 90.0 88.2 93.8 96.0 91.7 93.6 n 29 10 17 16 26 12 110 The minimum package of complementary services also calls for hospitals to be equipped to provide x-rays, ultrasounds, and autoclaves. Figure 2.1 shows the percentage of hospitals in each province that had at least one of each piece of equipment. X-rays were the least common, being found in 58.7 percent of hospitals (ranging from 18.8% in Sankuru to 75% in Lualaba), and autoclaves were the most common (ranging from 76.5% in Kasai Oriental to 100% in Tanganyika). Baseline Report 51 Figure 2.1. Percentage of hospitals with X-ray, ultrasound, and autoclave equipment, by province (MOH minimum package of complementary services) Surveyors assessed whether each HC had a source of electricity and if so, whether it was functioning at the time of the survey (Figure 2.2). The majority (76.2%) of HCs had no source of electricity, and fewer than 10 percent of HCs had electricity in Tanganyika, Kasai Oriental, and Sankuru. Among those that had a source of electricity, approximately three of ten did not have functional electricity at the time of the survey. Figure 2.2. Percentage of health centers with electricity, by province 58.6 60.0 70.6 18.8 68.0 75.0 58.7 100.0 100.0 82.4 43.8 72.0 83.3 80.7 89.7 100.0 76.5 87.5 88.0 91.7 88.1 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall X-ray Ultrasound Autoclave 18.8 0.0 2.1 2.3 36.9 18.9 17.1 7.1 3.2 2.1 0.0 14.3 5.4 6.7 74.1 96.8 95.7 97.7 48.8 75.7 76.2 0% 20% 40% 60% 80% 100% Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Yes, functioning Yes but not functioning None 52 The Impact of USAID’s Integrated Health Program in the DRC Overall, more hospitals had electricity than did HCs, with slightly fewer than one-half having a source of electricity (Figure 2.3). As was the case with HCs, hospitals’ access to electricity was lowest in Tanganyika, Kasai Oriental, and Sankuru. Among those hospitals that had electricity, there were outages on the day of the survey in all provinces, except Lualaba. Figure 2.3. Percentage of hospitals with electricity, by province Figure 2.4 displays the percentage of health facilities with improved sanitation. Across provinces, more than 80 percent of HCs and 90 percent of hospitals had improved sanitation, with the exception of Tanganyika, where the percentages were much lower (48.4% of HCs and 70% percent of hospitals). 20.7 30.0 23.5 31.3 56.0 50.0 34.9 20.7 10.0 5.9 6.3 24.0 0.0 13.8 58.6 60.0 70.6 62.5 20.0 50.0 51.4 0% 20% 40% 60% 80% 100% Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Yes, functioning Yes but not functioning None Baseline Report 53 Figure 2.4. Percentage of health facilities with improved sanitation, by province There was wide variation among provinces in the prevalence of private delivery rooms in health facilities (Figure 2.5). Among HCs, the percentage ranged from 21.6 percent in Lualaba to 91.5 percent in Kasai Oriental. The percentage of hospitals having private delivery rooms ranged from a low of 25 percent in Lualaba to a high of 100 percent in Tanganyika. Sud Kivu had the biggest discrepancy, with only 28.2 percent of its HCs having private delivery rooms compared with 65.5 percent of its hospitals. Figure 2.5. Percentage of health facilities with a private delivery room, by province 84.7 48.4 91.5 81.8 96.4 81.1 84.5 100.0 70.0 94.1 100.0 96.0 91.7 94.5 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health centers (n=328) Hospitals (n=110) 28.2 87.1 91.5 86.4 48.8 21.6 55.2 65.5 100.0 94.1 93.8 60.0 25.0 71.6 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health centers (n=328) Hospitals (n=110) 54 The Impact of USAID’s Integrated Health Program in the DRC Surveyors assessed whether HCs had selected tracer drugs in stock on the day of the survey (Table 2.4). Oxytocin was most frequently in stock, being available in 80.9 percent of HCs overall, followed by artesunate￾amodiaquine, and oral rehydration salts. Iron sulfate, and rifampicin and isoniazid were most commonly out of stock. Stock levels varied considerably among provinces. Overall, only 4 percent of HCs had all seven tracer drugs in stock. Table 2.4. Percentage of health centers that had selected tracer drugs in stock on the day of the survey, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Oxytocin 75.0 54.8 87.5 82.2 88.1 89.2 80.9 Artesunate-Amodiaquine 79.8 87.1 81.3 75.6 44.1 81.1 71.1 Oral rehydration salts 75.0 77.4 43.8 6.7 72.6 78.4 61.1 Depo Provera 73.8 22.6 20.8 86.7 57.1 83.8 59.9 Folic acid 61.9 38.7 52.1 53.3 58.3 67.6 56.8 Iron sulfate 27.4 22.6 31.3 13.3 39.3 35.1 29.5 Rifampicin and Isoniazid 13.1 38.7 29.2 31.1 31.0 27.0 26.4 All tracer drugs 2.4 0.0 2.1 0.0 8.3 8.1 4.0 n 86 30 47 45 83 37 328 Oxytocin was also the most commonly stocked tracer drug in hospitals, at 97.2 percent (Table 2.5). All tracer drugs were available in at least 50 percent of the hospitals on the day of the survey although, again, stock levels varied widely among the provinces. For example, oral hydration salts were present in all hospitals in Tanganyika but in only 6.3 percent of hospitals in Sankuru. Overall, 14.2 percent of hospitals had all seven tracer drugs in stock. Table 2.5. Percentage of hospitals that had selected tracer drugs in stock on the day of the survey, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Oxytocin 100.0 100.0 87.5 93.8 100.0 100.0 97.2 Rifampicin and Isoniazid 86.2 60.0 62.5 81.3 95.7 91.7 82.1 Artesunate-Amodiaquine 69.0 90.0 93.8 81.3 65.2 91.7 78.3 Oral rehydration salts 82.8 100.0 75.0 6.3 91.3 91.7 74.5 Folic acid 72.4 80.0 56.3 25.0 65.2 91.7 64.2 Depo Provera 72.4 20.0 31.3 93.8 69.6 66.7 63.2 Iron sulfate 55.2 60.0 37.5 18.8 65.2 75.0 51.9 All tracer drugs 10.3 0.0 0.0 0.0 30.4 41.7 14.2 n 29 10 17 16 26 12 110 Table 2.6 shows the percentage of HCs that had functioning basic equipment on the day of the survey (at least one functional piece of six types of basic equipment). Overall, more than one-half of the facilities had each type of equipment, and 37.4 percent had all six types. The percentage of HCs with all basic equipment was highest in Haut Katanga (54.8%) and lowest in Sankuru (2.2%). Baseline Report 55 Table 2.6. Percentage of health centers with all basic equipment on the day of the survey, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Stethoscope 96.4 93.6 95.8 51.1 97.6 100.0 90.6 Thermometer 98.8 87.1 93.8 40.0 98.8 97.3 88.8 Blood pressure monitor 91.7 64.5 89.6 33.3 96.4 97.3 82.7 Adult scale 83.3 87.1 77.1 60.0 90.5 91.9 82.4 Infant scale 67.9 61.3 83.3 42.2 75.0 75.7 68.7 Light source (spotlight) 85.7 61.3 43.8 11.1 81.0 51.4 62.0 All basic equipment 46.4 29.0 29.2 2.2 54.8 37.8 37.4 n 86 30 47 45 83 37 328 Basic equipment was more prevalent in hospitals, with more than 80 percent of hospitals having each individual item and 77.4 percent having all six pieces of basic equipment (Table 2.7). The percentage of hospitals with all basic equipment was highest in Sud Kivu (93.1%) and lowest in Sankuru (50%). Table 2.7. Percentage of hospitals with all basic equipment on the day of the survey, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Infant scale 100.0 100.0 100.0 87.5 100.0 100.0 98.1 Thermometer 96.6 100.0 100.0 87.5 100.0 100.0 97.2 Blood pressure monitor 100.0 100.0 100.0 81.3 100.0 100.0 97.2 Adult scale 100.0 100.0 93.8 93.8 91.3 100.0 96.2 Stethoscope 100.0 100.0 100.0 81.3 95.7 100.0 96.2 Light source (spotlight) 96.6 90.0 68.8 56.3 95.7 58.3 81.1 All basic equipment 93.1 90.0 68.8 50.0 87.0 58.3 77.4 n 29 10 17 16 26 12 110 An inventory of equipment related to infection control was carried out at HCs (Table 2.8). Most HCs (89.7%) had a safe method of final disposal of biohazardous materials. Gowns and sharps boxes were also highly prevalent in HCs. The least prevalent types of equipment were eye protection (8.2%) and test strips (4.3%). Haut Katanga was the only province in which any HCs had all pieces of infection control equipment. 56 The Impact of USAID’s Integrated Health Program in the DRC Table 2.8. Percentage of health centers with adequate infection control equipment, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Safe final disposal 89.3 96.8 97.9 77.8 88.1 91.9 89.7 Gowns 72.6 87.1 68.8 46.7 79.8 94.6 74.2 Sharps box 84.5 67.7 70.8 55.6 77.4 67.6 73.3 Gloves 51.2 61.3 87.5 53.3 72.6 83.8 66.9 Sink or basin 81.0 38.7 85.4 22.2 65.5 56.8 62.9 Clean water 77.4 48.4 62.5 11.1 54.8 29.7 52.3 Autoclave or steam sterilizer 46.4 32.3 27.1 48.9 42.9 37.8 40.7 Disinfectant (chlorine powder) 44.1 6.5 18.8 2.2 31.0 16.2 24.6 Masks 19.1 22.6 22.9 4.4 38.1 32.4 24.3 Eye protection 9.5 9.7 10.4 2.2 8.3 8.1 8.2 Test strip 6.0 0.0 2.1 4.4 7.1 0.0 4.3 All equipment 0.0 0.0 0.0 0.0 1.2 0.0 0.3 n 86 30 47 45 83 37 328 Similar to HCs, most hospitals (92.5%) had a safe method of final disposal of biohazardous materials (Table 2.9). Although more prevalent in hospitals than in HCs, eye protection and test strips were still relatively rare, as was disinfectant. Hospitals in Sud Kivu and Haut Katanga were the only hospitals that had all infection control equipment. Table 2.9. Percentage of hospitals with adequate infection control equipment, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Safe final disposal 96.6 90.0 100.0 81.3 95.7 83.3 92.5 Autoclave or steam sterilizer 89.7 100.0 81.3 87.5 87.0 91.7 88.7 Gloves 93.1 100.0 93.8 87.5 65.2 100.0 87.7 Sink or basin 82.8 100.0 81.3 68.8 95.7 91.7 85.9 Masks 96.6 90.0 68.8 87.5 69.6 75.0 82.1 Sharps box 93.1 80.0 81.3 50.0 82.6 91.7 81.1 Gowns 89.7 80.0 75.0 62.5 69.6 91.7 78.3 Clean water 82.8 80.0 87.5 31.3 82.6 66.7 73.6 Eye protection 72.4 30.0 37.5 18.8 43.5 58.3 47.2 Disinfectant 69.0 50.0 18.8 0.0 43.5 33.3 39.6 Test strip 37.9 0.0 12.5 0.0 13.0 8.3 16.0 Baseline Report 57 All equipment 10.3 0.0 0.0 0.0 4.4 0.0 3.8 n 29 10 17 16 26 12 110 The percentage of HCs and hospitals that offered a package of comprehensive sexual- and gender-based violence services is illustrated in Figure 2.6. Prevalence of services in HCs ranged from 9.1 percent in Sankuru to 45.9 percent in Sud Kivu. In all provinces, hospitals were more likely to have these services, ranging from 50 percent of hospitals in Tanganyika to 94.1 percent in Kasai Oriental. Figure 2.6. Percentage of health facilities offering a package of comprehensive sexual- and gender-based violence services, by province Overall, the majority of HCs (55.5%) and hospitals (83.5%) offered a permanent or long-acting method of FP (Figure 2.7). These services were least prevalent in both HCs and hospitals in Kasai Oriental. 45.9 25.8 29.8 9.1 31.0 13.5 29.3 86.2 50.0 94.1 62.5 92.0 66.7 78.9 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health centers (n=328) Hospitals (n=110) 58 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.7. Percentage of health facilities offering a long-acting or permanent method of FP, by province Figure 2.8 shows the percentage of health facilities that had at least one health worker trained in youth￾friendly FP services. No more than one-half of the HCs or hospitals in any province had such a health worker. In most provinces, hospitals had health workers trained in youth-friendly FP at higher rates than did HCs, with the exception of Tanganyika. Figure 2.8. Percentage of health facilities with at least one health worker trained in youth-friendly FP services, by province Facilities are encouraged to have FP information and resources specific to youth. Again, fewer than one-half of the HCs and hospitals in any given province had these resources (Figure 2.9). 68.2 45.2 21.3 47.7 60.7 75.7 55.5 93.1 80.0 58.8 93.8 84.0 83.3 83.5 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health centers (n=328) Hospitals (n=110) 30.6 25.8 10.6 27.3 23.8 40.5 26.2 41.4 20.0 29.4 37.5 44.0 50.0 38.5 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health centers (n=328) Hospitals (n=110) Baseline Report 59 Figure 2.9. Percentage of health facilities with FP information and resources specific to youth, by province Tables 2.10 and 2.11 list the essential staff, supplies, and equipment for administering long-acting or permanent methods of contraception. None of the surveyed HCs had everything necessary to administer any of the methods (Table 2.10). One of the major limiting factors in performing male and female sterilization in the HCs was the lack of trained health workers. Specific pieces of equipment (dissecting and ring forceps for male sterilization, and uterine elevators and Ramathibodi hooks for female sterilization) were also rarely present in HCs. Table 2.10. Percentage of health centers meeting minimum standards for essential staff, supplies, and equipment to support the provision of long-acting or permanent methods of contraception, by province Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Male sterilization At least one trained health worker 2.4 0.0 0.0 0.0 0.0 2.7 0.9 Bucket 76.2 80.7 64.6 15.6 83.3 83.8 69.3 Chlorine powder 44.1 6.5 18.8 2.2 31.0 16.2 24.6 Sharps container 84.5 67.7 70.8 55.6 77.4 67.6 73.3 Lidocaine 75.0 54.8 37.5 71.1 70.2 78.4 66.3 Dissecting forceps 2.4 0.0 0.0 0.0 4.8 10.8 3.0 Ring forceps 1.2 0.0 0.0 2.2 2.4 5.4 1.8 All 0.0 0.0 0.0 0.0 0.0 0.0 0.0 25.9 16.1 23.4 20.5 14.3 21.6 20.4 34.5 10.0 29.4 43.8 48.0 16.7 33.9 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health centers (n=328) Hospitals (n=110) 60 The Impact of USAID’s Integrated Health Program in the DRC Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Female sterilization At least one trained health worker 2.4 0.0 2.1 0.0 10.7 2.7 4.0 Bucket 76.2 80.7 64.6 15.6 83.3 83.8 69.3 Chlorine powder 44.1 6.5 18.8 2.2 31.0 16.2 24.6 Sharps container 84.5 67.7 70.8 55.6 77.4 67.6 73.3 Lidocaine 75.0 54.8 37.5 71.1 70.2 78.4 66.3 Uterine elevator 4.8 0.0 14.6 0.0 9.5 0.0 5.8 Ramathibodi hook 1.2 0.0 20.8 0.0 1.2 2.7 4.0 All 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Intrauterine device insertion and removal At least one trained health worker: insertion 11.9 25.8 14.6 0.0 36.9 18.9 19.2 At least one trained health worker: removal 15.5 25.8 14.6 0.0 38.1 18.9 20.4 Bucket 76.2 80.7 64.6 15.6 83.3 83.8 69.3 Chlorine powder 44.1 6.5 18.8 2.2 31.0 16.2 24.6 Sharps container 84.5 67.7 70.8 55.6 77.4 67.6 73.3 Iodine 52.4 48.4 37.5 44.4 46.4 37.8 45.6 Uterine elevator 4.8 0.0 14.6 0.0 9.5 0.0 5.8 Ring forceps 1.2 0.0 0.0 2.2 2.4 5.4 1.8 All 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Implant (insertion and removal): Norplant, Jadelle, Sino-Implant II At least one trained health worker: insertion 60.7 48.4 16.7 42.2 58.3 64.9 50.5 At least one trained health worker: removal 64.3 41.9 16.7 44.4 59.5 67.6 51.7 Bucket 76.2 80.7 64.6 15.6 83.3 83.8 69.3 Chlorine powder 44.1 6.5 18.8 2.2 31.0 16.2 24.6 Sharps container 84.5 67.7 70.8 55.6 77.4 67.6 73.3 Iodine 52.4 48.4 37.5 44.4 46.4 37.8 45.6 Trocar 14.3 3.2 8.3 0.0 19.1 8.1 10.9 Scalpel 6.0 0.0 6.3 0.0 17.9 0.0 7.0 Forceps 1.2 0.0 4.2 0.0 3.6 0.0 1.8 All 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Implant (insertion and removal): Implanon Iodine 52.4 48.4 37.5 44.4 46.4 37.8 45.6 Implanon applicator 28.6 25.8 4.2 11.1 28.6 43.2 24.0 Baseline Report 61 Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Scalpel 6.0 0.0 6.3 0.0 17.9 0.0 7.0 Forceps 1.2 0.0 4.2 0.0 3.6 0.0 1.8 All 1.2 0.0 0.0 0.0 0.0 0.0 0.3 n 86 30 47 45 83 37 328 By contrast, hospitals tended to have higher levels of readiness to provide FP services, although readiness in hospitals was still relatively low (Table 2.11). In three provinces (Tanganyika, Sankuru, and Lualaba), no surveyed hospitals had all personnel and equipment necessary to provide any method of permanent or long￾acting contraception. In hospitals, the limiting factor was not the lack of health workers trained to administer the methods, but rather the availability of equipment. Table 2.11. Percentage of hospitals meeting minimum standards for essential supplies and equipment to support the provision of long-acting or permanent methods of contraception, by province Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Male sterilization At least one trained health worker 20.69 10.0 12.5 6.25 8.7 8.33 12.26 Bucket 89.7 100.0 93.8 62.5 87.0 100.0 87.7 Chlorine powder 69.0 50.0 18.8 0.0 43.5 33.3 39.6 Sharps container 93.1 80.0 81.3 50.0 82.6 91.7 81.1 Lidocaine in stock on day of survey 96.6 80.0 81.3 100.0 91.3 83.3 90.6 Dissecting forceps 34.5 0.0 12.5 6.3 21.7 25.0 19.8 Ring forceps 31.0 0.0 6.3 6.3 13.0 8.3 14.2 All 13.8 0.0 0.0 0.0 8.7 0.0 5.7 Female sterilization At least one trained health worker 72.4 20.0 37.5 56.3 52.2 75.0 55.7 Bucket 89.7 100.0 93.8 62.5 87.0 100.0 87.7 Chlorine powder 69.0 50.0 18.8 0.0 43.5 33.3 39.6 Sharps container 93.1 80.0 81.3 50.0 82.6 91.7 81.1 Lidocaine in stock on day of survey 96.6 80.0 81.3 100.0 91.3 83.3 90.6 Uterine elevator 37.9 0.0 43.8 6.3 43.5 16.7 29.3 Ramathibodi hook 31.0 0.0 43.8 25.0 17.4 0.0 22.6 All 20.7 0.0 6.3 0.0 8.7 0.0 8.5 Intrauterine device insertion and removal At least one trained health worker: insertion 82.8 60.0 50.0 31.3 69.6 58.3 62.3 62 The Impact of USAID’s Integrated Health Program in the DRC Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba At least one trained health worker: removal 82.8 60.0 50.0 31.3 69.6 50.0 61.3 Bucket 89.7 100.0 93.8 62.5 87.0 100.0 87.7 Chlorine powder 69.0 50.0 18.8 0.0 43.5 33.3 39.6 Sharps container 93.1 80.0 81.3 50.0 82.6 91.7 81.1 Iodine* 52.4 48.4 37.5 44.4 46.4 37.8 45.6 Uterine elevator 37.9 0.0 43.8 6.3 43.5 16.7 29.3 Ring forceps 31.0 0.0 6.3 6.3 13.0 8.3 14.2 All 20.7 0.0 0.0 0.0 4.4 0.0 6.6 Implant (insertion and removal): Norplant, Jadelle, Sino-Implant II At least one trained health worker: insertion 89.7 70.0 50.0 75.0 78.3 58.3 73.58 At least one trained health worker: removal 86.2 70.0 50.0 75.0 78.3 58.3 72.64 Bucket 89.7 100.0 93.8 62.5 87.0 100.0 87.7 Chlorine powder 69.0 50.0 18.8 0.0 43.5 33.3 39.6 Sharps container 93.1 80.0 81.3 50.0 82.6 91.7 81.1 Iodine* 52.4 48.4 37.5 44.4 46.4 37.8 45.6 Trocar 51.7 10.0 25.0 18.8 34.8 16.7 31.1 Scalpel 27.6 10.0 25.0 31.3 47.8 8.3 28.3 Forceps 31.0 30.0 18.8 31.3 26.1 33.3 28.3 All 17.2 0.0 6.3 0.0 8.7 0.0 7.6 Implant (insertion and removal): Implanon Iodine* 52.4 48.4 37.5 44.4 46.4 37.8 45.6 Implanon applicator 51.7 30.0 0.0 25.0 26.1 66.7 34.0 Scalpel 27.6 10.0 25.0 31.3 47.8 8.3 28.3 Forceps 31.0 30.0 18.8 31.3 26.1 33.3 28.3 All 17.2 0.0 0.0 0.0 4.4 0.0 5.7 n 29 10 17 16 26 12 110 *Prevalence of iodine in facilities was collected in a different module and therefore has a different denominator from the other items in Table 2.11. Service Delivery The MOH designates minimum staffing levels for urban and rural HCs. Table 2.12 shows the percentage of HCs found to have the minimum numbers of nurses, midwives, lab technicians, and maintenance technicians on staff. Overall, 23.2 percent of HCs had a sufficient number of nurses, whereas the percentage with a sufficient number of the other cadres was much lower. In ten cases, there were no HCs with the minimum numbers of health workers in a specific cadre. Baseline Report 63 Table 2.12. Percentage of health centers with adequate staffing numbers and mix according to government guidelines,* by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Nurses 37.2 13.3 4.3 42.2 16.9 13.5 23.2 Midwives 3.5 0.0 0.0 0.0 4.8 5.4 2.7 Lab techs 0.0 0.0 0.0 0.0 2.4 0.0 0.6 Maintenance techs 1.2 0.0 4.3 0.0 9.6 2.7 3.7 n 86 30 47 45 83 37 328 *Rural HCs should have a minimum of four nurses (A1/A2), two midwives, one laboratory technician, and one maintenance technician. Urban HCs should have a minimum of eight nurses (A1/A2), four midwives, two laboratory technicians, and one maintenance technician. To explore the factors that influence health worker behaviors, health workers were asked whether they agreed with several statements about their knowledge and attitudes toward patients. The results for HC workers are presented in Table 2.13 and the results for hospital workers are presented in Table 2.14. Overall, more than one-half of the HC workers (55.6%) agreed with the statement that “patients I care for are not educated enough to make good health decisions for themselves,” which is lower than the 60.7 percent of hospital workers who agreed with this statement. However, almost all HC and hospital workers agreed with the statement that “I consider patients to be worthy of my respect regardless of how poor or low status they are” (97.1% and 99.7%, respectively). Other findings of note were that most workers at both HCs and hospitals agreed that it was important to engage patients in discussions and to provide information to patients, as indicated through several indicators. Nevertheless, a substantial number of health workers (25.1% of HC workers and 21.8% of hospital workers) agreed with the statement that “I was trained to provide clinical care; being respectful to every patient is not part of my job.” 64 The Impact of USAID’s Integrated Health Program in the DRC Table 2.13. Percentage of health center workers who reported agreeing with statements about their knowledge and attitudes, by province Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Patients I care for are not educated enough to make good health decisions for themselves 49.3 69.6 54.0 75.0 57.7 30.9 55.6 Patients I care for are not grateful for the efforts I make when I care for them 35.2 46.4 37.2 51.5 40.2 28.9 39.4 I consider my patients to be worthy of respect no matter how poor or low status they are 96.0 94.2 97.8 98.5 98.5 95.9 97.1 Patients often treat me without respect, so it is hard to treat them with respect 11.5 10.1 5.1 5.2 4.6 3.1 6.9 Patients I care for make bad decisions regarding their health no matter what I tell them 26.9 55.1 18.3 41.2 26.8 10.3 28.1 Engaging patients in discussions leads to better health outcomes than just telling them what is best for them 91.6 68.1 92.0 89.0 90.2 93.8 89.3 My patients will work hard to improve their health when they are given the proper information 86.3 82.6 96.4 94.1 87.6 87.6 89.3 My role is to provide clinical care, not to teach patients about how to take care of themselves 23.6 8.8 21.9 26.5 19.1 11.3 20.2 I do not spend a lot of thought about what patients may think about their experience at the clinic as I have other things to worry about 9.7 10.3 13.1 16.9 8.9 7.2 11.0 An important part of my job is to communicate with patients to make sure they understand their care 96.0 91.2 96.4 97.1 94.3 94.9 95.3 I try hard to think about the patients’ healthcare needs, not just solving their immediate problem 86.3 77.9 94.2 91.9 80.0 82.5 86.0 I was trained to provide clinical care; being respectful to every patient is not my job 29.7 32.4 15.6 41.9 13.7 21.7 25.1 When medicine is given, it is important that I explain well what it does for the patient and how it helps them 97.8 92.7 95.6 99.3 96.8 94.9 96.7 n 228 69 137 136 195 97 862 Baseline Report 65 Table 2.14 Percentage of hospital workers who reported agreeing with statements about their knowledge and attitudes, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Patients I care for are not educated enough to make good health decisions for themselves 51.6 70.0 62.9 80.0 60.0 48.8 60.7 Patients I care for are not grateful for the efforts I make when I care for them 34.4 40.0 30.7 60.0 41.3 26.8 38.5 I consider my patients to be worthy of respect no matter how poor or low status they are 98.9 100.0 100.0 100.0 100.0 100.0 99.7 Patients often treat me without respect, so it is hard to treat them with respect 8.6 0.0 3.2 4.0 6.7 2.4 5.1 Patients I care for make bad decisions regarding their health no matter what I tell them 30.1 63.3 19.4 44.0 21.3 19.5 29.9 Engaging patients in discussions leads to better health outcomes than just telling them what is best for them 89.3 73.3 93.6 94.0 89.3 90.2 89.5 My patients will work hard to improve their health when they are given the proper information 87.1 90.0 85.5 96.0 96.0 78.1 89.2 My role is to provide clinical care, not to teach patients about how to take care of themselves 21.5 3.3 16.1 24.0 9.5 10.0 15.5 I do not spend a lot of thought about what patients may think about their experience at the clinic as I have other things to worry about 9.7 16.7 11.3 12.2 1.4 7.5 8.9 An important part of my job is to communicate with patients to make sure they understand their care 95.7 100.0 93.6 100.0 94.6 92.5 95.7 I try hard to think about the patients’ healthcare needs not just solving their immediate problem 85.0 86.7 87.1 96.0 81.1 80.0 85.7 I was trained to provide clinical care; being respectful to every patient is not my job 26.9 26.7 17.7 34.0 9.5 20.0 21.8 66 The Impact of USAID’s Integrated Health Program in the DRC Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba When medicine is given, it is important that I explain well what it does for the patient and how it helps them 98.9 96.7 96.8 100.0 100.0 95.0 98.3 n 93 30 62 50 75 41 351 Medical Record Review While on site at the facilities, the surveyors reviewed the facility registers and recorded the volume of specified types of cases and services for the last completed calendar month as a means of assessing the quality of care being delivered. Here, we provide results related to labor and delivery care. Overall, records were reviewed for 3,300 deliveries at HCs and 1,090 deliveries at hospitals. At HCs, 71.4 percent (n=2,355) of the records included a delivery outcome (survival or mortality); at hospitals, 89.3 percent (n=973) of the delivery records included an outcome. Figure 2.10 shows maternal delivery outcomes separately for HCs and for hospitals, where an outcome was recorded. The registers showed that 93.6 percent of women who delivered in HCs survived childbirth with no complications, as did 88.4 percent who delivered in hospitals, which presumably have a higher percentage of high-risk deliveries. The percentage of women who had complications but survived was 5.5 percent at HCs and 10 percent at hospitals. The maternal death rate was recorded as 1 percent at HCs and 1.6 percent at hospitals. Figure 2.10. Maternal delivery outcomes, among those for whom an outcome was recorded, by health facility type Health workers record certain delivery-related complications in the facility register, specifically, antepartum hemorrhage, postpartum hemorrhage, and postpartum infection. Among the women who died in childbirth at a HC, nearly 70 percent had no complication recorded (Figure 2.11). The only complication recorded for women who died in childbirth at a HC was postpartum hemorrhage, at 30.4 percent. A larger percentage of 1.6 10.0 88.4 1.0 5.5 93.6 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Deceased Survived, complications Survived, no complications Health center (n=2,355) Hospital (n=973) Baseline Report 67 women who died during childbirth in hospitals had an associated complication recorded. The most common was postpartum hemorrhage, followed by antepartum hemorrhage and postpartum infection. Figure 2.11. Percentage of women who died during childbirth who experienced complications, by facility type Most of the women who did not die in childbirth had no recorded complications (94.5% in HCs and 89.9% in hospitals) (Figure 2.12). The most common complication among women who survived childbirth was postpartum hemorrhage, followed by antepartum hemorrhage and postpartum infection. Figure 2.12. Complications recorded for women who did not die during childbirth, by facility type MOH guidelines indicate that a woman should remain in the health facility for at least 24 hours following a normal delivery. Facility registers contain delivery and discharge dates, but not times. Therefore, we present the number of 6.3 37.5 25.0 43.8 0.0 30.4 0.0 69.6 0 10 20 30 40 50 60 70 80 90 100 Postpartum infection Postpartum hemorrhage Antepartum hemorrhage None recorded Health center (n=16) Hospital (n=23) 1.7 6.3 4.1 89.9 1.5 2.9 2.2 94.5 0 10 20 30 40 50 60 70 80 90 100 Postpartum infection Postpartum hemorrhage Antepartum hemorrhage None recorded Health center (n=2,355) Hospital (n=957) 68 The Impact of USAID’s Integrated Health Program in the DRC calendar days that women remained in the facility. Overall, 13.5 percent of women who had a normal delivery in a HC were discharged at two days and an additional 59.9 percent stayed three or more days (Figure 2.13). The lowest￾performing province was Sankuru, where 54.8 percent of women remained at least two days, and the highest performing was Lualaba, where 88.6 percent remained for at least two days. Figure 2.13. Lengths of stay in health centers after normal delivery, by province *Note: Maternal deaths and Caesarean sections are excluded from this analysis as guidelines pertain to discharges after a normal delivery. Lengths of stay after normal deliveries in hospitals are shown in Figure 2.14. Overall, rates were similar to those in HCs, with 13.5 percent of women being discharged at two days and an additional 63.7 percent staying three days or more. In hospitals, the lowest performer was Tanganyika (where 64% remained for at least two days) and the highest was Lualaba, at 91 percent remaining for at least two days. 65.3 46.4 66.6 49.9 56.1 71.0 59.9 16.7 11.8 12.5 4.9 14.1 17.6 13.5 4.6 24.7 3.5 16.2 3.9 4.3 7.7 13.5 17.1 17.5 29.0 26.0 7.1 18.9 0 20 40 60 80 100 Sud Kivu (n=846) Tanganyika (n=304) Kasai Oriental (n=464) Sankuru (n=445) Haut Katanga (n=817) Lualaba (n=369) Total (n=3,245) Three or more days Two days Next day Same day Baseline Report 69 Figure 2.14. Lengths of stay in hospitals after normal delivery, by province *Note: Maternal deaths and Caesarean sections are excluded from this analysis as guidelines pertain to discharges after a normal delivery. Clinical Vignette 1: Child Health A total of 961 health workers responded to a vignette describing a hypothetical case of a four-year-old boy with diarrhea who was brought to the clinic by his mother. The purpose of the vignettes was to assess providers’ knowledge and self-reported clinical practices. It should be noted, however, that clinical vignettes do not measure actual clinical practices. The majority of respondents (92.5%) were nurses (77.9%) or physicians (14.6%). There were few midwives (4.2%) and CHWs (3.3%). The vignette gave a short description of the patient and his symptoms. It asked the health workers what questions they would ask to fully understand the condition of the patient. After the questions by the health worker about the condition of the patient were collected, more information was given about the patient. Then, the vignette asked how the health worker would conduct the physical exam and, after gathering this information, the data collector provided the results of the physical exam. Based on these results, the health worker was asked to give a differential diagnosis and order laboratory tests. The vignette provided the results of the laboratory tests. Based on those results, the health worker was asked to determine a final diagnosis, indicate the appropriate treatment, and describe the counseling to be given to the patient or their caregiver before leaving the clinic. Table 2.15 shows the respondents’ distribution, by facility type and province. Table 2.15. Total number of health workers who responded to the child health vignette, by facility type and province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Health center 194 53 120 114 180 46 707 Hospital 59 22 53 39 62 19 254 n 253 75 173 153 242 65 961 69.2 51.2 68.1 61.2 56.2 75.8 63.7 17.5 12.8 4.4 14.3 14.9 15.2 13.5 2.6 26.7 0.6 4.1 3.4 2.0 4.8 10.7 9.3 26.9 20.4 25.5 7.1 18.0 0 20 40 60 80 100 Sud Kivu (n=234) Tanganyika (n=86) Kasai Oriental (n=160) Sankuru (n=147) Haut Katanga (n=235) Lualaba (n=99) Total (n=961) Three or more days Two days Next day Same day 70 The Impact of USAID’s Integrated Health Program in the DRC Questions Asked by Health Workers about the Patient Figure 2.15 lists questions that a health worker should ask when assessing a patient with this clinical presentation. The questions most frequently asked by health workers were related to temperature (fever), frequency of stools, and treatment given before the child was brought to the clinic. We noted that some respondents, ranging from 16.5 percent to 44.3 percent, missed these three questions. On the other hand, fewer than one-half of the health workers asked questions about the consistency of stools, vomiting, progression of diarrhea, eating and drinking, and the presence of blood in the stool. The proportion of health workers missing those questions ranged from 57.1 percent to 80.2 percent. In general, relatively higher numbers of physicians asked most of these questions compared with nurses who asked them (results not shown). Figure 2.15. Percentage of health workers who asked about aspects of medical history, by facility type Information about the patient’s social environment is important for arriving at a diagnosis and planning care. However, fewer than 10 percent of health workers based in both hospitals and HCs asked specific questions about the social environment of the child, as shown in Figure 2.16. 0.4 14.2 18.5 18.1 24.0 28.0 25.6 42.9 50.4 56.3 76.4 83.5 0.3 11.9 15.3 15.8 19.8 22.9 29.4 35.8 42.4 55.7 68.2 71.2 0 20 40 60 80 100 No questions Happened previously Mucus in stools Abdominal pain Blood in stools Eating and drinking Progression of diarrhea Vomiting Consistency of stools Treatment Frequency of stools Temperature Health center Hospital Baseline Report 71 Figure 2.16. Percentage of health workers who asked about aspects of social history, by facility type Physical Examination More than 50 percent of health workers correctly stated that, when performing the physical examination of a child with diarrhea, they would check for dehydration, observe the child’s affect or demeanor, check for fever, or perform abdominal palpation (Figure 2.17). However, between 9.8 percent to 49.2 percent of health workers did not mention these signs. The other important signs of skin turgor, heart rate, respiratory rate, weight, and capillary refill were missed by the majority of health workers. In general, higher proportions of hospital health workers mentioned that they would check most of the important physical signs than did their HC counterparts. 0.0 0.0 2.0 5.9 9.7 6.7 0.6 0.8 1.9 2.0 4.2 5.5 0 10 20 Religious affiliation Caretaker's occupation Mother's marital status Household composition Housing and WASH conditions Anyone else in the household ill? Health center (n=707) Hospital n=254) 72 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.17. Percentage of health workers indicating specific physical examinations, by facility type Differential Diagnosis Overall, only 52.4 percent of hospital health workers and 41.6 percent of HC health workers correctly indicated dysentery/shigellosis as part of the differential diagnosis. As shown in Figure 2.18, the proportions of correct differential diagnosis were variable across the provinces. The lowest proportions—13.2 percent and 30.2 percent for HC and hospital health workers, respectively—were in Sankuru and Kasai Oriental. The highest proportions of correct differential diagnosis were observed among health workers in Lualaba. 0.0 21.3 22.4 44.5 46.5 38.2 60.2 61.8 72.1 90.2 0.2 11.5 23.3 27.0 27.4 28.2 50.8 61.5 62.8 82.6 0 20 40 60 80 100 No examination Capillary refill Weight Respiratory rate Heart rate Skin turgor Abdominal palpation Temperature Affect/demeanor Dehydration Health center Hospital Baseline Report 73 Figure 2.18. Percentage of health workers who correctly identified bacterial dysentery/shigellosis as a likely diagnosis, by facility type and province Laboratory Tests Among health workers in HCs, 91 percent correctly mentioned direct microscopic examination of stool as the laboratory test to be ordered. However, only 15 percent of them mentioned conducting a stool culture. The proportion of those mentioning both tests was even lower (13.4%). As shown in Table 2.16, there was little variation in recommendations regarding laboratory tests across provinces. Table 2.17 shows that the proportions of hospital-based health workers who mentioned direct microscopic stool examination, stool culture, or both tests were slightly higher. Table 2.16. Percentage of health center-based health workers who ordered the correct tests for bacillary dysentery/shigellosis, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Direct microscopic stool exam 94.3 86.8 88.3 96.5 87.2 89.1 91.0 Stool culture 18.6 30.2 20.0 0.9 10.6 21.7 15.0 Both tests 17.0 24.5 15.8 0.9 10.6 21.7 13.4 n 194 53 120 114 180 46 707 50.0 52.8 32.5 13.2 46.1 69.6 41.6 71.2 68.2 30.2 41.0 46.8 79.0 52.4 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health center (n=707) Hospital (n=254) 74 The Impact of USAID’s Integrated Health Program in the DRC Table 2.17. Percentage of hospital-based health workers who ordered the correct tests for bacillary dysentery/shigellosis, by province Eastern Congo Kasai Katanga Total Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Direct microscopic stool exam 96.6 100.0 90.6 82.1 96.8 84.2 92.5 Stool culture 47.5 27.3 39.6 30.8 19.4 36.8 33.9 Both tests 45.8 27.3 30.2 18.0 19.4 26.3 28.7 n 59 22 53 39 62 19 254 Treatment Overall, fewer than one-half of the HC-based health workers correctly mentioned fever treatment as part of the management of the sick child, but three-quarters of them correctly mentioned fluids for dehydration and metronidazole for Shigella dysentery. Across provinces, HC-based health workers in Tanganyika had the lowest proportions of correct treatment for fever and dehydration (26.4% and 32.1%, respectively), whereas those in Haut Katanga had the lowest proportion (70%) of correct treatment for Shigella dysentery. HC health workers in Sankuru had the highest proportions of correct treatment for dehydration (82.5%) and Shigella dysentery (82.5%) and those in Lualaba had the highest proportion for correct treatment of fever (60.9%), as shown in Figure 2.19. By comparison, half of the hospital-based health workers correctly mentioned fever treatment as part of the management of the sick child, and 78.7 percent of them correctly mentioned fluids for dehydration. Metronidazole was mentioned by 70.1 percent of the hospital-based health workers for the treatment of Shigella dysentery. Across provinces, hospital-based health workers in Tanganyika had the lowest proportions of correct treatment for fever and dehydration (40.9% and 63.6%, respectively), whereas those in Sankuru had the lowest proportion of correct treatment for Shigella dysentery (46.2%). Hospital-based health workers in Lualaba had the highest proportions of correct treatment for fever (57.9%) and dehydration (94.7%). Hospital￾based health workers in Kasai Oriental had the highest proportion of correct treatment for Shigella dysentery (77.4%) (Figure 2.20). Baseline Report 75 Figure 2.19. Percentage of health center-based health workers who correctly mentioned metronidazole, fluids, or treatment for fever, by province (n=707) Figure 2.20. Percentage of hospital-based health workers who correctly mentioned metronidazole, fluids, or treatment for fever, by province (n=254) If we combine diagnosis and treatment, as shown in Figure 2.21, we see a drop in the overall proportion of HC-based and hospital-based health workers who correctly mentioned direct microscopy and metronidazole, respectively, for the diagnosis and treatment of Shigella dysentery. This trend was also observed across the provinces. 70.1 71.7 80.8 82.5 70.0 71.7 74.1 81.4 32.1 73.3 82.5 76.7 78.3 75.1 43.8 26.4 58.3 42.1 50.0 60.9 47.4 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Metronidazole (Flagyl) Fluids (oral or IV) Treatment for fever 71.2 77.3 77.4 46.2 77.4 63.2 70.1 83.1 63.6 75.5 92.3 69.4 94.7 78.7 55.9 40.9 45.3 51.3 50.0 57.9 50.4 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Metronidazole (Flagyl) Fluids (oral or IV) Treatment for fever 76 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.21. Percentage of health workers who gave the correct diagnosis test (direct stool microscopy), and treatment (metronidazole) for bacterial dysentery/shigellosis, by facility type and province Counseling Figure 2.22 shows that hygiene and feeding emerged as the most frequently mentioned topics for counseling for both HC-based and hospital-based health workers. The other important topics for counseling, such as rehydration, how to take medicines, and mode of transmission, were mentioned less frequently. 27.7 32.7 19.5 5.7 29.8 5.7 22.8 9.1 25.0 17.5 6.9 27.4 0.0 17.5 0 20 40 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health center (n=707) Hospital (n=254) Baseline Report 77 Figure 2.22. Percentage of health workers who mentioned specific messages for counseling, by facility type Clinical Vignette 2: ANC A total of 804 healthcare workers responded to a vignette presenting a 19-year-old woman making her first ANC visit. The woman was visibly pregnant and she estimated that she was at least 20 weeks into her pregnancy. She had not taken a pregnancy test and did not come for an ANC visit earlier because she lived far from the HC. Table 2.18 presents the distribution of health workers who responded to this vignette by province and facility type. The majority of health workers (78%) were based in HCs. Table 2.18. Total number of health workers who responded to the ANC vignette, by facility type and province Facility type Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Health center 166 52 118 88 168 35 627 Hospital 22 12 40 29 62 12 177 n 188 64 158 117 230 47 804 The vignette first asked the health workers what questions they would ask the woman before proceeding with a physical examination. Subsequently, the vignette gave the results of the physical examination and asked the health worker to order the most appropriate laboratory tests based on the physical examination results. The vignette then provided the results of the laboratory tests. Using the results of the physical examination and laboratory tests, the health worker was requested to assess the condition of the woman, prescribe a treatment, and propose counseling messages. 0.4 5.5 16.1 15.0 17.3 22.8 36.2 60.6 92.9 0.3 3.4 11.3 12.7 13.7 20.4 35.6 58.3 90.7 0 20 40 60 80 100 No counseling Diagnosis How to monitor When to follow up How transmitted How to take medicine Hydration Feeding Hygiene Health center (n=707) Hospital (n=254) 78 The Impact of USAID’s Integrated Health Program in the DRC Questions Asked by Health Workers about the Patient Most of the health workers (95%) asked a question about the timing of the last menstrual period, an important factor in confirming the duration of the pregnancy. However, fewer than one-half of them asked other important questions about her reproductive history, such as the number of pregnancies, live births, and miscarriages she had experienced and her disease history (Figure 2.23). Figure 2.23. Percentage of health workers who asked questions related to health history, by facility type Furthermore, small proportions of health workers, ranging from 0.6 percent to 30.5 percent, asked questions about the patient’s social history, as shown in Figure 2.24. The social history provides important context that informs a patient’s diagnosis and treatment. 0.0 9.6 11.3 10.2 11.9 24.9 37.9 41.2 42.4 39.6 44.6 94.9 0.3 6.9 8.3 9.3 10.5 16.9 31.9 34.9 38.0 38.6 42.1 95.2 0 20 40 60 80 100 No questions History of diabetes Family health history Past illnesses History of hypertension Number of children who have died Number of miscarriages Number of children alive Number of live births Complaints during this pregnancy Number of pregnancies Timing of last menstrual period Health center (n=627) Hospital (n=177) Baseline Report 79 Figure 2.24. Percentage of health workers who asked questions related to social history, by facility type Physical Examination Figure 2.25 shows that more than 50 percent of health workers—with percentages ranging from 52.2 percent to 81.9 percent—correctly indicated that they should examine the fundal height, blood pressure, and fetal heartbeat, and check for the presence of edema. Fewer than one-half of the health workers mentioned other key elements of a physical examination, such as measuring the woman’s weight, height, and body temperature, checking for abdominal palpation, and carrying out a breast examination. There were negligible differences across facility types. 0.6 0.6 2.8 2.3 11.9 12.4 30.5 0.7 1.0 1.1 4.0 7.8 9.6 28.9 0 10 20 30 40 Support network Religious affiliation History of sexual/intimate partner violence Education level Sexual history Occupation Marital status Health center (n=627) Hospital (n=177) 80 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.25. Percentage of health workers who indicated that they would perform various physical examinations, by facility type Laboratory Tests A higher proportion of hospital-based health workers than HC-based workers correctly indicated that they should order proteinuria, hemoglobin, and blood grouping tests. Nevertheless, among hospital-based health worker respondents, many failed to mention these tests: 31.6 percent (for proteinuria), 38.4 percent (for hemoglobin), and 68.9 percent (for blood grouping). In addition, more than one-half of both hospital-based and HC-based health workers failed to mention other key routine tests, such as tests for HIV, pregnancy, and syphilis, as shown in Figure 2.26. 0.0 23.2 28.8 31.1 31.1 41.8 48.0 45.8 63.8 63.8 80.2 81.9 0.2 13.9 17.9 20.6 33.5 35.1 48.5 51.2 52.2 68.4 69.5 79.1 0 10 20 30 40 50 60 70 80 90 100 No examination Breast exam Pulse Respiratory rate Height Temperature Abdominal palpation Weight Presence of edema Fetoscope Blood pressure Fundal height Health center (n=627) Hospital (n=177) Baseline Report 81 Figure 2.26. Percentage of health workers who mentioned various diagnostic tests, by facility type Assessment of Patient’s Condition Tables 2.19 and 2.20 show the percentage of HC-based and hospital-based health workers’ assessments of the patient’s condition. Overall, a higher percentage of hospital-based health workers (83.6%) correctly assessed the patient’s condition as preeclampsia compared with 64.8 percent among their HC-based counterparts. Across the provinces, Sud Kivu scored highest (76.2%) among HC-based health workers, with Tanganyika scoring lowest (50%). Among hospital-based health workers, those in Kasai Oriental scored the highest (100%) and Tanganyika had the lowest score (70%) in correctly assessing the patient’s condition as preeclampsia. Table 2.19. Percentage of health center-based health workers who gave various assessments, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Preeclampsia 76.2 50.0 57.1 75.9 50.0 55.8 64.8 Healthy pregnancy 8.3 18.6 2.9 12.1 22.7 21.2 14.0 Other/don't know 15.5 31.4 40.0 12.1 27.3 23.1 21.2 n 166 52 118 88 168 35 627 0.6 11.3 31.1 26.0 33.3 41.8 61.6 68.4 55.9 3.0 4.9 16.9 19.1 42.4 45.5 47.5 52.3 56.9 0 10 20 30 40 50 60 70 80 90 100 No tests Hematocrit Blood grouping Syphilis Pregnancy test HIV Hemoglobin Proteinuria Malaria Health center (n=627) Hospital (n=177) 82 The Impact of USAID’s Integrated Health Program in the DRC Table 2.20. Percentage of hospital-based health workers who gave various assessments, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Preeclampsia 87.1 70.0 100.0 81.8 89.7 83.3 83.6 Healthy pregnancy 6.5 15.0 0.0 9.1 10.3 8.3 9.0 Other/don't know 6.5 15.0 0.0 9.1 0.0 8.3 7.3 n 22 12 40 29 62 12 177 Treatment Overall, fewer than one-half of the HC-based health workers (44.7%) indicated that they would refer a preeclampsia case to the hospital (which the DRC MOH guidelines recommend) (Table 2.21). Across the provinces, Sankuru had the highest percentage of referrals to a hospital (75.3%) and Sud Kivu had the lowest percentage (28%). Among hospital-based health workers, the administration of hypotensive drugs was indicated by 74 percent of the respondents (Table 2.22). Provincial comparisons showed that Sud Kivu scored the highest (87.1%) on hypotensive treatment and Haut Katanga scored the lowest (58.6%). Table 2.21. Percentage of health center-based health workers who recommended various treatment options, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Hypotensive drug 62.5 44.1 22.9 28.9 23.9 59.6 42.3 Transfer to hospital 28.0 34.8 57.1 75.3 31.8 36.5 44.7 Anticonvulsive drug 10.7 11.0 0.0 1.8 15.9 7.7 8.3 Induction of labor 1.2 2.0 0.0 0.6 1.2 0.0 1.0 No treatment 4.2 5.9 22.9 4.8 5.7 19.2 7.2 n 166 52 118 88 168 35 627 Table 2.22. Percentage of hospital-based health workers who recommended various treatment options, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Hypotensive drug 87.1 60.0 83.3 77.3 58.6 75.0 74.0 Transfer to higher level of care 9.7 7.5 16.7 22.7 6.9 0.0 10.2 Anticonvulsive drug 12.9 7.5 33.3 13.6 3.5 8.3 11.3 Induction of labor 3.2 0.0 0.0 0.0 0.0 0.0 1.2 No treatment 0.0 10.0 0.0 0.0 0.0 8.3 2.8 n 22 12 40 29 62 12 177 Baseline Report 83 Counseling Figure 2.27 shows that the counseling most frequently given was the reduction of salt intake, which was mentioned by 56.6 percent of HC-based workers and 68.4 percent of hospital-based health workers—a treatment that is critical for women with preeclampsia. We noted that only 34 percent of HC-based health workers mentioned referral to a hospital among their counseling topics. Other important counseling topics, such as the reduction of physical activity, increasing water intake, taking medications correctly, and watching for danger signs signaling the need for emergency care, were mentioned by fewer than 25 percent of both HC￾based and hospital-based health workers. Figure 2.27. Percentage of health workers who gave various recommendations, by facility type Clinical Vignette 3: Family Planning A total of 595 healthcare workers responded to a vignette presenting a 22-year-old woman who visited a clinic because she was interested in using contraceptives. The majority of respondents (70%) were HC-based health workers. The vignette first asked the health workers what questions they would ask the woman. The vignette then described the obstetrical history of the woman and provided some information on her marital status, parity, and sexual activity; notably, that she was married, had no children, and wanted to delay having children for at least three years. After that, the vignette asked whether the health workers would counsel the woman in choosing a contraceptive method and what information they would provide when counseling the woman about the FP method she was using. The vignette also asked the health workers how they would determine which FP method to recommend and what factors they would consider when making that decision. After counseling and discussion, the woman indicated her choice to use the implant method. However, this method was not currently being offered in the clinic. The vignette subsequently asked the health workers what 0.6 11.3 9.6 8.5 23.2 23.7 32.8 24.3 5.7 50.9 68.4 1.8 4.5 7.3 9.6 12.6 17.5 21.9 23.6 34.0 40.7 56.6 0 10 20 30 40 50 60 70 80 90 100 No counseling Signs that emergency care is needed Causes, symptoms, risks Increase protein intake How to take medicine Increase water intake When to return for follow-up Reduced physical activity Referral to higher level of care Bed rest Minimize salt intake Health center (n=627) Hospital (n=177) 84 The Impact of USAID’s Integrated Health Program in the DRC they would do in that event. The woman emphasized that she did not want anyone to know that she was using contraception and asked that it be kept confidential. Last, the vignette asked how the health workers would respond to her concerns about confidentiality. Table 2.23 shows the distribution of the participants, by facility type and province. Table 2.23. Total number of health workers who responded to the FP vignette, by facility type and province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Health center 141 22 39 77 111 27 417 Hospital 52 7 28 25 56 10 178 n 193 29 67 102 167 37 595 Questions the Health Workers Asked the Woman As shown in Figure 2.28, fewer than one-half of both HC-based and hospital-based health workers stated that they would ask specific important questions about the medical history of the woman, such as information on her menstrual cycle, obstetrical and gynecological history, medications, FP methods used, recent intercourse, and other health conditions. The proportions of health workers reporting these questions ranged from 7.4 percent to 38.9 percent. Figure 2.28. Percentage of health workers who mentioned questions about aspects of medical history, by facility type Other questions asked were related to the social history of the client (Figure 2.29). Among those, the most frequently mentioned question was about the marital status of the client. However, both HC-based and hospital-based health workers demonstrated little interest in other important social information, such as contraceptive preference, length of marriage, pregnancy intentions, and sexual history. 0.6 9.0 11.2 31.5 41.6 38.2 1.0 7.4 15.1 21.6 33.6 38.9 0 10 20 30 40 50 60 70 80 90 100 No questions Other health conditions Recent intercourse Medication/FP history Ob/Gyn history Menstrual history Health center (n=417) Hospital (n=178) Baseline Report 85 Figure 2.29. Percentage of health workers who mentioned questions about aspects of social history, by facility type Counseling on Choosing a Contraceptive Method Overall, fewer than one-half of both HC-based and hospital-based health workers responded that they would counsel the woman on choosing a contraceptive method. Figure 2.30 shows that the highest percentages of both HC-based and hospital-based health workers willing to provide counseling were in Tanganyika (86.4% and 85.7%, respectively). The lowest percentages of health workers willing to provide counseling were found in Lualaba among HC-based health workers (29.6%) and in Kasai Oriental among hospital-based health workers (28.6%). 1.2 0.6 5.6 10.1 11.2 6.2 12.4 9.6 28.1 73.6 1.0 2.0 6.0 6.2 8.2 8.6 9.4 10.1 26.6 68.6 0 10 20 30 40 50 60 70 80 90 100 History of S/IPV Religious affiliation Education level Occupation (husband) Occupation (self) Sexual history Pregnancy intentions Length of marriage Contraceptive preferences Marital status Health center (n=417) Hospital (n=178) 86 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.30. Percentage of health workers who would counsel the woman in choosing a contraceptive method, by facility type and province Among both HC-based and hospital-based health workers, the most frequently mentioned reasons for not providing counseling on contraceptive methods were “no or not enough children” (64.3% and 81.1%, respectively) and “husband not present” (54.9% and 53.3%, respectively). The other commonly reported reason was “she is married” (36.2% of HC-based and 33.3% of hospital-based health workers) (Figure 2.31). Figure 2.31. Among health workers who indicated that they would not offer a contraceptive method, reasons given, by facility type 50.4 86.4 41.0 45.5 39.6 29.6 46.3 52.9 85.7 28.6 48.0 48.2 70.0 49.2 0 20 40 60 80 100 Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Eastern Congo Kasai Katanga Overall Health center (n=417) Hospital (n=178) 2.3 3.3 2.3 33.3 53.3 81.1 0.5 1.8 5.1 36.2 54.9 64.3 0 20 40 60 80 100 Provider's religious beliefs Contraindicated Condoms are sufficient She is married Her husband is not present She has no/not enough children Health center (n=224) Hospital (n=90) Baseline Report 87 Topics for Counseling about Family Planning Side effects and the availability of FP methods on the day of visits were the most frequently mentioned counseling topics by both HC-based and hospital-based health workers, with percentages ranging from 63.2 percent to 72.4 percent (Figure 2.32). Other important topics, such as effectiveness, duration of protection, correct use, safety, and discomfort during use of the contraceptive method, were mentioned by fewer than one￾half of the health workers. Figure 2.32. Among health workers who stated that they would offer a contraceptive method, counseling given, by facility type Criteria for Determining Which Family Planning Method to Prescribe Figure 2.33 shows the criteria mentioned by health workers for determining the method to prescribe. Of these, side effects and effectiveness of the method emerged as the most frequently mentioned criteria (percentages ranging from 47.1% to 54.9%). It is worth mentioning that the patient’s preference was mentioned by 36.8 percent and 37.9 percent of HC-based and hospital-based health workers, respectively. 0.0 9.9 9.9 7.4 18.4 19.5 33.3 20.7 32.2 40.2 72.4 65.5 0.0 6.6 7.1 7.7 15.5 22.3 29.5 29.5 39.9 43.5 63.2 64.3 0 20 40 60 80 100 No counseling Cost Provider's recommendation Discomfort during administration Safety Effectiveness in preventing STI's Methods available consistently Correct use Duration of protection Effectiveness in preventing pregnancy Methods available today Side effects Health center (n=193) Hospital (n=87) 88 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.33. Among health workers who stated that they would offer a contraceptive method, criteria for determining the method to prescribe, by facility type What Health Workers Would Do Were the Preferred Method Unavailable If the contraceptive method preferred by the client was not available at the clinic, the HC-based health workers indicated three options: (1) referring the client to another clinic (71.5%); (2) asking the client to choose another method (41.5%); or (3) ending the consultation (2.2%). The hospital-based health workers indicated two options: (1) referring the client to another clinic (54%); and (2) asking the client to choose another method (60.9%) (Figure 2.34). 1.2 5.8 13.8 20.7 19.5 21.8 29.9 25.3 37.9 47.1 50.6 4.9 7.8 13.5 14.5 22.3 22.3 26.4 26.9 36.8 48.2 54.9 0 20 40 60 80 Competence of providers to administer Peers' acceptance Cost of method Preferences regarding pregnancy/spacing Patient's confidence in correct use Husband's acceptance Patient's medical history Availability of method Patient's preferred method Effectiveness Side effects Health center (n=193) Hospital (n=87) Baseline Report 89 Figure 2.34. Among health workers who stated that they would offer a contraceptive method, response to the preferred method being unavailable, by facility type Confidentiality The majority of health workers—more than 80 percent—stated that they would respect the request for confidentiality from the client when offering a contraceptive method. Moreover, one-third of them (38.9% of HC-based and 32.2% of hospital-based health workers) would encourage the client to inform her husband about her use of the contraceptive method. It is worth noting that 5.2 percent of HC-based and 4.6 percent of hospital-based health workers would refuse to provide a contraceptive method if the client did not inform her husband about her use of the contraceptive method (Figure 2.35). Figure 2.35. Among health workers who stated that they would offer a contraceptive method and their response to a request for confidentiality, by facility type Affordability of Services Transparency in fee schedules enables patients to make informed decisions about their health services. Figure 2.36 shows the percentage of HCs that had their standard fee schedule posted for patients to see. It also shows whether a facility had a reduced fee schedule for indigent patients and whether that fee schedule was posted. 0.0 60.9 54.0 2.2 41.5 71.5 0 10 20 30 40 50 60 70 80 90 100 End consultation Choose another method Refer elsewhere Health center (n=193) Hospital (n=87) 1.2 4.6 32.2 81.6 2.2 5.2 38.9 82.4 0 20 40 60 80 100 Encourage her to tell other people Refuse to provide until she informs husband Encourage her to tell her husband Reassure that care is confidential Health center (n=193) Hospital (n=87) 90 The Impact of USAID’s Integrated Health Program in the DRC Overall, 64 percent of HCs had a posted general fee schedule. This ranged from 50.6 percent in Haut Katanga to 86.1 percent in Sud Kivu. Posted indigent fee schedules were less common. In some provinces, this was because few facilities had an indigent fee schedule (e.g., Sankuru), whereas others had fee schedules but did not post them (e.g., Tanganyika). Figure 2.36. Percentage of health centers that posted a standard fee schedule, had an indigent fee schedule, and posted an indigent fee schedule, by province There was a higher prevalence of posted fee schedules for general fees (74.1%) and indigent fees (50.9%) in hospitals (Figure 2.37). Lualaba was the most transparent, with 91.7 percent of facilities posting a general fee schedule, and 91.7 percent posting an indigent fee schedule (representing 100% of those that had an indigent fee schedule). 86.1 51.6 53.2 54.6 50.6 78.4 64.0 61.6 54.8 31.9 13.6 44.6 64.9 46.3 54.7 16.1 12.8 9.1 27.7 54.1 32.0 0 20 40 60 80 100 Sud Kivu (n=86) Tanganyika (n=30) Kasai Oriental (n=47) Sankuru (n=45) Haut Katanga (n=83) Lualaba (n=37) Eastern Congo Kasai Katanga Overall (n=328) Standard fee schedule is posted Facility has indigent fee schedule Indigent fee schedule is posted Baseline Report 91 Figure 2.37. Percentage of hospitals that posted a standard fee schedule, had an indigent fee schedule, and posted an indigent fee schedule, by province Allowing for payment after treatment for emergencies and for labor and delivery enables patients to receive care in a timely matter, rather than delaying until user fees can be paid. Nearly 93 percent of HCs treated emergency cases before payment, and 90.9 percent accepted payment after labor and delivery (Figure 2.38). These rates were high in most provinces, with the exception of Sankuru, where no more than 75 percent of facilities accepted payment after emergencies or labor and delivery. 89.7 70.0 75.0 62.5 56.0 91.7 74.1 55.2 30.0 31.3 31.3 60.0 91.7 51.7 50.9 10.0 18.8 18.8 40.0 91.7 39.8 0 20 40 60 80 100 Sud Kivu (n=29) Tanganyika (n=10) Kasai Oriental (n=16) Sankuru (n=16) Haut Katanga (n=25) Lualaba (n=12) Eastern Congo Kasai Katanga Overall (n=108) Standard fee schedule is posted Facility has indigent fee schedule Indigent fee schedule is posted 92 The Impact of USAID’s Integrated Health Program in the DRC Figure 2.38. Percentage of health centers that accepted payment after treatment of emergencies and labor and delivery, by province The majority of hospitals also treated emergency (94.4%) and labor and delivery cases (97.2%) before demanding payment (Figure 2.39). Again, rates were high in most provinces, with Sankuru a low outlier. Figure 2.39. Percentage of hospitals that accepted payment after treatment of emergencies and labor and delivery, by province Health facility directors were asked to state the ways in which they would respond to a case in which a patient could not pay the fees that he or she owed. Table 2.24 shows the responses from HC directors. The most common reaction was to treat the patient as indigent, offering them free or reduced-cost services (59.2%). HCs also commonly allowed patients to give a guarantee, such as a signed promise to pay or an item held as 98.8 96.8 89.4 70.5 97.6 94.6 92.7 90.7 100.0 95.7 75.0 90.4 97.3 90.9 0 20 40 60 80 100 Sud Kivu (n=86) Tanganyika (n=30) Kasai Oriental (n=47) Sankuru (n=45) Haut Katanga (n=83) Lualaba (n=37) Eastern Congo Kasai Katanga Overall (n=328) Allow payment after treatment for emergencies Allow payment after labor and delivery 96.6 90.0 93.8 81.3 100 100 94.4 100 100 100 81.3 100 100 97.2 0 20 40 60 80 100 Sud Kivu (n=29) Tanganyika (n=10) Kasai Oriental (n=16) Sankuru (n=16) Haut Katanga (n=25) Lualaba (n=12) Eastern Congo Kasai Katanga Overall (n=108) Allow payment after treatment for emergencies Allow payment after labor and delivery Baseline Report 93 collateral (46.7%), or allowed patients to pay in-kind or work off the debt at the facility (42.4%). In some instances, the case was referred to the CODESA for collection (13.7%) or the patient was detained at the facility until he or she could pay (11%). Only a small percentage of the HCs refused to give services (0.3%) or future services (2.4%) to patients who could not pay. Provinces varied considerably in the prevalence of strategies used. Table 2.24. Health centers’ responses to patients who could not pay fees owed, by province Eastern Congo Kasai Katanga Overall Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Free/reduced cost, considered indigent 50.0 48.4 61.7 50.0 73.5 64.9 59.2 Patient can give a guarantee 55.8 64.5 40.4 20.5 43.4 56.8 46.7 Patient can pay in-kind or work off debt 34.9 64.5 21.3 86.4 28.9 46.0 42.4 Owes/pays later/referred to CODESA 19.8 0.0 19.2 25.0 3.6 13.5 13.7 Not discharged until can pay 8.1 0.0 6.4 34.1 10.8 5.4 11.0 No recourse 3.5 3.2 4.3 2.3 6.0 2.7 4.0 Refused services in future 1.2 12.9 0.0 6.8 0.0 0.0 2.4 No services are given 0.0 0.0 0.0 2.3 0.0 0.0 0.3 n 86 30 47 45 83 37 328 Table 2.25 summarizes the responses from hospital directors. Hospital directors’ most common responses to patients who could not pay were to consider them indigent (68.5%) and to allow them to give a guarantee (41.7%). Compared with HCs, hospitals were less likely to allow patients to pay in-kind or work off debts (25%), and were more likely to detain a patient until they paid (18.5%). Again, there was considerable variation among the provinces. Table 2.25. Hospitals’ responses to patients who could not pay fees owed, by province Eastern Congo Kasai Katanga Sud Overall Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Free/reduced cost, considered indigent 48.3 80.0 62.5 62.5 84.0 91.7 68.5 Patient can give a guarantee 51.7 60.0 31.3 12.5 40.0 58.3 41.7 Patient can pay in-kind or work off debt 27.6 20.0 18.8 56.3 4.0 33.3 25.0 Owes/pays later/referred to CODESA 10.3 10.0 25.0 31.3 4.0 0.0 13.0 Not discharged until can pay 31.0 20.0 0.0 37.5 12.0 0.0 18.5 No recourse 0.0 0.0 6.3 0.0 8.0 0.0 2.8 94 The Impact of USAID’s Integrated Health Program in the DRC Refused services in future 0.0 10.0 0.0 0.0 4.0 0.0 1.9 No services are given 0.0 0.0 0.0 0.0 0.0 0.0 0.0 n 29 10 16 16 25 12 108 Health Behaviors Data elements identified as both outcomes and non-outcomes for the impact evaluation were included as covariates in the regression model used to generate propensity-score weights. These weights were used to identify health facilities that did not receive the HSS intervention but that were similar in nature to health facilities that did receive the intervention. These identified health facilities will serve as the comparator or control group. The best comparison group is the one with the same characteristics as the intervention group, and the propensity-score weights help identify health facilities to include in this comparison group. The 12 outcome data elements used to establish propensity-score weights are presented in Table 3.1, which shows both the unweighted and weighted mean and standard deviation stratified by control/treatment site. An absolute standardized mean difference value, represented in Table 3.1 as Cohen’s d, which is less than 0.1, indicates attainment of a distributional balance of the DHIS2 data elements between the control and treatment sites. Balance of the covariate distribution across groups is important because it allows for more robust causal conclusions (i.e., did the HSS intervention truly have an effect?). Under the weighted scenario below, standardized mean differences for all data elements included as covariates were under 0.1. This indicates that an appropriate group of control sites was identified that do not show an appreciable level of bias with respect to treatment sites, as measured through a common pool of DHIS2 data elements. Table 3.1. Pooled estimates from multiply imputed data Unweighted Weighted DHIS2 data elements (transformed to rates) Control sites mean (Standard deviation) Treatment sites mean (Standard deviation) Std. diff. (Cohen's d) Control sites mean (Standard deviation) Treatment sites mean (Standard deviation) Std. diff. (Cohen's d) Rate of live births per 1,000 population of women of reproductive age 5.73 (5.9) 6.20 (5.85) 0.080 5.87 (5.65) 6.20 (5.85) 0.057 Rate of live births <2,500g per 1,000 population of women of reproductive age 0.27 (0.77) 0.25 (0.78) 0.026 0.23 (0.65) 0.25 (0.78) 0.028 Rate of moderate acute malnutrition per 1,000 population of children ages 6–59 months 1.87 (4.48) 3.97 (9.36) 0.286 3.30 (6.38) 3.97 (9.36) 0.084 Rate of exclusive breastfeeding per 1,000 population of children under 6 months 141.93 (228.36) 138.08 (212.16) 0.018 129.11 (221.34) 138.08 (212.16) 0.041 Rate of attendance at the fourth ANC visit per 1,000 population of women of reproductive age 3.68 (4.46) 4.43 (4.81) 0.162 4.06 (4.55) 4.43 (4.81) 0.079 Rate of measles vaccination per 1,000 population of children ages 6–59 months 6.70 (7.2) 7.31 (7.02) 0.086 6.92 (7.28) 7.31 (7.02) 0.055 Rate of pentavalent vaccination per 1,000 population of children ages 6–59 months 6.71 (7.14) 7.44 (7.18) 0.102 7.05 (7.31) 7.44 (7.18) 0.054 Rate of new acceptors of modern contraceptive methods per 1,000 population of women of reproductive age 3.48 (6.79) 4.61 (13.81) 0.104 4.18 (7.01) 4.61 (13.81) 0.039 Rate of insecticide-treated net distribution during ANC visits per 1,000 population of women of reproductive age 5.36 (5.67) 4.51 (5.09) 0.158 4.40 (4.95) 4.51 (5.09) 0.022 Rate of severe malaria treatment per 1,000 population of children ages 6–59 months 1.06 (2.75) 1.06 (2.61) 0.000 0.96 (2.33) 1.06 (2.61) 0.040 Rate of severe pneumonia treatment per 1,000 population of children ages 6–59 months 0.36 (1.76) 0.39 (1.24) 0.020 0.34 (1.82) 0.39 (1.24) 0.032 Rate of severe diarrhea/dehydration treatment per 1,000 population of children ages 6–59 months 0.10 (0.39) 0.12 (0.49) 0.045 0.10 (0.38) 0.12 (0.49) 0.046 96 The Impact of USAID’s Integrated Health Program in the DRC Abt Associates conducted a baseline household survey in 2019 in all provinces supported by the project. Data from women of reproductive age (15‒49 years) pertaining to the care-seeking experience and level of participation in health services were tabulated. Table 3.2 shows the percentage of patients who reported experiencing long wait times at public facilities during their most recent visit. Overall, the largest percentage of patients reported long wait times for ANC, with 35.2 percent of those who sought it reporting that they waited a long time. More than one in four patients who sought RECO services also reported long wait times. Long wait times tended to be reported most often in Sankuru. Table 3.2. Percentage of women who sought care in a public facility who reported long wait times, by service and province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Overall ANC 29.6 41.0 13.0 70.2 30.7 19.1 35.2 RECO services 37.9 47.2 6.2 76.0 3.5 18.5 26.8 Delivery 22.9 24.8 10.3 29.8 7.8 18.6 15.4 Child vaccination 17.9 26.9 7.7 34.9 8.0 16.1 15.3 Treatment of child illness 16.0 23.6 11.1 39.1 6.0 11.8 14.8 FP 17.4 10.2 6.2 22.7 7.5 29.8 11.8 n 46 47 126 111 136 119 585 Nearly 65 percent of women who sought ANC reported that the service was unaffordable (Table 3.3). This percentage was highest in Sankuru, at 92.6 percent. More than 30 percent of patients reported that RECO services, treatment of child illness, and delivery were also unaffordable. Table 3.3. Percentage of women who sought care in a public facility who reported that the services listed were unaffordable, by service and province Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Overall ANC 58.0 75.2 54.0 92.6 49.5 46.6 64.5 RECO services 47.7 72.6 38.4 70.6 22.5 17.5 38.3 Treatment of child illness 44.0 63.5 33.7 77.5 17.6 27.7 38.1 Delivery 44.7 45.4 37.7 70.0 20.5 35.2 37.4 Child vaccination 32.8 48.0 20.2 23.7 12.0 18.7 23.4 FP 37.9 22.5 23.9 37.9 12.5 22.4 22.1 n 46 47 126 111 136 119 585 Figure 3.1 shows the percentage of caregivers of children under five who reported that they sought care the last time the child had a fever. Most (82.2%) of the children with a fever received care. This percentage ranged from 78.1 percent in Kasai Oriental to 95.3 percent in Lualaba. Baseline Report 97 Figure 3.1. Percentage of caregivers who sought care the last time their child under five had a fever, by province Among the caregivers who reported that they did not seek care for a child with a fever, the most cited reason was lack of money (Table 3.4). Lack of access to a HC was reported by 14.3 percent of caregivers, and the rest were treated at home, did not feel that care was necessary, or gave another reason. Table 3.4. Percentage distribution of reasons why care was not sought for fever (n=number of cases of fever for whom care was not sought) Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Overall Lack of money 83.5 57.2 71.8 73.8 23.2 62.5 64.7 Treated at home 50.8 6.3 24.2 39.7 19.7 44.7 29.4 Not necessary 12.4 9.9 13.5 0.0 45.0 0.0 16.7 Lack of access to HC 13.9 53.1 16.3 19.1 0.0 25.9 14.3 Other reason 0.0 0.0 2.2 5.4 20.4 24.4 6.0 n 11 8 46 44 15 14 138 To assess the degree to which RECOs were operating in project areas, the percentage of HAs in which at least one women reported using a RECO to care for herself or her children in the past three months was calculated (Figure 3.2). Overall, more than one-half (56.7%) of HAs appeared to have functioning RECOs. RECOs had the highest coverage in Sud Kivu, at 80 percent of HAs, and the lowest in Kasai Oriental and Sankuru, with 35 percent. 84.3% 83.3% 78.1% 81.1% 85.1% 95.3% 82.2% 0% 20% 40% 60% 80% 100% Sud Kivu (n=70) Tanganyika (n=64) Kasai Oriental (n=225) Sankuru (n=224) Haut Katanga (n=97) Lualaba (n=129) Eastern Congo Kasai Katanga Overall (n=809) 98 The Impact of USAID’s Integrated Health Program in the DRC Figure 3.2. Percentage of HAs in which at least one woman reported that she used RECOs for care for herself or her children in the past three months Similarly, the percentage of HAs in which at least one woman reported accessing various services at a community care site such as an iCCM site was calculated (Table 3.5). The most frequently reported services accessed were maternal health counseling, followed by WASH counseling, and child health and nutrition counseling. Table 3.5. Percentage of HAs in which at least one woman reported accessing the following services at a community care site Eastern Congo Kasai Katanga Sud Kivu Tanganyika Overall Kasai Oriental Sankuru Haut Katanga Lualaba Maternal health counseling 60.0 60.0 20.0 35.0 10.0 70.0 42.5 WASH counseling 60.0 65.0 25.0 30.0 20.0 55.0 42.5 Child health and nutrition counseling 55.0 50.0 25.0 35.0 10.0 70.0 40.8 Deworming medication 60.0 40.0 25.0 35.0 35.0 50.0 40.8 Newborn care counseling 55.0 60.0 5.0 35.0 15.0 50.0 36.7 Mosquito net distribution 80.0 30.0 20.0 30.0 10.0 40.0 35.0 Malaria treatment (ACT) 35.0 45.0 20.0 35.0 15.0 45.0 32.5 FP counseling 60.0 25.0 15.0 25.0 25.0 40.0 31.7 Iron/folate distribution 45.0 35.0 15.0 30.0 10.0 50.0 30.8 Rapid diagnostic testing for malaria 25.0 35.0 25.0 35.0 20.0 40.0 30.0 ORS distribution 25.0 25.0 10.0 20.0 10.0 25.0 19.2 FP commodities distribution 30.0 20.0 10.0 15.0 10.0 25.0 18.3 TB directly observed therapy 20.0 5.0 5.0 20.0 10.0 20.0 13.3 n 20 20 20 20 20 20 120 80.0 75.0 35.0 35.0 40.0 75.0 56.7 0 20 40 60 80 100 Sud Kivu (n=20) Tanganyika (n=20) Kasai Oriental (n=20) Sankuru (n=20) Haut Katanga (n=20) Lualaba (n=20) Eastern Congo Kasai Katanga Overall (n=120) Baseline Report 99 Questions assessing women’s civic participation detected low rates (Table 3.6). In the past 12 months, 2.8 percent of women reported participating in a public or consultative meeting at the village level, and participation was lower at all other levels. Table 3.6. Percentage of women who reported that they participated in a public or consultative meeting organized by the administrative/political authority or civil society at various levels in the past 12 months Eastern Congo Kasai Katanga Sud Kivu Tanganyika Kasai Oriental Sankuru Haut Katanga Lualaba Overall Municipality 0.9 0.0 0.6 0.3 0.8 1.4 0.8 Ward 1.8 0.9 0.7 0.4 0.8 2.3 1.2 Chiefship 1.3 0.6 0.1 0.3 0.4 0.7 0.7 Group 0.9 0.8 0.1 0.5 0.6 0.6 0.6 Village 2.9 5.3 3.3 3.1 1.7 3.2 2.8 n 1084 941 1116 1097 1014 1019 6271 Women were asked whether they had participated in a community scorecard meeting in the past 12 months (Figure 3.3). Less than one percent of all women had participated. The highest reported level of participation was in Tanganyika, at 2.0 percent. Figure 3.3. Percentage of women who participated in a community scorecard meeting in the past 12 months Key Informants’ Perspectives This section begins with background information on our key informants, followed by a summary of the health system in the DRC from the perspective of key informants. Following this background, we provide information on USAID IHP and select technical approaches of the project, such as SBC, monitoring and evaluation (M&E), and child health and nutrition activities, as described by key informants working in these technical areas. We 0.8 2.0 0.5 0.5 0.7 1.5 0.8 0.0 0.5 1.0 1.5 2.0 2.5 Sud Kivu (n=1,084) Tanganyika (n=941) Kasai Oriental (n=1,116) Sankuru (n=1,097) Haut Katanga (n=1,014) Lualaba (n=1,019) Eastern Congo Kasai Katanga Overall (n=6,271) 100 The Impact of USAID’s Integrated Health Program in the DRC then present information on government and nongovernmental collaborating institutions and projects, such as the Direction Provincial de la Santé (DPS [provincial directorate of health]), Inspection Provincial de la Santé (IPS [provincial health inspector]), Food for Peace (FFP), USAID/DRC’s Integrated Governance Activity (IGA), and the World Bank, including a description of their work approaches and activities, strengths and weaknesses, and the ways in which these organizations plan to collaborate with USAID IHP, as conveyed by key informants working in these institutions. Background of Informants Key informant interviews were carried out between April 16 and June 12, 2019 in Kinshasa and Kolwezi, the capital of Lualaba province. In Kinshasa, key informant interviews were conducted with MOH representatives (3); USAID IHP senior staff involved in program development, implementation, and M&E (4); USAID IHP partners leading behavior change interventions (1); and USAID staff overseeing USAID IHP (3). Based on preliminary interviews with USAID and USAID IHP staff, we decided to interview representatives of other USAID programs, including FFP (2) and the IGA (1) collaborating on USAID IHP activities. We also interviewed one representative from the World Bank overseeing PBF activities. Although we had planned to interview a staff member from Bluesquare—a USAID IHP partner responsible for the development of the project’s data-sharing platform—we learned that this USAID IHP partner was mostly involved in the mechanics of setting up systems rather than data quality control and assurance. Therefore, we held only an informal session with the Bluesquare representative. In the Lualaba province, we interviewed three government health officials, including a senior provincial health official, a representative from the inspector’s office, and a health information systems expert. We also interviewed an USAID IHP representative overseeing interventions in Lualaba. An informal interview was conducted with a MCZ working in Kolwezi. A total of 21 interviews were conducted with 20 informants. Two informants were interviewed on two occasions and, in one instance, two informants working on the same program were interviewed simultaneously. Thirteen informants were physicians; most also had a Master of Public Health degree. These informants had a wealth of experience in health service delivery and health program development and implementation. Informants from USAID IHP and USAID likewise had vast experience in HSS. Other informants included a statistician, a pharmacist, an agronomist, a nutritionist, an expert in rural development, an expert in M&E, and an expert in international public policy. Only one informant had less than 10 years of work experience; two informants had more than 30 years of experience in health programming, development, and implementation. Description of the DRC Health System Informants consistently described the health system as a pyramid comprising three levels: the periphery or operational level, provincial or intermediary level, and the central level. At the periphery or operational level were the health zones, consisting of a technical team directed by a MCZ, and the health structures, including a reference hospital; HCs led by an infirmière titulaire (IT [head nurse]); and health posts for people living in more distant locations. At the provincial level was the DPS, which was responsible for planning and assisting in the execution of health activities in the province. The office of inspection (Inspection de la Santé) was in charge of audits and controlled health activities and services to ensure that government standards and regulations were followed. The Ministère Provincial de la Santé (provincial ministry of health) was under the leadership of the provincial governor. Its mandate was to support health activities. At the central or national level, 13 departments were under the leadership of the secretary general of health, who oversaw the technical aspects of health service provision and care. The inspector general of health serves was an internal auditor, who oversaw the observance of good governance and management, and adherence to government health standards and norms. The central level was described as the foundation of the health system, where directives, policies, standards, and laws designed to govern healthcare and ensure the implementation of the national health strategy were enforced. There were other structures, such as the Centre de Distribution Régional (CDR [regional distribution center]), which stored and distributed medicines to health structures that were essential to the overall healthcare system. Baseline Report 101 A reform designed to diminish the highly bureaucratic DRC health system was launched in 2006, mandating that the IPS become a separate entity providing independent oversight to ensure that national health standards were followed. As part of the reform, decentralization of the health system was instituted, with an intermediary￾level institution (DPS), to be situated closer to the population, established to give more local autonomy in the management and administration of health services. As of the baseline data collection, the DPS and IPS were located in 26 provinces. Each entity had technical units—the DPS had six internal offices and the IPS had four internal offices—to execute their different mandates and objectives according to the national health strategy. Key informants reported that although the reform was endorsed in 2006, its execution did not occur until 2015. Informants described decentralization of the national health system strategy as being well-conceived and organized, encompassing key elements. We were told that the roles and responsibilities of actors at the various levels, and the mechanisms used to strengthen health systems were clearly described in documents defining national standards; that tools needed to implement the system were available; and that laws related to the provision of healthcare were in place. There was a separate national community strategy, which was also lauded for its design and detail. There was also a recently-developed financial strategy. Several informants claimed that the strong health systems framework and supporting documents attracted international donors. Informants also noted that the country has been successful in instituting decentralization so that DPS and IPS offices were separate entities. There was general agreement that the country had a wealth of highly capable human resources, with strong expertise in the provision of healthcare. Although the national health strategy was lauded, there was universal agreement that the health system was not being implemented as planned. Primary obstacles reported related to underfinancing and mismanagement of resources, which prevented the system from functioning as described in the national health plan, known as the Plan National de Développement Sanitaire (PNDS [national health development plan]). We were told that the Congolese government made minimal financial contributions, with informants reporting that the government contributed from three percent to seven percent of the annual national budget, and that a smaller percentage reached the MOH, with a large proportion of the health budget remaining at the central level. This amount fell far short of the 15 percent of the national annual budget cited as needed for the health system to function as planned. We were told that the national health budget was controlled by the Ministry of Finance and Budget, which maintained complex systems for dispersing money, thus hampering other ministries from accessing the funds due them. Under these complex systems, the MOH did not fully understand its annual budget or have the capacity to know when funds were dispersed, preventing it from accessing funds. Recent changes in the system should allow the MOH to set up its own financial and administrative units to ensure that money was allocated directly to MOH line items. Although this should increase the MOH’s access to funds, inadequate government investment forced the country to continue to rely on international donors, which provided more than 50 percent of the healthcare budget. Moreover, informants reported that fee-based payments for health services accounted for more than 40 percent of healthcare financing. They indicated that the government invested between US$21 to US$22 per person for healthcare services each year, whereas guidelines for low-income countries showed that this figure should be US$80 per year. Informants said that the failure of the government to invest in healthcare impacted national productivity and revenue. Various approaches had been used to strengthen financial support and to decrease individual health costs, including fee-based services, community-based health mutuelles, and PBF. An underfinanced system influences the low use of health services, which informants reported was related to limited geographical access, the low quality of services, the fact that many services were not routinely offered, and the unaffordable costs. Although there was agreement that trained MOH personnel had solid technical skills, the working conditions did not allow them to use their capabilities. Informants stated that the failure of the government to inject adequate resources to provide healthcare personnel with essential equipment and supplies affected health worker motivation. We were also told that only a small percentage of the health workforce received a salary, leading to the creation of parallel health systems. Moreover, paid salaries often went to “phantom” health workers who were no longer working. Lack of pay incentivized health workers to raise health 102 The Impact of USAID’s Integrated Health Program in the DRC service fees, which affected access to healthcare in a context where poverty was rampant. It was not clear how USAID IHP would help address health worker motivation and user fees. Poor governance was cited as another major challenge, with informants saying that the government failed to implement standards, directives, and oversight to allow the health system to function, as spelled out in the national health strategy. One informant said the following: The skills are there. The real problem with the health system is what? It is that people are not paid, how will they work? People stop working in the health system to create other parallel systems where they will be remunerated. The government is the regulator, sets the standards, gives directives. The government must assure healthcare to the population, take charge of healthcare. In order to do that, the government must allocate the budget needed, provide finances so that the health system is solid, effective, and responds to population needs. So, in the government role as regulator they must give the budgetary means. In addition to poor management and limited resources, informants highlighted many other problems. Although the reforms were followed at the central level, informants claimed that many key actors at the provincial level still did not understand the rationale for key changes, such as separating the functions of the DPS and IPS. Some informants suggested that many provincial-level health personnel were reluctant to take charge of the management of the healthcare system, as needed. Others emphasized that corruption and lack of accountability continued to plague the system, undermining progress in decentralization. Several informants mentioned that the role of the provincial MOH was not clear in relation to the DPS. International donors also did not provide support to the provincial MOH, creating conflict between the DPS and provincial MOH. Other problems related to ethnic and political affiliations, which informants said caused MOH personnel to refuse to work together. Informants consistently cited problems with the timeliness of reporting and the completeness of data entry. Generally, data quality was reported to be poor, with many highlighting manual data entry at the HC level and limited Internet access as major obstacles. The medical supply chain was hampered by delays getting supplies through customs, difficulties transporting medical supplies to the CDRs and subsequently to health zones, late and inaccurate estimates of medical supply needs by health personnel, and insufficient capital limiting health workers’ ability to order drugs as needed, all leading to regular stockouts. Poor identification and management of disease epidemics were another major challenge. Integrated Health Program Approach Technical Programs and Cross-Cutting Approaches The USAID IHP uses an integrated approach focusing on six technical programs: malaria, MCH, FP, nutrition, TB, and WASH. Malaria programs will be implemented across all health zones, and MCH and FP activities will be executed in most target zones. The other programs will be implemented according to the availability of resources, ongoing activities being executed by IPs, and local needs for assistance. For example, TB interventions will be concentrated in geographical areas with high TB prevalence, such as Lualaba. WASH activities, which require different inputs and technical expertise, will be implemented in limited geographic areas (eight zones each year), with the goal of improving their cost-effectiveness. We were told that technical approaches will be aligned with the MOH policies and the national health strategy. The fact that USAID technical programs have separate funding streams with specific objectives and budgets influences the degree to which USAID IHP technical programs will be implemented across the project health zones. MCH has the biggest percentage of financial support, followed by malaria and FP. USAID IHP will rely on other USAID projects, such as FFP, to assist with the implementation of other technical areas, such as nutrition and WASH, which receive limited funds through USAID IHP. There are also crosscutting approaches, such as strengthening health systems, the provision of medications, improving information systems, finance, community development, youth, and gender, with separate funding allocated for each approach. Gender will be an important focus of USAID IHP, with informants highlighting that women are often marginalized in the Congolese context, thus affecting their decision-making power, access Baseline Report 103 to healthcare, and health status. Although a gender analysis had been conducted, Abt Associates staff were not completely satisfied with the final report, which was described as lacking detail and contextualized information. USAID IHP was trying to address other crosscutting aspects of healthcare service delivery, such as respect for beneficiaries, transparency and reduction of fraud, and community participation. Because the needs are huge, the project will focus on geographical areas where USAID believes a difference can be made, thus potentially creating discrepancies in quality and consistency of health service delivery. Informants stated that project activities will focus on two corridors located in Katanga and Kasai to increase synergy with other USAID programs and, thereby, improve overall project impact. We were also told that USAID had some priority health zones where it would aim to ensure that a full package of services was available. In health zones, USAID IHP will first concentrate on strengthening higher-performing HCs, with these structures used as models and learning centers for other HCs and health posts in the same zone. Informants reported that community development will attempt to reinforce the national approach composed of CAC and relais communautaire (RECO [community relay]), volunteer health workers involved in community sensitization and outreach. First steps will include the selection and training of RECO, the establishment of iCCM sites, community sensitization on key health issues, and implementation of specific behavior change strategies, such as improved care-seeking at health facilities. Although community activities will engage local associations in mobilization efforts, informants stated that associations were not found everywhere. USAID Involvement and Oversight A contracting officer’s representative (COR) in the USAID health team is in charge of overseeing USAID IHP and ensuring that the contract agreement (objectives and deliverables) is followed. The COR is accountable to the USAID contracts office regarding ongoing project execution. The COR works closely with Abt Associates, providing general oversight and management, following daily project activities, and reviewing all technical and financial reports submitted. The COR leads a team of USAID technical experts who contribute to the project design and are responsible for providing ongoing technical advice and oversight related to their specialties. Although we were told that any decisions to modify the contract were made in consultation with the USAID technical team, only the COR can officially make changes, including those related to finances. The COR is also in charge of coordinating activities with other USAID programs and collaborating partners. Perceptions of the Project In general, informants described USAID IHP as a complex project comprising many interventions and actors, with several informants reporting that it was overly ambitious in relation to the challenges in the field. Given the geographical size and diversity of the project, USAID IHP will partner with other organizations to piece together a cohesive approach that covers all six technical programs in the nine target provinces. Informants reported that program modifications will be made based on the availability of other donors to fill needs. Informants emphasized that good coordination (to avoid duplication of activities and resources and to ensure complementarity) will be critical to project success. Informants recognized that maintaining high quality when working with numerous partners will be challenging, requiring supervision to verify that quality standards were met, as defined in the USAID IHP protocol. Although the project appeared to be well-financed, with funding at around US$314 million, informants expressed concern that resources would not adequately cover 178 zones and more than 3,000 HCs. Informants stressed that many project objectives—such as strengthening government leadership capabilities; technical knowledge; and organizational, planning, and decision-making skills—were far-reaching and ambitious. However, USAID staff emphasized that these were critical steps to ensure that the government took ownership of and implemented the national health strategy, as planned. At the provincial level, informants stressed that the DPS capacity must be strengthened to support health zones, as needed, and that health zones, in turn, must have the capabilities to provide adequate oversight and assistance to hospitals and HCs. One informant said: 104 The Impact of USAID’s Integrated Health Program in the DRC It is not easy. We hope the conditions [in the health system] will end, that it will change one day. It does not have to continue like this, but as the saying goes, it's a long journey. But the journey is essential. It took time to build the national system, the national health system, now it's time to know how we can make it work. How to take ownership of the system. We had debts, that's true, but we must take ownership, and with that, we hope that things will go well. For example, the gentleman you met at the health government office [someone with whom the research team had been unimpressed], if there was any ownership, I do not think we would hire such people. Work Completed at the Time of Interviews During a workshop in the fall 2018 attended by MOH officials, former and current USAID IHP staff shared thematic strategies that PROSANI Plus had executed—as the predecessor project to USAID IHP—and discussed approaches to be implemented under USAID IHP. In the last quarter of 2018, USAID IHP staff invited all DPS directors located in target provinces to meet in Kinshasa to develop annual workplans (operational action plan, plan d’action opérationnel [PAO]), which were approved in early December 2018. The workplans included objectives and activities based on the PNDS strategy, plus cross-cutting approaches, such as those related to gender and community development. Start-up action plans were also developed. At the time of our study, PICAL assessments were underway to examine how the DPS system functioned in regard to human resources, financial resources, and specific outputs, with action plans developed to address functional problems. We were told that activities were gradually being implemented at the provincial and zonal levels, including planning and review meetings at the provincial level, training of health workers serving beneficiaries, supervision of health workers, and community sensitization efforts. The supply chain strategy was still being finalized, and there was acknowledgement that the project was experiencing problems getting medications to the CDRs, which was under the mandate of Chemonics. We were told that the distribution of other materials and supplies was slow. USAID IHP did not have a website, which we were told was being developed. Social and Behavioral Change Key Partners and Approaches SBC efforts are led by an advisor from Pathfinder, who had been involved in the predecessor project. Activities focus on Objective 3 of the project, which promotes the adoption of healthy behaviors. The SBC team was working with Matchboxology, a South African organization, and Breakthrough Action, based at Johns Hopkins University in the United States, to develop communication strategies. Matchboxology concentrates its efforts on healthcare workers and Breakthrough Action will focus on community-based activities. We were told that the SBC approach will use a human-centered design (called HCD) approach to develop behavior change strategies. Respondents described the current health system as highly medicalized and dominated by health professionals, with little involvement by community members and minimal appreciation of health problems. Aligned with the national community strategy to empower communities to take greater charge of healthcare, a primary component of SBC activities involves communication efforts aimed to encourage community engagement and oversight of health services. At the provincial level, the USAID IHP’s SBC team works with DPS personnel in the communication sector. Respondents reported that DPS communication staff have little authority, which impeded the timely implementation of communication activities. At the zonal level, the SBC team will work closely with the animateur communautaire (AC [community animator]), who will be involved in supervising community activities. We were told that during PROSANI Plus, supervision by the AC was inadequate and that another supervision strategy may need to be developed. Baseline Report 105 Proposed Activities Mini campaigns, which require little funding, will constitute a primary component of community activities. Mini campaigns involve identifying a health problem in conjunction with health zone staff; initiating a dialogue with community members to define recommendations and develop a feasible action plan to address the health problem; and implementing community mobilization and sensitization related to the health problem. HCD will be used to ensure that local contextual factors are considered when designing mini campaigns. Campaigns follow an action plan that spells out the timeline and the roles for participants, and will generally offer a small incentive for those involved. We were told that mini campaigns will typically involve the participation of community agents, such as RECOs, CODESA members, influential leaders, and community organizations, to sensitize the population on important health practices and promote the use of health services, with an overall goal of establishing a long-term approach that encourages the adoption of healthy behaviors. Another activity, which was also implemented during PROSANI Plus, was the “Champion Community” model, which encourages communities to design interventions, mobilize resources, and control intervention activities and the use of funds. Led by influential leaders, communities identify a community activity that may or may not be related to health, and define an action plan that includes objectives and expected results. The AC at the zonal level is responsible for helping communities develop proposals and seek funds. Project cycles last six months and are evaluated by external groups, such as NGOs or government structures. Other planned SBC activities included a family campaign, which will use serial stories to convey information about health problems through the media, theater, or radio spots. Another intervention was the journée porte ouvertes, which is an open house event during which community members are invited to HCs to learn about health services. Community activities will also include the establishment of iCCMs sites, which are located in remote locations, where RECOs are trained to treat childhood illnesses with basic medications and refer patients to health facilities. Jeux concours—contests used to motivate behavior change—were also mentioned as an intervention. The SBC approach also aims to train government community agents, such as RECO and CODESA members, who will be provided with small sums of money so that they can function as described in the national strategy. We were told that a training plan for community agents will be developed for each province. Key informants noted that the current selection of CODESA members was problematic. Specifically, they contended that better￾educated community members were selected, creating a social bias that influenced the CODESAs to liaise more closely with health personnel than with community members. In an effort to create a more balanced approach, USAID IHP will involve community organizations in the promotion of healthy behaviors. The USAID IHP gender specialist will assist in developing strategies that involve women in community activities. Informants indicated that efforts will be made to share lessons learned, harmonize strategies, and improve approaches by collaborating and coordinating with other groups involved in SBC. They include the following: key government institutions working on communications; government officials, IPs and other stakeholders participating in coordination meetings at the central, provincial, and zonal levels; USAID staff and partners; and community-level actors, such as CODESA and CAC representatives. We were told that during the development of operational action plans, USAID IHP will provide assistance to health zones to ensure scalability of sound and effective messaging that aligns with agreed on health goals. The project also plans to share SBC activity results with international audiences during academic conferences and through peer-reviewed, scientific papers. Work Completed at the Time of Interviews At the time of our interviews, the SBC advisor was working closely with Matchboxology on the development of an action plan involving HCD. Several activities were underway, including the design and implementation of mini campaigns focusing on TB and malaria, which coincided with national TB and malaria days. Other activities included an evaluation of CODESA members and preparations for a workshop on community scorecards (USAID IHP is supporting government efforts to pilot community scorecards in health zones); and “Ne Pas Nuire” or Don’t Do Harm, which aimed to develop strategies to avoid causing additional harm when 106 The Impact of USAID’s Integrated Health Program in the DRC implementing humanitarian aid. The team was also preparing for a week-long campaign on exclusive breastfeeding to be held in August 2019. We were told that communication efforts at the provincial level had begun, with interviews carried out in the DPS and some Central Health Office Areas (BCZs), hospitals, HCs, and communities to develop communication strategies in the provinces of Kasai Oriental (March 2019), Haut Katanga and Lualaba (May 2019), and Tanganyika and South Kivu (June 2019). Respondents indicated that three to four health zones (including a mix of urban, peri-urban, and rural zones) will be identified in six target provinces in which communication approaches will be implemented. As part of this process, workshops will be held in August 2019 during which communication strategies will be developed in conjunction with personnel from Matchboxology in each of the target provinces, with the goal of finalizing six communication plans by the end of the fiscal year. In addition, Breakthrough Action was carrying out research to inform its community-based activities in Kasai Oriental and Haut Katanga in preparation for family campaigns. We were informed that Matchboxology and Breakthrough Action were working together to ensure that strategies focusing on health personnel and community beneficiaries were complementary. A combination of mass media and interpersonal community approaches were likely to be used to convey messages. Monitoring and Evaluation Team Members and Partners The project’s M&E unit is led by a director, with two support staff in Kinshasa and one USAID IHP staff member in charge of M&E in each USAID IHP province. At the time of the study, not all M&E provincial positions had been filled. The M&E team was supported by two M&E specialists from Abt Associates headquarters who had been providing ongoing assistance in Kinshasa. As mentioned earlier, USAID IHP was working with Bluesquare, a Belgian company delivering innovative technology. Bluesquare was in charge of setting up and improving information systems, and ensuring that data system platforms were functioning and could be used by government staff at different levels. We were told that Bluesquare was not involved in ensuring data quality (which was the responsibility of government personnel working on data collection, entry, and validation), and that data were gathered in HAs and validated during monthly meetings in health zones. At the provincial and central level, data were supposed to be reviewed regularly for inconsistencies, and the DPS was responsible for contacting health zones when concerns arose about data. One of the M&E team’s first tasks was to prepare the project M&E plan, which defined how each of the three program objectives would be monitored and evaluated. At the time of this study, there were 118 project indicators, including outcome, process, and output indicators. Respondents stated that additional indicators will be added to monitor other project components, such as environmental activities required by USAID. We were told that the team will fix annual targets at the provincial and regional levels for each indicator. Problems with the Health Information Systems Informants highlighted multiple problems at different levels of the health information system that affected data quality. The most frequent issue highlighted was that data were often incomplete and received late. They explained that the information technology (IT) unit was responsible for compiling monthly HA data, which were entered on paper forms at HCs. Informants reported that the data submitted by facility- and community-based health staff responsible for monitoring monthly activities were often inaccurate. We were told that many staff had not been properly trained in data entry procedures, which was partly due to ongoing turnover of facility- and community-level staff. Moreover, HAs frequently did not have the appropriate forms and, therefore, data were often entered and transmitted on pieces of paper, sometimes in an unintelligible format. Informants explained that monthly data entry required extensive time, and that the IT team could easily make mistakes while transcribing data, especially because zonal staff often failed to fulfill their responsibilities for providing training in Baseline Report 107 and monitoring data management. The IT team may also have reasons to manipulate the data, which we were told can occur at all levels of the system. Once the HA data were compiled, the IT units were required to deliver the information to the zonal offices, which can be as far as 250 km away. Another problem was that health facilities that no were longer functioning may not be removed from the health information system. Another major obstacle related to the Internet infrastructure, which often did not work. (One informant reported that 60 of the 178 health zones did not have an adequate Internet connection.) In these cases, the person in charge of data entry must travel to another health zone that had an Internet connection. After the data were entered, they were validated with the zonal team and other ITs. When errors were discovered, a return visit to the health zone with Internet connection was needed to make appropriate corrections in the DHIS2 systems. Our informants suggested that because of fatigue, errors identified during the validation session were often not corrected. In general, our informants suggested that the health staff were not motivated to complete all the necessary steps to overcome challenges confronted during data entry and validation. One informant said the following: We have so many problems in the country, which cause difficulties operating the system. After entering the data, the data are incomplete. Sometimes it's 60 percent [complete], sometimes it's 70 percent, and that is what is reported. And when you get into what is reported, that's where the problem of quality now comes in. And these problems are due to what? We cannot call it lack of motivation, but it is because the working conditions do not allow for health staff to do things normally to guarantee the quality of the data. We were told that the M&E team will work on strengthening the capacity of the information systems, with a first step of improving Internet connections, which in some cases will require reactivating the very small aperture terminal (VSAT) system. USAID IHP was exploring ways to ensure that data entry forms were available on a regular basis across all health zones. Work Status at the Time of Interviews At the time of our study, the M&E team was focused on the development of the study design and tools for the mapping exercise, which involved facility-based and household surveys to be carried out in all 178 USAID IHP heath zones before the start of field activities. The main purpose of this exercise was to delineate the type of assistance being offered by other IPs and donors, the time period for the implementation of activities, and ways to work with other organizations so that target health zones will benefit from a package of integrated services involving the six USAID IHP technical approaches. The data will be used as baseline information, in addition to several other data sources, including the DHIS2, the Malaria Indicator Cluster Survey (2017), and ongoing monitoring carried out through other activities, such as SBC. After completion of the mapping exercise, responsible partners and the timeframe for activities were to be spelled out in the contrat unique (unique contract) developed for each province. Informants asserted that in the future, coordination with IPs will be critical to prevent duplication of data collection, and to ensure that data were available on a timely basis and used to strengthen project implementation. Child Health and Nutrition Key Partners and Activities The advisor on child health and nutrition was in charge of overseeing the implementation of a package of child health activities linked to five government programs, including PRONANUT (Programme National de Nutrition), PEV (Programme Elargi de Vaccination [National Nutrition Program]), PCIME (Prise en Charge Intégrée de Maladies de l'Enfant) [Integrated Management of Childhood Illnesses], PNIRA (Programme National de Lutte contre les Maladies Respiratoires Aiguës) [National Acute Respiratory Disease Control Program], and PNECHOL-MD (Programme National d’Elimination du Choléra et de Lutte contre les autres Maladies Diarrhéiques) [National Program for the Elimination of Cholera and the Fight against Other Diarrheal Diseases] at the central, provincial, and zonal levels. Activities related to malaria will be covered separately by the USAID IHP malaria program 108 The Impact of USAID’s Integrated Health Program in the DRC advisor. We were told that a major focus of the child health approach was to ensure that the five child health programs were well integrated in the service packages so that sick children can reach health facilities in a timely manner and were provided care according to official treatment protocols. Although the malaria and nutrition programs were well established, the other three programs were reported to lack coordination and technical support. Training, supervision, and monitoring of health workers (especially in relation to the treatment of malaria, ARI, and diarrhea disease) were expected to be major focuses of child health activities. Although health workers in all USAID IHP provinces should already be trained on the treatment and prevention of the target programs, high attrition of health personnel necessitated regular training. We were told that the mapping exercise would be used to identify training needs. Informants also emphasized the importance of supervision to ensure that health workers were treating children according to national and international standards. USAID IHP will provide support to the DPS technical team and the zonal leadership team, including the MCZ and nurse supervisor, for supervision of facility- and community-based activities. Informants expected that an ongoing challenge will be ensuring that medications and medical supplies were made available in a timely manner. We were told that USAID IHP will provide support to ensure ongoing production and distribution of reporting tools for the DHIS2. Ongoing Collaboration and Potential Challenges Our informants stressed that collaboration within USAID IHP was very good, and included formal weekly meetings with staff representing different technical sectors. Informants cited effective ongoing planning between the SBC and child health group as an example. Although informants stated that the MOH supported the focus on strengthening health systems, they noted that USAID IHP was encountering problems related to financial agreements, with some DPS offices suggesting that they will not sign a contract unless they can control project funds. One informant said the following: In fact, the ministry is in favor of health systems strengthening for supporting planning activities, training, and all that, but there is a problem; it is the funding problem. For them, they believe that when we say strengthening it means we will strengthen their ability to manage themselves, not for USAID to manage in their place. This remains a problem because when we explained the situation, they said ‘no, we won’t work as your substitutes. If you want us to work then give us the money to manage.’ You will see that we have also become competent in the management of financial resources. There are provinces that bluntly state that if we are not going to give the money, they are not going to sign the contract. Informants maintained that the financial aspects of project support were always challenging, especially the remuneration of health workers. It was reported that there was little ownership of the work, with implementation of many activities based on obtaining money. As a result, the longevity of interventions was undermined. As soon as funding ended, people looked for work elsewhere. At the time of the study, common problems confronted in the USAID IHP zones included declines in vaccination coverage of measles and bacillus Calmette-Guérin due to stockouts of syringes and needles, difficulties with the DHIS2 data (February and March 2019 were cited as months when data were not available), and project activities from the predecessor project having slowed or stopped due to the lack of funding. Interestingly, some informants were unaware of the drug stockouts plaguing USAID IHP provinces or problems involving Chemonics getting medications to the CDRs that were reported by other key informants. Not all informants had been informed that community health agents would be responsible for transporting medications from the zonal offices to the health facilities. Work Status at the Time of Interviews A workplan for child health activities from October 2018 to September 2019 had been developed, with USAID IHP efforts concentrating on coordination with the five national-level child health programs. Another focus was to establish a collaboration plan with FFP staff in Kinshasa, with the USAID IHP staff having made two field Baseline Report 109 visits to FFP sites to assess how USAID IHP and FFP could work together. USAID IHP and FFP decided to collaborate in all 12 health zones where FFP was implementing activities, with USAID IHP taking the lead on facility-based activities and FFP overseeing community-based approaches, including those related to nutrition. Informants reported that funding and coverage of nutrition activities were limited, but explained that nutrition will be offered through the iCCMs sites, with training provided so that RECOs can better identify and refer malnourished children to HCs. It was noted that ready-to-use therapeutic foods were not provided by USAID, and UNICEF will not support treatment of malnourished children in all USAID IHP zones. Overall, the USAID IHP nutrition approach will focus on prevention, including counseling during Consultation Préscolaire (CPS [preschool consultation]) and mother support groups (L'Alimentation du Nourrisson et du Jeune Enfant) [Infant and Young Child Feeding]; dietary demonstrations promoting local foods; and the sharing of information through text messaging. When we asked about the PEV, we were told that USAID IHP will support the maintenance of the cold chain and field-based approaches aimed at reaching all eligible children, including children living in isolated areas with limited access to health structures. During our data collection, USAID IHP was developing an action plan for October 2019 through September 2020. We were told that the aim was to use results from the mapping exercise to guide proposed activities for fiscal year 2020. However, the mapping exercise was taking longer to implement than planned and, as a result, the annual workplan mentioned above had to be developed before the mapping results were available. The slow start of USAID IHP activities was a commonly reported weakness. Differences with the Predecessor Project The previous project, PROSANI Plus, targeted 78 health zones and focused on service delivery, whereas USAID IHP targets 178 health zones and is concentrating on HSS. The additional 100 health zones were added to USAID IHP to meet an agreement made with the MOH and other partners to provide support for malaria in all health zones located in the nine project provinces as part of the U.S. President’s Malaria Initiative. Although PROSANI Plus was composed of different programs working on such technical areas as FP and TB, USAID IHP integrates all technical approaches under one project. One informant made the following statement: When USAID talks about integration, it means integrating all programs under one project. With the previous project, there were different projects working on different technical areas, like FP and TB; M&E was done by MEASURE Evaluation. Now it is one project with all the money under IHP. Informants indicated that there should be general continuity in terms of other crosscutting approaches, such as the training of health personnel, supervision, and the provision and distribution of medications. Another major difference is that USAID has a contract with Abt Associates, whereas the predecessor project functioned under a cooperative agreement. A contract gives USAID program directors much more decision￾making power and technical oversight for project development, implementation, and the use of funds. We were told that USAID designed the project, with Abt Associates responsible for the execution of activities and ensuring that indicators were closely followed and results achieved. Any program adaptations need approval from USAID. One informant said the following: It is their [USAID’s] program. In the contract, we are an instrument in the hands of USAID to implement their program. Our informants reported that contracts have strict requirements, involve extensive documentation, and permit limited direct government access to cash funds. From the USAID perspective, the nature of a contract ensures greater accountability, with USAID providing daily oversight to ensure that program activities are implemented as planned, all with the aim of improving overall project results. We were told that the switch to HSS and limiting direct cash access is part of the USAID Journey to Self-Reliance strategy, which focuses on strengthening systems, improving work environments, and shifting the approach such that governments are in a better position to fund and implement high-quality health systems. 110 The Impact of USAID’s Integrated Health Program in the DRC As part of the contract, USAID is providing Abt Associates with funding for operational costs. The agreement also includes 19 fee-based indicators to be paid based on the completion of deliverables. The contract does not allow for budgetary modifications without consulting the USAID COR, who must go through the contracts office to make changes. In addition, Abt Associates is required to get approval from USAID to carry out all activities. Another difference between USAID IHP and the predecessor project concerns the direct involvement of other USAID projects, such as FFP and the IGA, and of collaborating partners executing interventions designed to complement USAID HIP activities. Examples include the World Bank, which is implementing PBF in the Katanga region; FFP, which is working on community nutrition and WASH in South Kivu, Tanganyika, and Kasai Oriental; and Access aux Soins de Santé Primaire, which was implementing an integrated health program in Kasai Centrale at the time of the interviews. The provision of medications has also changed. The former project purchased and distributed medications to health structures, although The Global Fund to Fight AIDS, Tuberculosis and Malaria was responsible for ordering HIV and TB drugs. Toward the end of PROSANI Plus, USAID established a global project in charge of ordering medications for all USAID projects around the world. The new project is composed of two branches, Procurement and Supply Management, which is responsible for procuring and transporting medications to countries, and Global Health Supply Chain Technical Assistance, which is based in countries and oversees the quantification, regulation, quality control, storage, and distribution of medications. Global Health Supply Chain Technical Assistance was also working with the CDRs to improve storage and management. Under the predecessor project, Management Sciences for Health was granted a government waiver allowing it to bypass onerous customs regulations. However, with the new USAID approach, medications have to go through customs, which has caused severe delays (up to a year) in getting medications to CDRs. USAID was relying on a subpartner, Chemonics, to procure and distribute medications and to carry out quality control, which is causing challenges. During interviews, we learned that estimated drug costs initially provided by Chemonics were inflated because Chemonics had based estimates on specialized rather than generic medications, and made larger orders than requested by USAID. With Chemonics, which is also responsible for getting medication from CDRs to health zones (normally the mandate of the CDRs), there have been long delays in distributing medications, causing stockouts. At the time our interviews in April 2019, only 60 percent of essential generic medications ordered in July 2017 had arrived in the CDRs, with USAID having to make an agreement with the World Bank to help fill the gap in drug supplies. Rather than relying on the health zone, another change made was that the project had decided to provide resources to community personnel, such as CAC representatives, CODESA members, or RECOs, to transport medications from health zones to facilities. USAID believed that this approach would better ensure that medications were transported quickly and would reduce the possibility of drugs being stolen. Problems with Chemonics were attributed to a lack of experience and the fact that Chemonics did not have an official office in the DRC but was relying on sub-partners. Our informants stressed that the project’s success depended on the availability of drugs and expressed concern that inadequate and late deliveries could undermine results. One informant said the following: This issue of drugs. A difference between this project and the former project is that the former project had the opportunity to purchase drugs. You cannot implement a health project without drugs because health means treating people. You strengthen the health system but give the procurement of drugs to someone else? The results depend on medications. What a design, the project structure is quite complicated. This is the first time I have seen a health project where another group is responsible for the drugs. We will continue to support the [health] system, but for the system to work, it is necessary to have the drugs. My fear is what? If the drugs are not available, people will not understand. Even if the system is strengthened, in what way will this be effective if the drugs are not there.... Baseline Report 111 USAID Integrated Health Program Challenges The project has faced many unforeseen delays. They include a period of more than four months when another organization bidding for the project contested the award given to Abt Associates; turmoil and uncertainty leading up to the Congolese presidential elections; mandatory evacuations of U.S. government staff around the time of the Congolese presidential elections; and the U.S. government shutdown in early 2019. At the time of the key informant interviews, Trafficking in Persons (TIP) sanctions posed a major threat to the project. We were told that if TIP sanctions were enforced, which informants expected would be decided in June 2019, USAID IHP would have to identify alternative sources of funds. USAID was consulting legal experts about project revisions that would have to be made with the enforcement of TIP. There was much talk about “descoping,” which would involve revising the workplan designed through 2021 to remove activities that directly supported the government or government entities, thus changing the overall focus of the project. Uncertainty about TIP put USAID IHP in a difficult situation with government collaborators in terms of the start-up of activities. Another factor slowing implementation related to cash transfers to government entities. Although health zones received and managed a monthly budget under the previous project, with the contractual agreement, USAID IHP was forced to closely monitor how all project money and resources were used. Specifically, the contract stipulated that USAID IHP controlled the purchase of all materials and supplies, and monitored the use of even small sums of money. At the time of the study, USAID IHP was developing a system for regular cash transfers to more than 3,000 health structures in 178 zones needed for ongoing activities, such as monthly supervision carried out by health zones (US$850), reporting meetings held in HAs (US$15), and zonal meetings (US$50). We were told that monitoring to prevent the misuse of funds would be laborious; for example, Abt Associates must verify that monthly supervision of activities by the DPS and health zone teams was occurring as described in the protocol. In addition, subcontracts established with government entities will require that each recipient receiving U.S. government funds followed rules related to the management of Federal funds. This approach will also require a great deal of time and work, requiring the monitoring of finances and deliverables of all subcontractors. We were told that there had been some misunderstandings between Abt Associates and USAID regarding the project design and its execution. For example, the initial workplan developed by Abt Associates was apparently more closely aligned with the PNDS strategy than with the USAID approach. Over time, Abt Associates personnel were gaining a greater understanding of the USAID approach and vision for the project. In general, the Abt Associates team received favorable endorsements from other key informants, who noted the organization’s strong technical background and experience, commitment to HSS, and flexibility. Another problem arose from the fact that most medications provided through the previous project had run out and were not being replenished quickly, due to challenges faced by Chemonics. During interviews, we learned that computer software was being introduced for tracking the status of drug stocks in all CDRs and HCs across the country. Once operational, the program will predict stockouts of drugs in health facilities based on their use and will notify health workers when they need to order drugs. As stated above, the mapping exercise was taking longer than planned, slowing the implementation of workplans. Last, because of security concerns, many USAID staff were not permitted to visit more remote, rural areas, preventing them from getting a first-hand understanding of field conditions and operations. Collaborators Direction Provincial de la Santé The DPS in Lualaba was established in 2015 and is composed of 14 health zones. There were six offices in the DPS: technical support, resource management, information systems, hygiene, teaching in health sciences, and inspection control. Each office had a director who coordinated activities and technical analysts, with additional staff including secretaries, drivers, and security guards. Of the 75 staff officially needed to fulfill the work 112 The Impact of USAID’s Integrated Health Program in the DRC requirements of a DPS, the office included 32 personnel, of which only two were women, both in administrative positions. We were told that professional positions in Lualaba were mostly occupied by men and that representatives on health committees were also generally men. However, this was changing because more women than men were currently in medical programs, and women were more willing to serve in rural areas. The Lualaba DPS rented a building that had recently been sold, and at the time of the study, the director did not know where the office would relocate. We were told that the DPS followed an annual workplan and that each trimester the DPS was evaluated based on activities implemented according to the annual plan. In addition, each health entity (BCZS), hospital, HC) was responsible for designing an annual plan that aligned with the DPS workplan. The DPS participated in a contrat unique, which was guided by the PAO, spelling out roles and activities of IPs and donors and informing trimester activity workplans. We were told that funds from partners often arrived late, causing activities to fall behind. For example, DPS staff reported that they did not have funds to implement planned activities in the first trimester of 2019, which affected their performance and forced them to revise their workplan. Our informants indicated that the DPS attempted to follow the PNDS as a guideline for implementing health interventions. They stated that—although the national strategy was well designed with all essential components—the Lualaba DPS office did not have the capacity to follow national norms and implement the strategy as proposed. In addition, the provincial steering committee in charge of making decisions and ensuring that activities were executed according to national health norms was described as ineffective, in part because the governor had failed to take a leadership role to ensure the support of health zones. We were told that other working groups and committees set up to ensure coordination of activities at the provincial and zonal levels lacked leadership, finances, and adequate participation of government representatives. As a result, recommendations and resolutions established during meetings were not generally applied. It was reported that the DPS office was severely hampered by the lack of resources, with informants indicating that the government failed to provide even the most essential support, such as funding for the construction and rehabilitation of health structures, equipment and materials, equipment maintenance, supervision of health services, and health worker salaries and primes (i.e., salary supplements), needed for the health system to function. Key informants said that only three percent of health workers in Lualaba received their salaries, many of whom were already at retirement age and no longer working. Even the director of the DPS did not receive a regular salary. In addition, 50 percent of health personnel had received the prime de risque. In 2018, the provincial government provided three prime payments to all health workers in the province, with higher primes given to workers in more isolated areas. Apparently, this was a first-time occurrence that served as an important incentive. In general, lack of government support was reported to force the DPS to rely on support from IPs and health worker fees. IPs may provide primes to DPS, BCZS, hospitals, and some HC personnel as motivation, with informants indicating that the IP primes were more generous than the government premiums. During our interviews, the DPS director provided a detailed description of the challenges that health workers faced, emphasizing that the work was demanding and that workers should be paid appropriately for their efforts. As an illustration of the sacrifices made, he presented a case study involving a RECO who had traveled two days and 110 km each way to obtain vaccines for his HA. Lack of government support forced health workers to rely on out-of-pocket payments by sick patients to generate revenue, which was managed internally by staff to pay health workers and invest in health structure needs. We were told that health workers developed approaches to increase payments, which generally involved increasing user fees, which, in turn, affected the use of health services. Another strategy used to get money was called pharmacie de tiroir et poche (pharmacy in the drawer and pocket), which was a common practice involving health workers selling facility drugs for their personal benefit. Lack of salaries also led to high attrition and, in turn, created vacancies in health structures. Our informants reported that the turnover of well-trained staff placed a burden on the system, leading to increased training needs and lowering health worker capacity. We were also told that when projects with primes or other incentives ended, trained workers often searched for other, more lucrative opportunities. When talking about motivation and USAID IHP, one informant said the following: Baseline Report 113 Motivation is a serious problem in almost all health zones because the government does not pay everyone, and even when it pays, it is not enough. The USAID partner comes and does not give a prime. PROSANI does not give a prime to the staff, which causes terrible demotivation. This is the weakness of the USAID partner; the approach prevents sustainability of activities because when the partner leaves, everything closes and stops. There is no good appropriation of activities that PROSANI implements because when PROSANI is there, it is PROSANI that does everything. PROSANI brings all the funds. When there is training, it is PROSANI who has the money. PROSANI pays for trainers, food. We will take a house and PROSANI pays; the training facility, PROSANI also pays for that. This does not give a sense of responsibility to representatives of the Ministry of Health. We feel that this is their project, they are the ones who come with the money, it is their project. This informant had numerous criticisms of USAID IHP assistance offered through the predecessor project, indicating that USAID IHP was strictly in charge, providing funds, making decisions, and executing activities, and because USAID IHP failed to delegate responsibility, it was catastrophic when it left. He cited joint supervision visits carried out with the DPS as an example, indicating that USAID IHP determined the terms of reference and provided feedback to health personnel without soliciting input from the DPS counterpart. The same informant reported that as a MCZ, he received money and fuel for supervision from USAID IHP, but the zone did not have working vehicles. Some informants described a general disinterest of government authorities in strengthening the health system due to the assumption that money and technical assistance would be provided through international sources. Another major DPS challenge related to tracking the status of health personnel, with DPS personnel explaining that they did not have an updated list of staff currently working in health zones. Although the DPS was developing ways of maintaining a list of active personnel, it was difficult to stay current on health worker deaths or new recruits. This was partly because health workers were commonly hired through unofficial mechanisms. Other weaknesses mentioned included limited technical capacity of DPS personnel; the inability of health workers to provide quality care; poor management of materials and equipment, especially vehicles; and the inability of the inspection control office of the DPS to function properly. One informant said the following: One of the big weaknesses that we regularly observe is related to the inspection. As you know, we are a provincial health division. Next to it they put inspection, but the provincial health division also has the role of inspection, internal inspection, which was established before the external inspection. That is to say that the provincial division has an office of inspection to control healthcare structures, such as hospitals, health zone offices, health centers, to ensure control in terms of human resources, material resources, financial resources, whether they are used well or not. It is an office that remains compartmentalized, which does not work so well, I think, due to overlap, or to the confusion between the inspection structure, which is a decentralized entity at the national level, and the inspection office, which is part of the office of the provincial health division, which must carry out its role of internal control. We were told that the DPS went through a PICAL assessment supported by Abt Associates in 2016‒2017, which helped define roles and increased leadership capabilities to delegate responsibilities to staff members. The same project provided assistance to identify needs in terms of capacity building, training, and resource management, which some informants claimed enhanced the way the DPS office functioned. Informants reported that the USAID IHP predecessor project offered training to DPS and zonal staff on a range of topics (e.g., FP, revitalization of CPS, SNIS, DHIS2, TB, etc.) to strengthen capacity. Monitoring and Evaluation DPS personnel stated that training on using the DHIS2 software was first carried out in 2012 and a refresher workshop was held in 2016. Health zones are responsible for submitting health service data at the end of each month, with data entry typically taking about five days. The DPS Office of Health Information was responsible for overseeing the monitoring and analysis of health zone data. Respondents reported that Internet access presented an ongoing challenge; although the VSAT was installed, the health system did not have money to pay for Internet subscriptions or credit for a modem. Even when there was a connection, it may not be strong enough to access the Internet or to use the DHIS2 system. These problems forced health workers to find 114 The Impact of USAID’s Integrated Health Program in the DRC alternative ways to access the Internet, such as using personal modems or traveling to areas with Internet connection, which may cause delays in submission of monthly reports. Other problems mentioned included lack of data entry forms, forcing health personnel to use pieces of paper for data entry; inadequate training for health workers on data entry; transcription mistakes; long distances between the HC and the zonal offices (up to 250 km), causing data to arrive late; and old and improperly maintained computers. An M&E staff member reported that the head nurses frequently did not understand the SNIS and, as a result, submitted monthly reports with errors or missing data. He said the following: The big challenge relates to Internet connectivity. Although modems were given, units for an Internet connection are provided irregularly. It is a really big challenge. And the other challenge is that head nurses don't have electronic tools, they have to report on hard copies, and the distances between the central zonal offices and the health structures are really huge. Sometime, the health structures bring the reports late, and if it is not late, there is also the problem with the quality of the [work of] the head nurses. Because in most rural areas, the head nurses are sometimes uninformed. If a head nurse has little understanding of the SNIS, you get reports with lots of data that has been crossed out with a lot of changes or gaps, and there are a lot of inconsistencies, which creates problems. We were also told that HC and zonal staff lacked the capacity to analyze and interpret the data to inform their field activities. Informants indicated that the zonal reviews were rapid and often superficial, and that more time was needed (two days rather than one day was suggested) during monthly zonal meetings to validate data and identify errors generated by health structures. The DPS held a meeting with all provincial health zones every three months to review data. During these meetings, the DPS Office of Health Information , along with zonal representatives, identified errors, identified data that did not make sense, and gaps (an especially big problem), and gave feedback to the health zone about corrections, with each zone graded on data quality. Monthly data reported were also analyzed by the provincial M&E team and were used to plan routine activities, such as supervision visits. An analysis was also done of annual health zone data, which were used to inform annual workplans. DPS informants highlighted the need for more training and improved capacity on the use of the DHIS2. Disease Surveillance Health zones generally employed a passive disease surveillance system, with HCs transmitting a weekly report to the zonal offices when disease was detected. More active reporting was done for certain illnesses, such as cholera. RECOs were encouraged to bring suspected cases of disease (measles or polio were mentioned) to the HC and, during epidemics, RECOs were requested to go house-to-house to identify cases. Weaknesses in the surveillance system were that HCs may not report on a weekly basis, reporting forms were not available, or health personnel were not available to deliver weekly reports to the zonal offices. Moreover, when a report was delivered to the zonal offices, the person delivering the report may not be able to describe the suspected illness cases appropriately. Informants stated that health personnel at all levels required additional training to ensure more accurate diagnoses. Provision of Medications Lualaba shared a CDR with Lubumbashi, which DPS informants said prevented them from having adequate control of the acquisition and management of drugs. DPS staff stated that USAID IHP was in charge of ensuring that CDRs were adequately stocked with essential medications and with delivering drugs from the CDRs to health zones. DPS staff indicated that since Chemonics started its contract (more than one year before the interviews), there had been deficits in medicine stocks. In fact, at the time of data collection, there was a serious shortage of medications, with informants reporting that Chemonics had been unable to stock adequate supplies, causing shortages in the CDRs and stockouts of certain medications at health facilities. We were told that this was forcing HCs to purchase medications from Pakistani- and Indian-run depots, with informants questioning the quality of the supply chain and of the drugs themselves. We learned that inexpensive counterfeit medications had recently infiltrated the local market and that government workers frequently purchased them to stock their pharmacies. It was also noted that small, private pharmacies had recently sprung Baseline Report 115 up to fill the void in drug availability. Another problem was that health personnel lacked the capacity to manage medications, which served as health structure capital, often failing to maintain enough revenue to replenish stocks for subsequent months. Inadequate funds forced health structures to order insufficient quantities of drugs to meet ongoing needs or to resort to the purchase of low-quality drugs. Health personnel at the HC and zonal level were also lacked capacity to quantify ongoing needs; this, along with poor drug management, led to regular stockouts. Another issue was that health workers tended to over-prescribe medications. Performance-Based Financing The PBF approach in Lualaba started in 2017 in six zones, with the other eight zones added in 2018. Informants explained that the DPS and zonal offices were given a score based on a list of deliverables that they had to meet every trimester. If they reached 20 to 50 percent of the total score, they received 50 percent of the potential monetary benefit, and if they reached more than 50 percent, they received the full amount, which was US$30,000 for the DPS and health zone offices. For hospitals, indicators were based both on quality of care and number of patients treated, with hospitals potentially receiving more than $30,000 per trimester. Other incentives, such as building a maternity ward, were also offered. When health structures met trimester performance targets, 50 percent of the funds received were intended to be used for health worker primes and 50 percent for health structure investments, with health personnel responsible for making investment decisions. Primes were supposed to be based on health personnel qualifications, seniority, and performance related to caseloads and accomplishments during the trimester, with both quantity and quality of care assessed. There was agreement that health workers may be incentivized to manipulate health structure performance data to meet targets and to increase scores and monetary benefits. Community Health According to the national plan, community members should participate in healthcare provision related to generating financial resources and the management of health facilities. CODESA members were supposed to represent the population, serving as a liaison between health structures and local populations. Another aim was to establish community committees and structures to ensure community participation in healthcare. In general, we were told that the engagement of community members in decision making and resource generation designed to strengthen the health system was limited. Moreover, little effort was made to understand the perspective of community members on local health needs and problems faced in accessing care. Informants also indicated that CODESA members generally established ties with health personnel, butfailed to advocate for the needs of the local population. Although CODESA redynamisation (revitalization), which included the promotion of female CODESA members, started in 2016 with funds from UNICEF, initiatives were only carried out in some HAs and, according to our respondents, the efforts were ineffective. RECOs were described as frequently inactive. When active, informants said that RECOs failed to fulfill their roles according to the national community strategy. Specifically, we were told that RECOs were not identified according to the official selection criteria, did not receive adequate training, and often committed without understanding that the position was voluntary. Respondents reported that in Lualaba, RECOs were frequently female, with some informants noting that women were more motivated to work and that men in Lualaba province were highly mobile. Informants reported that the Management Sciences for Health-led IHP project provided assistance to establish iCCM sites in remote areas. Informants also mentioned the following community-based initiatives: champion communities, which involved activities related to community agriculture, gardening, and health; mini campaigns focusing on FP and TB; revitalizing CPS; CODESA meetings; training of RECOs on sensitization to HIV and TB; training of RECOs on screening and prevention of malnutrition; and engagement of community members in the prevention of malnutrition. 116 The Impact of USAID’s Integrated Health Program in the DRC USAID Integrated Health Program Activities Initiated at the Time of the Interviews The Lualaba DPS participated in a workshop in Kinshasa in late 2018 to develop the PAO. Another workshop was held in Lualaba in February 2019 to officially adopt the PAO and its activity plan. In addition, USAID IHP had sponsored a meeting with other IPs to introduce the project and review workplans. Some training of DPS staff had been carried out, but the topical areas were not mentioned. The provincial USAID IHP office staff also visited all health zones in the province to meet with zonal staff and IPs. At the time of the interviews, USAID IHP had also started to support health zone supervision. USAID IHP had provided support for World Tuberculosis Day, with activities carried out in both communities and prisons. Although USAID IHP had promised to provide support for National Malaria Day, which occurred during our visit, money to implement activities did not arrive in time. It is important to note that DPS personnel were not aware of the funding changes made in USAID IHP compared with the predecessor project. Most DPS informants were anxious to sign a contract and to start activities. Inspection Provincial de la Santé The first IPS office was opened in June 2016, with only one inspector serving since then. The office had all 26 staff officially required for the four IPS divisions to function: pharmacy; teaching science and health; resources, administration, and finances; and medical technique. However, we were told that the government did not provide adequate resources for work activities to be implemented as planned. The IPS Lualaba headquarters was situated in the inspector’s home compound, with one office space for all 26 staff. All office furniture and materials were donated by the former IHP project. The IPS had three motorcycles, with two provided by the predecessor project and one by the DPS, which were used to carry out inspection visits. Informants reported that, initially, there was inadequate clarification about the roles of the DPS and the IPS and how they were supposed to interact, with the two offices often duplicating roles. Over time, and with the involvement of members of the Comité Provincial de Pilotage (CPP [Provincial Steering Committee]), which was presided over by the governor and included IPs, the two offices formulated a working framework conducive for both groups to carry out their prescribed roles. Although some difficulties still existed, especially in relation to interactions between the IPS and the DPS office of inspection control, it was clear that the DPS office was in charge of internal audits, whereas the IPS did external audits. We were told that coordination bodies designated to provide oversight of the IPS at different levels, including the CPP, the Conseil d’Administration and Comité de Gestion (Board of Directors and Management Committee) at the zonal level, the Comité Directeur de l’Hôpital (Hospital Management Committee) in reference hospitals, and CODESA at the community level, were ineffective due to a lack of funds, poor participation in and management of meetings, and insufficient monitoring of IPS activities. At the time of our interviews, the IPS was following an annual workplan and a trimester activity plan, which were designed in accordance with DPS activities. The IPS was assisted by the World Bank under the PBF project, with deliverables based on field visits and the identification of irregularities in healthcare services. Such irregularities can lead to administrative and penal sanctions. Examples of situations that merited penal sanctions included fictional workers on the government payroll, or workers receiving a prime for work in a health structure when they were officially working in another location. We were told that the Lualaba IPS—because of the lack of funds—had mostly worked in the eight zones funded by the previous IHP project, most of which were accessible by road and closer to Kolwezi. Visits to health zones must be approved by the governor’s office, with the terms of reference also shared with the MOH. Health zones were not informed of the visits in advance. Staff used motorcycles or local buses for transport, and were supposed to carry out field visits 10 to 15 days per month. Visits generally entailed one day in the zonal offices, two days in the reference hospital, and two days in three to four HCs. Visits were guided by an inspection checklist and a list of questions. When IPS staff arrived in a zone, they were required to hold a meeting with the MCZ and chef de village (village chief), who must sign a form indicating that the team had arrived to carry out an inspection. Five days after a field visit, a report was submitted to the MOH in Kinshasa Baseline Report 117 and the provincial governor, with recommended actions sent to the DPS and the concerned health structures. When irregularities in the use of finances were detected, the IPS requested that the health structure return the misappropriated funds. If this was not done, the IPS could impose sanctions. We were told that the biggest challenge the IPS faced was a lack of funds to carry out visits as planned. As to irregularities that IPS staff identified, the biggest problem was the mismanagement of funds and medications. A specific challenge in Lualaba was that many health structures were privately run by mining companies and, therefore, did not follow official standards. Other problems were that health structures did not have the appropriate equipment, such as incinerators, to follow government standards or lacked qualified personnel. The IPS reported a recent surge in the circulation of “fake” drugs, indicating that health workers were tempted to buy these drugs because they were cheaper, and that health workers did not have enough capital to stock their pharmacies with high-quality medications. Another problem was the use of expired medicines in the health structures. The IPS said that corruption was rampant and serious, especially because it affected lives. The IPS had a big role in curtailing corruption in the province. When asked whether a hotline would help, we were told that health workers must be motivated to denounce irregularities and that community members may be afraid to report corrupt practices. At the time of the study, the IPS office was establishing a budget for control and audit visits. Respondents indicated that the second-trimester plan required US$45,000, although USAID IHP had only promised US$5,000. USAID IHP had also supported the inspector’s participation in an institutional capacity￾strengthening training in Lubumbashi in February 2019. We were told that the focus was on leadership and that gender was also discussed. USAID IHP had not provided any other support, and informants stated that USAID IHP was still setting up its offices. Respondents raised questions about the delay in contracts and the slow start of USAID IHP. Medical Director of a Health Zone The MCZ indicated that the previous IHP project had provided the zone with extensive support, highlighting funding for supervision, medications, and community activities, such as iCCM. We were told that after May 2018, when USAID’s PROSANI Plus ended, activities in the health zone slowed significantly because funds were no longer available. Activities especially affected included the delivery of medications (although medications for malaria and HIV continued, as did the provision of certain FP methods); data entry and reporting of DHIS2 data (due to a lack of credit for modems); and community services, such as CODESA meetings and RECO activities. At the time of the interview, the MCZ reported that the DPS was requesting that health structures provide money to the zonal offices to support supervision visits and Internet connectivity. Our informant reported that USAID IHP representatives had given a presentation on the USAID IHP approach to health zone personnel, but that USAID IHP was still defining key strategies. Since the beginning of 2019, USAID IHP had started to provide some assistance to health zones, such as the development of health zone PAOs and financial support to hold meetings with the health zone board of directors. The MCZ also reported that USAID IHP, in conjunction with the DPS, had carried out a supervision visit to the health zone. The MCZ expressed concern that the slow start of USAID IHP was affecting the execution of ongoing zonal activities. Food for Peace FFP works with vulnerable populations to provide assistance for a range of activities, such as agricultural production, nutrition and health, WASH, and FP. This assistance was offered through two programs. The first supported people affected by emergencies and primarily involved food distribution, money transfers or vouchers, the promotion of agricultural production, and the treatment of malnutrition. The second program, Development Food Security Activities (DFSA), focused on community development to enhance food security activities (through improved food production and the management of natural resources), and the prevention of chronic and severe malnutrition through improved breastfeeding and complementary child feeding practices. 118 The Impact of USAID’s Integrated Health Program in the DRC Development interventions also involved WASH; economic activities, such as credit approaches; FP; literacy; and interventions to address the structural causes of chronic malnutrition related to gender discrimination and poor governance. Community sensitization activities were conducted through Care Groups targeting mother leaders responsible for delivering messages to other mothers on health, nutrition, and FP. At the time of our key informant interviews, FFP had three staff members, including an interim director, an agricultural specialist, and a nutrition specialist. Respondents indicated that the annual budget for emergency activities was US$180 million and US$20‒US$30 million for DFSAs. Emergency activities typically lasted anywhere from 12 to 18 months. FFP had three ongoing DFSA projects that started in 2016 in 12 health zones located in South Kivu (six health zones), Tanganyika (three health zones), and Kasai Occidental (three health zones). The projects will run through 2021. The USAID IHP will collaborate with FFP in the DFSA project areas, with joint activities structured around a results framework and a workplan designed to complement USAID IHP activities. The FFP team emphasized the value of working together, stating that FFP implements revenue-generating activities that will facilitate payment for health services. Moreover, informants stated that in the past, FFP had received criticism from beneficiaries who were referred to health facilities at which child nutrition services were unavailable. With the HSS approach, the hope was that HCs would be higher functioning and would offer comprehensive services. One informant provided the following explanation: We have mothers of malnourished children who were referred to health structures as part of the program. It was a problem for FFP because sometimes people were referred to health centers where services were not being offered. The beneficiaries would accuse us of sending them to facilities where there were no services, nothing offered. This had a negative effect on the care-seeking messages we were disseminating. Working together will really help. One foreseeable challenge mentioned by informants was that the timeframe of the two projects was different. Integrated Governance Activity The IGA was a cross-sectorial program primarily collaborating with the USAID health and education sectors and Congolese government institutions in efforts to strengthen policy frameworks. Our informant highlighted the benefits when USAID technical offices worked together to ensure a more holistic approach and to maximize outcomes. However, we were told that working with other sectors can be time-consuming, can increase workloads of the CORs, and typically required strong entrepreneurial skills and the desire to learn about other technical approaches and projects. The IGA promoted transparent, credible, and inclusive elections; human rights; good governance; and anti￾corruption, with the aim of improving public service delivery at the local level. The IGA also helped government entities pass legislation conducive to improving the delivery of public services. At the grassroots level, the IGA worked with local municipalities to identify funding priorities and to build skills in financial management and participatory budgeting of local projects, often building technical skills of local institutions, such as CODESA. The IGA also had a civil society component that used community scorecards to equip people with skills to better advocate for priority health service needs and help provide local services related to health. As part of this effort, the IGA was conducting an impact evaluation to assess how improved skills in financial management and community scorecards strengthened transparency and health services, and affected health outcomes. The IGA office in Kinshasa had four national and two international staff. Their mandate, which focused on Objective 2 of the Country Development Strategy started in 2017 and will be implemented for five years in provinces where USAID IHP is being executed, including South Kivu, Haut Katanga, Lualaba, Kasai Centrale, and Kasai Oriental. At the time of the study, the IGA was under TIP sanctions, which stipulated that USAID￾funded programs could not work with the Congolese government. Although the IGA had been able to continue to use money obligated before the sanctions, the money was running out and the IGA was forced to stop supporting government structures. When talking about TIP, our IGA informant said the following: Baseline Report 119 That was decided I think in early December [2018. And that has, you know, affected our health and education collaborations. The entire mission is kind of reeling from TIP. It has been catastrophic. Our judgment day is coming. In the next couple of months, we will know if DRC is removed from TIP Tier 3 ranking. If we get downgraded to Tier 2, then we go back to normal. We were told that the TIP ranking was based on an annual report drafted by the U.S. Department of State that assesses government commitment to counter trafficking. The assessment reflects Institute of Medicine reports on the incidence of sexual- and gender-based violence, sex trafficking, and child labor. Although the DRC had been at Tier 3 for many years, the U.S. government had previously signed a waiver. Our informant explained that to be removed from Tier 3, the Congolese government must demonstrate concrete ways that it was addressing the trafficking violations and making improvements. If the DRC was not removed from Tier 3 in July 2019, another year without assistance to government would be applied. The IGA had the mandate to work with governments and was examining modifications to its approach that would allow it to meet this mandate. The IGA had held joint planning discussions and meetings with USAID IHP—the biggest partner it expected to work with—at the central and provincial levels to identify potential partners for collaboration. In USAID IHP provinces, the IGA planned to work on strengthening governance through improved legislation and policy in the health sector at local and provincial levels. Given the size and breadth of USAID IHP, the IGA would defer to USAID IHP to cultivate strong relationships and technical assistance in governance and capacity building with the DPS. At the local level, the IGA planned to work with CODESA on its role in strengthening community participation and advocacy, the accountability of health workers, and reinforcing health systems in HAs. Interestingly, a recent IGA-funded community political analysis stated that CODESA members had been co-opted by HC workers and local officials, who used their relations with CODESA to get free healthcare. The assessment also showed that, although regular elections of CODESA were mandated, elections had not been held for approximately five to ten years in most zones. Results also highlighted that community members were uninformed about the role of CODESA, when CODESA elections should be held, and what they should expect from local health officials. Results showed that citizens were not using accountability mechanisms, such as laws designed to encourage the responsiveness of health personnel to local health needs. At the time of the study, the IGA was working with a legal team to understand which entities it could work with (including CODESA) under the TIP sanctions. Our informant indicated that the IGA partner implementing field interventions would continue to participate in USAID multisectoral meetings to identify ways for the IGA to support activities that did not involve the Congolese government. World Bank The World Bank was involved in several projects in the DRC, including interventions focused on strengthening health systems through PBF, supporting the government to improve health reforms and administration, and addressing gender-based violence in South Kivu. It was also working on the development and implementation of a new health structure in the DRC, and was planning a large, multisectoral nutrition project to be executed in the Kasai region. At the central level, the World Bank was supporting the creation of a new office that would oversee information systems. It was also leading several studies, including research examining the feasibility of universal health coverage, ways to ensure the retirement of health workers when they reached retirement age, and the mobilization of resources for the health sector. The PBF project (PDSS) was being implemented in approximately 179 zones, providing care to at least thirty million people in 11 provinces (including the USAID IHP provinces of Lualaba, Haut Katanga, and Haut Lomami) with the aim of motivating health workers and reducing healthcare costs. Although the project supported all DPS offices in the 11 provinces, the degree of support was based on assistance provided by other partners and local needs. In the Katanga region, the project started in 2015 and was scheduled to continue until 2022, with our informant indicating that there would likely be a five-year extension. We were told that the Congolese government had recently adopted a national PBF strategy guided by the World Bank approach, with the World Bank providing technical support to a pilot project replicating the approach in three to four health 120 The Impact of USAID’s Integrated Health Program in the DRC zones in Kinshasa. When asked about the sustainability of the PBF approach, our informant argued that all projects require regular funding and, therefore, PBF was similar to other health interventions. At the same time, our informant contended that PBF would only be sustainable when it could be funded and operated by the government. In provinces where PBF was being implemented, the World Bank participated in a joint contract (contrat unique) to capture financial commitments and activities of different partners at the central, DPS, and health structure levels, and to ensure that activities were well-coordinated and standardized. The World Bank had established performance frameworks with government collaborators that defined expected trimester results and associated costs. As part of the approach, the World Bank encouraged the government to establish agencies in charge of buying services (agence achat des services) in hospitals and HCs based on a defined package of services with set payment costs. These agencies were in charge of verifying whether reported services were provided (e.g., number of assisted births, vaccinations given, collaboration with local associations), and provincial administrators evaluated the service quality. Based on services provided, payments were supposed to be made every trimester according to trimester results. Our informant said that the validation of services provided presented many challenges. For example, external checks may not correspond with the results given by the health structure; many health structures were inaccessible, making a formal validation difficult; and in some areas, high seasonal migration altered expected service provision. The World Bank worked with the USAID IHP predecessor project in Haut Lomani. Although our informant stated that PBF activities were being implemented in USAID IHP health zones in the Katanga region, he noted that USAID IHP had not yet started field activities. However, he was aware that USAID IHP had worked with DPS directors to develop PAOs. In addition, he recently attended a talk presented by the Chief of Party of USAID IHP during a groupe interpersonnels de santé, a group comprising donors and IPs that administer health services. The meeting provided information on the USAID IHP mandate and clarified his understanding of the USAID IHP approach. Baseline Report 121 Qualitative Baseline Report, Health Zone of Bunkeya Background Information We collected data between November and December 2019 in the Bunkeya health zone situated in the Lualaba Province, a province established in 2015. The evaluation was conducted in the Kikobe and Kalwa HAs. Kikobe is a higher performing HA and Kalwa is a lower performing HA, according to child health indicators for service use for key childhood illnesses and vaccination coverage. In each HA, we conducted in-depth interviews with an IT (head nurse), a member of the CODESA (health development committee), a RECO (community health worker), and a village chief or a village chief representative. We also administered in-depth interviews with one physician and the hospital administrator at the reference hospital, and conducted a key informant interview with the Bunkeya health zone medical officer (MCZ). We also carried out observations of facility infrastructures (3) and health worker-patient interactions (27). The average age of in-depth interview informants was 45 years and the majority (8 of 10) were male (Table 4.1). Both ITs had A2 level training. ITs, CODESA members, RECOs, and village chief informants had 10 years of schooling, on average, whereas the two physicians had completed 18 years of education. Informants had nine years of work experience in their roles, on average. All but two informants participated in other work, most frequently farming. In one HC, the health staff also worked as teachers at the local school. We administered focus group discussions with 12 caregivers of children under five years of age in Kikobe HA and eight caregivers in Kalwa HA. Discussions focused mainly on child health services and care seeking for sick children. Table 4.1. Background information collected from in-depth interviews and key informants in Bunkeya Variable In-depth interviews (10) Key informant (1) Average age (years) 45 37 Average years of education 11 18 Sex - Male - Female 8 2 1 N/A Religion - Catholic - Protestant 4 6 1 N/A Average years of experience in their roles 9 6 Participation in other work* - Farming - Teaching - Nun - Preacher - Judge for the village chief - None 6 1 1 1 1 2 1 N/A N/A N/A N/A N/A Average number of household members 6 4 *One informant had more than one additional job. 122 The Impact of USAID’s Integrated Health Program in the DRC Facility-Based Services Infrastructure The Kikobe HA is comprised of eight villages and includes one HC and one iCCM site. The HC is in the town of Kikobe, situated 12 km from Bunkeya, the capitol of the health zone. The Kikobe HC is a small building built in 2008 and is owned by the Catholic Church. Staff include the IT (A2), the assistant infirmier titulaire (ITA) (A1), and a nurse (A3) who acts as the pharmacist. Solar panels provided by the ACCESS project around 2008 were not functioning and electricity was not available. The HC has a maternity made of local brick and a thatched roof. The HC recently used funds provided by the PDSS (the World Bank health systems development project) to build a veranda, showers, and latrines. Informants reported that the small size of the HC forced providers to treat adults and children, and sick and well patients, in the same area. The Catholic Church recently requested that health officials vacate the building so that the Church could use it for their ongoing activities. At the time of the evaluation, zonal health officials, in conjunction with community members, were planning the construction of another HC. The Kalwa HA has 10 villages, with one HC, one health post. and two iCCM sites. Two of the 10 villages, including Kalwa, where the HC is located, are on the main road. The other villages are in the interior of the HA and are often inaccessible, especially during the rainy season. The HA is vast and sparsely populated. The Kalwa HC, which is located 55 km from Bunkeya, is a small brick building with a mud floor. Originally a house, health zone officials started renting the building in 2008. No renovations have been made since that date. Staff include the IT (A2), an ITA (A2), and two A3 staff recruited locally. Hospitalized patients receiving treatment are in the entrance; across from these hospital beds is a waiting area for sick patients. In 2019, the HC used PDSS funds to build a small annex for sick patients. That same year, community members constructed a second building, which is used as a storeroom and pharmacy. Community members also built a small maternity made of brick. The reference hospital was originally constructed in 1953 and is comprised of numerous units, including a separate ward for children. A generator is run in the evenings and to perform surgery, and solar panels are used at night to light the hospital corridors and grounds. Informants indicated that the building requires ongoing and significant renovations, which are financed through monthly hospital revenue and assistance from the European Union (EU) and PDSS. Although the hospital is meant to serve Bunkeya health zone residents, most patients come from outside the zone. This is due to the configuration of the health zone, which is surrounded by two health zones located in Haut Katanga Province, with villages in the neighboring province far from zonal reference hospitals. The hospital is also known for being less expensive and accepting all patients, with informants indicating that this reflects the approach of the Spanish nuns in charge of hospital administration. Additional information on the facilities’ infrastructure is presented in Table 4.2. Baseline Report 123 Table 4.2.Facility infrastructure, supplies, and medications for child health services based on observations in Bunkeya Kikobe Higher performing HA Kalwa Lower performing HA Reference Hospital Has the health center been renovated/construction added in the last five years? A veranda and latrines were built, and a maternity made of brick and thatched roof. Two small buildings adjacent to the HC were built; one building was constructed by local community members and one was built with PDSS funds. Yes, ongoing renovations are made with funds from the EU and PDSS. Is electricity available in the health center? No, the solar panels were not working. Yes, solar paneling was working. A generator was run in the evening from 18h to 23h. Solar panels were used to light corridors and walkways on the hospital grounds. Is there a separate area of the building where child health services are provided? No, treatment for children and adults was provided in the same room. No, all treatment was provided in the same room. There was a separate children’s ward. Is an infant treatment table available? No No Infant tables were located in the maternity. Is an infant scale available? The infant scale was not working. The infant scale was not working. Yes, an infant balance was working and located in the maternity. Is a salter weighing scale with trousers available? Yes Yes Yes Is a weighing scale (munie) available? No There were two; one was located in the maternity and the other was in the consultation area. Yes Is a height measure available? Yes Yes Yes Are growth monitoring kits available? There were mid-upper arm circumference (MUAC), height measures, and growth monitoring forms. There were MUAC, height measures, and growth monitoring forms. Yes 124 The Impact of USAID’s Integrated Health Program in the DRC Kikobe Higher performing HA Kalwa Lower performing HA Reference Hospital Is sterilizing equipment available? Yes, in the treatment consultation area. Yes, one was in the maternity and one was in the consultation area. Yes Does the health provider have key instruments, such as a stethoscope, thermometer, and timer? A stethoscope and thermometer were available, but not a timer. Gloves were not available. A stethoscope, thermometer, timer, and gloves were available. A stethoscope, thermometer, timer, and gloves were available. Are essential medications available, such as zinc, oral rehydration solutions (ORS), amoxicillin, artemisinin-based combination therapies (ACTs) and other malaria drugs? ACT had been out of stock for several weeks. There were many stockouts, including stockouts of ACT, zinc, SRO. Supplies of amoxicillin, vitamin A, mebendazole, and metronidazole were insufficient. The hospital had these medications, but ACTs, mebendazole, and metronidazole were only available in small quantities even though the hospital had received a delivery from Chemonics just before the observations. Are treated mosquito nets available? Mosquito nets have not been available for several years. The center was out of bed nets. Yes, bed nets were available in the maternity for women who had recently delivered. Are rapid diagnostic kits for malaria available? No Yes, but they were using their last carton. Six and one-half cartons were available. According to the health providers, they would go through this supply very quickly. Are all essential vaccinations available? No. Because the HC refrigerator was not working, vaccines were stored in the Bunkeya BCZS. All childhood vaccines were available and there was a large stock. All childhood vaccines were available in adequate quantities. Is there a refrigerator which is functioning? No. The refrigerator, which operates on solar energy, was not working. A second kerosene- run refrigerator had not been used for more than five years because the HC cannot afford to buy fuel. Yes. A solar-run refrigerator provided by ACCESS was working well. Yes, the refrigerator was provided by the Programme des équipement pour les structure sanitaire (PESS) three to four years before the evaluation. Baseline Report 125 Kikobe Higher performing HA Kalwa Lower performing HA Reference Hospital Are fee schedules posted? Yes, in both French and local language. Yes, in local language. Yes, in both French and local language. Are there educational materials, such as posters on the walls? Yes, including messages about FP, vaccines, and TB. They were informative and esthetically pleasing. Yes, including messages about FP, vaccines, and TB. They were informative and esthetically pleasing. Yes, including messages about FP, vaccines (the vaccine calendar), and transmission of TB. Are there behavior change communication (BCC) materials/aids for the health workers to use? Yes, there were lots of flip charts (PCIME, FP, prevention of malnutrition, postnatal care [CPON]) which were in good condition, but covered in dust and clearly not being used. There were many flip charts (fewer than in Kikobe), but some appeared to be extremely old. Once again, they were covered in dust and clearly not used. No. They do not have aids to help convey messages. Health workers indicated that this would be useful. Is there evidence of activities to discourage fraud and increase transparency, such as a hotline or a complaint box? No, we were told that a suggestion box had been hung in the past but was damaged by termites. No. It was reported that a carton box had been used as a suggestion box previously. The box did not last long. Yes, there was a suggestion box located in a more private location of the hospital. Services Offered Health Centers Health services consisted of treatment consultations for outpatients, 24-hour care for more severe cases, basic surgery, and preventive care. ITs explained that patients were not officially hospitalized, but because they traveled long distances for care, they might be asked to be observed for several days during treatment. Neither HC met the minimum package of health services and personnel, with both lacking trained midwives, adequate numbers of trained A1 or A2 nurses, and a laboratory and lab technician. In general, diagnostic capabilities appeared to be basic. Both facilities had a maternity where traditional midwives assist normal deliveries, and both HCs provided treatment for HIV/AIDS and TB cases. Treatment consultations were available 24-hours a day, with consultation fees for children set at 2500 Congolese franc (FC) in Kikobe and 2000 FC in Kalwa. Nurses followed a 2008 treatment protocol for malaria, diarrhea, and ARI in children, which had recently been revised based on the introduction of new medications, such as ACT. When medications were stocked, health workers were supposed to give ACT for malaria, amoxicillin for ARI, and mebendazole or ORS and zinc for diarrhea treatment. At the time of our study, ongoing stockouts of drugs, such as ACT and zinc, prevented health workers from following treatment protocols. Treatment for malnutrition was not available and only involved counselling. Blood transfusions were provided, but a physician in the reference hospital stated that the blood was not tested for HIV/AIDS. This informant reported that pharmacies also gave blood transfusions without testing for HIV/AIDS, reporting that many children identified as HIV-positive had parents who were HIV-negative and attributing a rise in HIV/AIDS to blood transfusions. 126 The Impact of USAID’s Integrated Health Program in the DRC During treatment consultations with child caregivers, nurses claimed to provide information on preventive care, mentioning breastfeeding, complementary feeding, hygiene, handwashing, preparation of potable water, birth spacing, vaccinations, and the importance of using a mosquito net. Preventive services reported by ITs included prenatal care (CPN), well-baby visits (CPS), FP consultations, and vaccinations, and one IT claimed that the HC provided CPON. Data triangulation showed that the lower performing health center did not offer CPS, although education sessions on child health were held with mothers on a weekly basis. Both HCs relied on water from a local river, which was filthy, with informants reporting that the lack of potable water presented a tremendous health problem in the zone. In the higher performing HC, a quality assessment had been carried out by the Infirmier Superviseur (IS [zonal nurse supervisor]) just before our study; the same assessment had not been conducted in the lower performing facility. In addition, the PDSS is mandated to evaluate quality of care each trimester, as part of the process of determining whether the HC achieved pre-specified targets. Reference Hospital Sick patients were encouraged to first go to the HC (centre niche) [nested center]located next to and linked to the hospital. Providers in the centre niche referred serious cases to the reference hospital, which provided treatment for a wide range of illnesses and conditions, including emergency obstetric care. Nurses followed treatment protocols received more than 10 years ago for common childhood illnesses (diarrhea, ARI, malaria), which were posted in consultation rooms. Physicians followed a therapeutic guide developed years ago by the World Health Organization (WHO), and more recent treatment guides specific to DRC and established by the Congolese government. Informants stressed that it was critical that they and other health workers follow the protocols when treating patients. Informants stated that the hospital lacked adequate diagnostic capabilities, forcing personnel to rely on government laboratories in Fungurame, a mining town 110 km from Bunkeya. Informants described the Fungurame laboratories as unreliable and slow. Informants also stated that the hospital was understaffed. Hospital clinicians said that malaria was by far the most common childhood illness, especially during the rainy season when families lived in makeshift huts located by their fields and did not use mosquito nets. Especially during the rains, informants reported that parents sought care only when the situation was urgent, such as when a child experienced convulsions, and suffered from anemia and required a blood transfusion. We were told that severe malaria cases arrived in waves and at night, often inundating hospital personnel. Hospital informants stated that children were frequently malnourished, and that treatment for malnutrition had not been available since 2018. We also learned that severe pediatric cases frequently suffered from diarrhea, vomiting, and other gastroenteric problems. A full package of preventive care was offered through the centre niche linked to the hospital. The hospital also offered some preventive services for children, such as vaccinations, BCC, and household visits carried out by RECOs. Meetings with parents of hospitalized children were conducted to share information about child health. Hospital staff also worked with RECOs to monitor patients who had recently received hospital treatment and to track cases of HIV/AIDS and TB. Physician informants indicated that a quality assessment of hospital services had been conducted by the DPS several months before our study. Hospital needs included improvements in treatment capacity, diagnostic laboratories, and more capable personnel. Equipment Equipment in the Kikobe HC was basic, with informants reporting that much of it was provided in 2010 by the ACCESS project. Equipment included beds for sick patients, cabinets, benches, a desk for the IT, plastic chairs, a cooler, and a microscope. The solar-operated refrigerator did not function, forcing providers to store vaccines in the BZCS; there was also a petrol-operated refrigerator, but informants reported that they could not afford to purchase fuel. The maternity had a delivery bed, and other beds for women pre- and post-delivery. The baby scale was broken, but a salter scale with trousers and a height measure were available. An adult scale was not Baseline Report 127 available. No bed nets were in stock. The center lacked reporting forms (including for SNIS) and official registers, including a maternity register. Informants said that the center needed beds, mattresses, and equipment for blood transfusions, and mosquito nets, maternity kits, scissors, scalpels, blouses, and a table for newborns. The Kalwa HC had one bed for sick patients, some cabinets, a desk for the IT, a bench for people waiting for treatment, a table where medications were placed during consultations, and a refrigerator run by solar panels. The maternity had a delivery bed, but due to the confined quarters, there was no space at the base of the bed for the birth attendant to stand. There were other beds in the maternity for women in labor or who had recently delivered. The newborn scale was broken, but a salter scale and height measure were available. Two adult scales were available and working. Bed nets were not available. When asked about materials needed, informants mostly mentioned medications. Both ITs reported that, since their arrival at the HCs, they had never received new equipment. When materials or equipment were needed, or equipment repairs were necessary, the ITs contacted the BCZS, which subsequently informed the DPS. The BCZS administrateur gestionnaire (AG [health zone financial and administrative manager]), needed to give approval before repairs were made, even though costs were absorbed by the HC. One CODESA member from Kikobe explained, If the refrigerator is not working, as is the case here, before calling a repairman, we must inform and get permission from the health zone. If the repair does not last, there is little we can do. We cannot make decisions about equipment in the health center because it is under the orders of the health zone. They are our superiors and we must inform them, and they will tell us what to do. Hospital informants stated that the hospital had essential materials for the treatment of childhood illnesses. Equipment was obtained through a range of sources, including the PESS, donors, NGOs, and the Catholic Church, and was often provided randomly as charity and already used. For example, the hospital unexpectedly received a container of secondhand beds, cabinets, and hospital supplies from USAID, which were originally destined for the Kasai region but could not be delivered due to conflict and unrest. The hospital also purchased secondhand equipment, such as a 1997 portable x-ray machine and microscope, from Europe. Informants were unable to confirm whether the hospital met basic equipment standards, reporting that basic was defined according to the context. Hospital needs included diagnostic equipment so that it did not need to rely on the laboratories in Fugarume, a computerized machine to perform bio-chemistry analyses, surgical equipment, a dialysis machine, and a respirator, which we were told would be lifesaving for children suffering from certain pathologies. Informants reported that, although the government had promised to provide a high-quality x-ray machine and a generator, the equipment disappeared after arriving in Kolwezi. Table 4.2 presents a list of equipment, supplies, and medications available in the hospital and HCs based on our observations. Medications The MCZ indicated that 68 medications were included in the official drug package, not including medications provided by the different national government programs. Informants stated that drugs provided by national programs, such as for HIV/AIDS and TB, were received regularly in sufficient quantities. However, ACT, which is known to be an effective, high quality, malaria drug, was never delivered in adequate quantities and was not available in Congolese pharmacies. Therefore, when stockouts of ACTs occurred, which was the case at all facilities evaluated during our study, health workers had to resort to less effective drugs or drugs that produced severe side effects, such as quinine or artemether. At the time of the evaluation, informants stated that ACTs for adults would be available soon; was not clear when the children’s dose would be delivered. During the data collection, informants reported stockouts of many drugs (e.g., zinc, ACT, fanzidar, paracetamol, malaria prophylaxis, mebendazole, ORS), which forced providers to use alternative, less quality therapeutics. ITs indicated that the HAs had not received drugs since July 2019, four to five months before the evaluation. Health providers also mentioned regular shortages of testing kits for such illnesses as malaria, HIV, and syphilis, and for blood typing. The informants stated that if they relied solely on drugs delivered by Camelu (the company delivering medications from the CDR and sub-contracted by Chemonics), they would always have stockouts. 128 The Impact of USAID’s Integrated Health Program in the DRC Informants described Camelu drug deliveries as irregular and occurring every four months rather than the scheduled three months; they also said that orders were not respected, did not coincide with their needs, and were generally insufficient, forcing health providers to purchase medications. The MCZ added that ITs often made irrational decisions when ordering drugs. There was consensus that insufficient medications and supplies, such as kits to test for malaria or to assess blood type, presented a tremendous obstacle to meeting population needs and providing safe and effective treatment. When describing a delivery received from Chemonics at the time of our evaluation, a medical doctor said, If I show you the quantity of what we received, you will see it is disgraceful. A few products, a single grouping kit. One kit. During this period, when we will need to give several transfusions, the kits will last less than three days. That is going to help us with what? A few small cartons of products and bags for intravenous therapy. If we wait for products from the CDR (delivered by Camelu), we will always be out of stock. Before [during the predecessor IHP project] it was a bit regular, deliveries were quarterly. We could count on our orders being filled each quarter. We would place orders; we were sure we were going to have supplies even though our orders were never fully filled. Now we are supplied with products depending on the availability of Camelu. Sometimes I can't understand why they do certain things. I was there to collect the products yesterday, I asked myself how they can give such small quantities to the reference hospital. For the BCZS, we can understand, they receive everything, mosquito nets in large quantities, but there are, for example, liquids. They should provide according to policy. The health centers do not use intravenous therapy, you supply them with liquids to do what? There is a lot that we don’t understand. There was a large quantity of products for the BCZS. But for the hospital there was only a small amount. Health workers reported that when stockouts occurred, they contacted the BCZS to see whether the zone had the drug. If not, ITs traveled to Lubumbashi or Likassi where they could purchase drugs in private pharmacies recommended by the government to sell regulated products in bulk. Some drugs, such as paracetamol, mebendazole, and most antibiotics, were easy to replace, but special imported drugs, such as ACT, were not. Although hospital staff appeared to try to replace drugs before stockouts occurred, the ITs seemed less proactive. Physician-informants suggested that private pharmacies selling unregulated drugs were emerging in larger communities, and it was likely that health workers would resort to the purchase of these drugs. In the lower performing HA, where the HC was located three kms from a large community in Haut Katanga province, health workers provided prescriptions for medications from pharmacies in the adjacent province when stockouts occurred. In addition, the plethora of pharmacies selling unregulated drugs allowed community members living in more populated areas to self-treat before seeking care from trained providers. One hospital clinician said, Bunkeya is full of pharmacies, health posts, all kinds of places. If it is 40, I don’t know how many, so people start there and when things are not going well, that’s when they come [to the hospital]. Up to 2012 or 2013, lots of patients we received were simple cases. A normal malaria case treated in five days, severe malaria and the person gets better and leaves. Up to 2014, it was like this, and then pharmacies exploded. But now we see really serious cases. Because they [the patients] have already been everywhere. We receive serious cases and sometimes after an hour the patient dies. Or the patient is brought to the hospital already dead. Focus group participants mentioned that insufficient medications and drug stockouts were a common problem with HC services, forcing women in Kalwa to buy drugs from local pharmacies. Service Utilization Health worker informants said that the major obstacles to care seeking from formal healthcare, especially from higher-level facilities, were economic constraints, distance, lack of or inappropriate transport for sick patients, and poor roads. During the rainy season, when people live in remote locations next to their agricultural fields, delays were especially prolonged due to the long distances to health facilities. Because residents practice subsistence farming, they lack cash, making it difficult to pay for treatment until after the harvest. As discussed, the proliferation of pharmacies, especially in more populated areas, encourages community members to first seek care from unqualified providers, causing delays in care seeking. (It should be noted that before our evaluation, Baseline Report 129 which occurred before the main harvest, there was a maternal death that the MCZ attributed to a delay in reaching a facility. At the same time, informants reported that police request 100,000‒200,000 FC, an unofficial penalty, from women giving birth at home, thus coercing women to deliver in the facilities.) During focus group discussions, female caregivers claimed to obtain treatment primarily at the HCs, explaining that the HCs were nearby, treatment was effective, and no other formal health services were available, with some adding that HC care increased the survival of their children. Some mother participants from Kikobe also stated that they traveled to the reference hospital, which was perceived to provide higher quality of care. However, several mothers expressed concern that the limited space at the facilities forced sick and well patients to mix. In Kikobe, several mothers highlighted the lack of laboratory facilities to diagnose illness. One participant stated, There is no lab to find out what is bothering the child. They treat the child without examining him [diagnosing the problem], how will the child be cured? Mothers stated that a primary obstacle to seeking facility care was the concern that medications were unavailable; other barriers mentioned included belief systems contradicting biomedical treatment, parental negligence, and lack of money, with mothers suggesting that health workers would not treat on credit. One mother stated, People go to the local healer maybe believing that if I go to the hospital, they will not treat my child…It is money. If you have money you will go quickly to the center and if you don’t have money you will hesitate because you cannot go there with your hands empty, without money they are not going to provide treatment. Another mother said, It is the money. If you don't have money how can you go to go to health center? You will have to remain at home with the disease. As regards traditional remedies, in the higher performing HA, facility- and community-based health workers reported that local healers did not exist, with informants insisting that people only sought care at the HC, a local pharmacy, or the iCCM site. In the lower performing HA, informants cited a preference for traditional care that coincided with belief systems related to witchcraft or herbal remedies as a major barrier to health service utilization, also noting that traditional practitioners treated on credit. These informants described the existence of strong beliefs in witchcraft, which often raised suspicions that somebody had cast a spell on a sick child and commonly guided parents to first or simultaneously seek care from traditional practitioners who were respected for their healing skills. A village leader said, People are attracted to what the traditional healer says. Since we are in a village, people always think that maybe somebody cast bad spirits on my child and then lied by claiming others want to harm the child. It could even be your own brother [who wants to harm the child]. Here a traditional healer [in this case soothsayer] is someone we respect. Everything they say is followed to the T. When the soothsayer bans food, people obey. But the same prohibition given by the nurse will never be followed. A CODESA member said, Here witchcraft is at the forefront of people's minds, if someone gets sick, we first think that witchcraft is involved. When this is suspected, families take the sick child to the traditional healer first… With us the big problem is witchcraft, when the child gets sick, we first think that maybe it's the grandmother who wants to kill my child and directly, instead of going to the hospital, the parents first go to see the traditional healer, thinking maybe he can have a remedy for this. The hospital comes later. Later, the same CHW shared some of his personal beliefs, stating, They [the health facility workers] are good, they are good. But sick people who die in our health area, they are the ones who are possessed by bad spirits. In the hospital, they can treat the illness if it is from God. If someone is already bewitched in their home, someone like that cannot get better [through health center treatment]. 130 The Impact of USAID’s Integrated Health Program in the DRC Hospital informants also said that prevailing beliefs influenced care seeking from traditional healers, with people commonly consulting devins (healers and soothsayers who specialize in treatment related to the supernatural and sorcery) or using herbal therapeutics before seeking formal care, thus delaying care seeking and interfering with effective biomedical treatment. When talking about barriers to care seeking, one physician said, I don’t think that it is just economics, because if someone has an economic problem, we always treat. But there are other factors related to culture, no, not culture, beliefs. This area is known for sorcery and things like that. There is even a proverb in Kisanga that says, ‘People cannot accept [put closure on] a death, unless someone is accused [of the death].’ Before coming here, there are people who come directly, but those who don’t come directly often go to the ‘banganga,’ traditional practitioner. When it doesn’t go well, they come here. Sometimes they come and get the diagnosis here, but the person says, no, I need to see a local practitioner because it is something spiritual, it is something else. Lots of patients at the hospital, a big percentage, take indigenous medicines, a lot. We talk to them; we even look in their bags to find the medications. The last case was terrible. We did a cesarean; they had given her indigenous medicine. When we opened her stomach, we found five liters of black liquid in the intestines. In another instance, where the woman had severe abdomen problems, we operated and found bandages that she had swallowed. I make a lot of noise because it is not okay. The person comes, you operate, the post evaluation is good. The fourth day you see the patient vomiting. These are the problems we have here. Both in children and adults. Another physician said that especially during the rainy season, people delay care, stating, During this period, they only come when they are pushed by disease. For surgical pathologies, they know the diagnosis, but they leave the hospital, stating they cannot do it now. But after the harvest they will come back. There is a difference between the city and the village. Here it’s the bush, I can show you someone who came today with a very big tumor. He went through all the traditional healers, all the charlatans. Yesterday, a woman was brought to me, at the end of pregnancy, her breasts increased in volume, she just gave birth two weeks ago, or three weeks ago. They kept her with the traditional practitioner for a long time, she was anemic and the blood had not yet finished, but she died. So that is how it is in this period, it's the urgency that brings them [to the hospital]. Take all the women, take all the men, lift their shirts off and you will see scarifications everywhere. That is to say that they see traditional healers. It is due to pain that brings them to the hospital…But you will see people taking products. Last week a woman came to the hospital, she was 5 cm dilated, she was pregnant, she left and came back after 20 minutes to give birth. What happened, a serious tear that you cannot imagine. Why take these drugs? They cannot tell you that they are going to see the traditional healer. But that is what happens when they are sick, they go to the traditional healer first. Even if you try to discourage it. When we go to the sick rooms and search their bags, we find the bottles. In the maternity we hear these stories every day. Portions, concoctions. After extensive probing, participants in both focus groups admitted that traditional therapies were widely used, with parents either opting to seek traditional care as a first option or after visiting a biomedical facility. Mothers stated that the perceived cause of the illness determined which approach was appropriate—biomedical or traditional. If the first treatment did not work, it was assumed that the cause was misdiagnosed, and an alternative treatment was needed. One mother from Kikobe said, There is the diarrhea of God and then there is another diarrhea which is of Satan. If you take the child to the hospital and there is no change, you go to see the traditional healer who will give roots, which you prepare and give to the child and the diarrhea stops. Caregivers indicated that traditional healers, who live near their agricultural fields during the rainy season, can refer to a biomedical facility if the treatment they provide was ineffective. Mothers also stipulated that local healers cannot provide certain biomedical care, such as blood transfusions or effective medicines for malaria. Management and Governance Coordination The health zone held biannual board meetings that were presided over by the DPS director. ITs participated in monthly reporting comité de gestion (COGE [management committee]) meetings held in the BCZS during which annual operational plans and monthly activity schedules were monitored. During these meetings, each HA Baseline Report 131 presented data based on a series of health indicators. The MCZ indicated that HA reports were complete and timely but contained many errors. He also mentioned that ITs lacked capacity to use the monthly results to inform HA activities. The BZCS also held weekly coordination meetings with office staff to monitor zonal activities and transfer information. In the higher performing HA, the CODESA members reported periodically attending the monthly zonal reporting sessions. In this HA, it was also mentioned that both CODESA members and RECOs participated in monthly HC data entry and reporting. RECOs stated that on Sundays, meetings were held with the HC nurses to determine weekly workplans and key health messages to deliver. CHWs also claimed to hold community meetings to discuss development strategies with community members. CODESA members and RECOs in the lower performing HA did not participate regularly in monthly reporting meetings and did not appear to attend HC meetings. Village chiefs claimed that they did not participate in coordination meetings, but they might be invited to the HC when a special guest arrived or to be briefed after the IT attended an important meeting. Hospital staff participated in weekly coordination meetings convened by the BCZS and involving senior personnel; they also attended monthly COGE meetings with ITs from each HA. In addition, they attended meetings to review epidemiologic data and care provided to indigent populations. Other meetings may occur with NGO staff or the PDSS coach during periodic visits. Informants said that coordination meetings allowed health providers to share recent information on health problems, treatment practices, cases studies, and lessons learned, and general work challenges. Informants consistently mentioned the importance of exchanging ideas and experiences, which they indicated can help improve work performance and serve as a motivation. Meetings also facilitated the dissemination of health￾related news beyond Bunkeya, such as health directives or problems shared by the provincial or national level. Accountability Mechanisms HC personnel claimed to have set up a suggestion box. In Kikobe, the IT said that the box became infested with termites, and in Kalwa, the box was made of cardboard and got damaged quickly. At the time of the study, both ITs stated that a new, more durable box made of wood was being made by a carpenter. Although the ITs reported that the suggestion box was designed for community members to provide feedbackabout health services, both nurses admitted that the box was generally empty and community members preferred to share feedback verbally. Kikobe informants said that when RECOs received verbal complaints, they informed the CODESA president and subsequently called a meeting with the IT to address the problem. Community members may also share complaints with the village chief; if the problem was minor, the chief tried to address it on his own. Otherwise, the chief informed the CHWs, who subsequently called a village meeting. Informants reported that CHWs were effective in resolving community issues quickly. Only if the problem was serious would a written grievance be submitted to government officials in Bunkeya. In Kalwa, complaints were commonly transmitted verbally to the nurses. The health providers and CHWs used to hold bi-monthly meetings to discuss and address concerns raised by community members, with decisions shared with the population. However, these meetings were no longer held, and feedback mechanisms to address community grievances were not functioning. The reference hospital had a suggestion box that served as a mechanism to receive feedback on hospital care and ways to improve services. One hospital informant said, There are always problems. The suggestion box allows us to talk to each other, to identify issues, and to revise our approaches. The suggestion box was initiated by a project working with PROSANI Plus, but since the start of the PDSS, it received less attention. The Catholic nuns working at the hospital kept the key and oversaw opening the box; the hospital compiled and analyzed suggestions, shared results with health personnel, and took action, as needed. Examples of suggestions included a recommendation to establish a cafeteria, complaints about health provider behaviors, or problems with the hospital environment. Although the hospital informants believed that the 132 The Impact of USAID’s Integrated Health Program in the DRC suggestion box served to identify and address problems, they stressed that people tended to exaggerate. Patients and family members also shared complaints verbally with trusted hospital staff. Everybody agreed that a telephone line to report problems would be well received. We were told that community members liked to have contact with medical personnel, but did not have phones or the means to buy phone credit. Although both HCs were supposed to have official phones provided by the PDSS, one IT indicated that the phone had been stolen. There was agreement that accountability mechanisms were important to identify and address problems. However, informants stated that low literacy rates limited their use and the usefulness of a suggestion box. Informants also expressed concern about encouraging community members to amplify problems or that mechanisms could be used to escalate personal conflicts. One hospital informant stated, It's not a bad idea in principle, but you know, in our community at least since I've been here, I've come up against a lot of hype. Lots of hype. People accuse a lot; they can say anything. So, I don't know, maybe it’s a problem in rural areas, in the city I don't know…But the suggestion box is good, it helped us improve certain things, to consider what people think. A village chief said, Some problems that occur are based on rumors. Sometimes people dramatize negative rumors, they may use a megaphone to diminish the reputation of another person. Where there are people, there is no shortage of conflict. Someone can take a conflict to another level to harm the work of the other person involved. Referral Systems The health zone referral system involved different layers, starting at the community level where RECOs were charged with referring sick community members identified during household visits. If a sick person refused to follow a RECO’s recommendation to get facility treatment, health providers can visit the household. Other levels included iCCM sites (according to the MCZ, more than 12 existed in the health zone and were introduced with PROSANI Plus), health posts, and HCs, with reference hospitals providing the highest level of care. In instances when reference hospitals were unable to treat, patients were referred to more sophisticated hospital care, generally in Likasi or Lubumbashi, but often families refused due to economic constraints. Sick community members generally traveled by foot or bicycle to HCs and relied on motorcycles to get to the reference hospital. Although the zone had an ambulance, patients had to absorb the costs, which were not affordable for most. Moreover, during the rainy season, most HAs were inaccessible by car, making motorcycles the only option. Transport costs by motorcycle from the HCs to Bunkeya ranged from less than US$1 to US$20, depending on the distance. The MCZ considered transport for sick patients as a critical need. Although it is preferable that patients first be assessed and, if possible, treated at HCs, the reference hospital accepted sick patients who went directly to the hospital, with informants indicating that these cases were often extremely serious. Referrals from the HA to the reference hospital were typically made by the IT and involved patients who could not be managed in the HCs. Health workers claimed to be aware of treatment protocols and their treatment limitations. Patients or their family members can also request a referral. Referred patients should be provided with a referral slip; counter referral slips describing the patient’s condition and recommended treatment were provided by the hospital when patients were referred back to the HC. Although one IT insisted that referred patients were accompanied by a health provider, our data suggested that this rarely occurred. Hospital informants claimed that only one of the eight HC head nurses typically contacted the hospital to learn about the status of referred patients. Although reference hospital doctors had received training on referrals, other health workers had never participated in such training, except for RECOs in Kikobe, who reported receiving training sponsored by PROSANI. Hospital informants suggested that economic incentives encouraged nurses to retain sick patients at HCs and, especially for distant and inaccessible HAs, we were told that nurses attempted to treat patients for dangerously Baseline Report 133 long periods, frequently only referring patients when the condition was dire. One reference hospital physician said, References are made, but the peripheral health centers refer fewer cases. They often detain the sick, who frequently arrive in a really bad state, and sometimes they bring them already dead. Because community emergency funds did not exist, it was up to family members to pool resources. Some Kikobe informants said that the IT sometimes assisted with transport costs. Due to poverty, many people did not accept care from a higher level because of fear of the costs. Health Care Financing HCs relied extensively on monthly revenue, with 50 percent allocated to ongoing operational costs and 50 percent going to health personnel. The health zone was supported by the PBF program (PDSS), which is funded by USAID and aims to improve access to affordable healthcare. The PDSS was launched in late 2018, several months later than the planned July 2018 start up, when money for facility rehabilitation and construction was made available and initial performance validations started. Before implementation, an evaluation was carried out to assess patient fees and establish payment schemes. According to informants, the project categorized HAs according to their location and target population, and payment schemes were adjusted accordingly, but with input from community members. The PDSS has a long list of indicators, with corresponding payment schedules varying according to the HA categorization. Each trimester, third-party evaluators were supposed to validate performance indicators in target facilities. Informants indicated that half of the trimester payments were allocated for the operation of health services and the remaining funds were used to supplement health personnel payments. We were told that participation involved endless paperwork, but that the financial rewards motivated health workers to complete forms in a timely and quality fashion. Informants reported that some PDSS evaluators were unreasonably demanding, refusing to pay for certain indicators. However, informants in facilities where payments had been received lauded the approach, stating that it served as a tremendous motivation for health professionals and ensured affordable healthcare. The MCZ was also positive, mentioning that the salary supplement made a huge difference in work performance. Just before our study, a workshop was held in Kolwezi to review and adjust treatment schedules and address ongoing challenges. Subsequently, the PDSS sent coaches to resolve problems identified and claimed that performance validators would be more flexible in the future. With the start of the PDSS, HCs reduced consultation fees to 3500 FC for adults and 2000 FC for children in Kikobe, whereas in Kalwa, treatment for adults and children was decreased to 2000 FC. The majority of informants agreed that the fees were affordable for most residents, encouraging people from even remote areas to frequent the HC, with ITs reporting an increase in service utilization. In Kikobe, the HC had received two or three PDSS payments of around US$1,800 per trimester, whereas Kalwa had received only one payment of US$800. The MCZ speculated that Kalwa was not receiving regular payment due to its low performance score, which he attributed to the use of unqualified staff (non-nurses) and poor community outreach. Another factor was that patients included residents from the neighboring health zone located outside Lualaba province. ITs stated that the PDSS had failed to adhere to the planned schedule of performance validations and payments and was chronically behind schedule. Ongoing medicine stockouts forced HCs to purchase medicines at higher costs and, compounded with the reduced patient fees, impacted the benefits obtained through the PDSS approach. Nevertheless, those receiving PDSS payments were positive about the financial benefits of the approach. The MCZ confirmed that the start has been fraught with challenges, noting that two Bunkeya HAs had not yet received their trimester assistance, that Kalwa had been evaluated and received assistance only one time, and that the BCZS had not received two PDSS payments. In December 2018, the reference hospital received the first PDSS payment for infrastructure rehabilitation, and in the spring 2019, the hospital started receiving financial support based on PDSS indicators. Since then, PDSS 134 The Impact of USAID’s Integrated Health Program in the DRC validation of indicators and subsequent payments were made on a trimester basis, with payments to the hospital between US$16,000 and US$18,000. Under the PDSS, fees for five days of hospital care were 11,000 FC for children and 16,000 FC for adults, which represented a significant reduction in patient costs. Fees included costs for the bed, health services, and consultations, and all medications, except for intravenous therapy, injections, and transfusions, and antibiotics for certain illnesses requiring high doses, such as meningitis. Our informants reported that some payment schemes were poorly calibrated, such as US$.50 coverage for CPN, which was meant to cover various tests (e.g., for HIV, syphilis, malaria), certain medications, personnel and treatment costs, and enrollment forms. Hospital informants also mentioned that for the approach to work, deliveries of medications and tests provided by other IPs must be timely and adequate, with stockouts avoided. A benefit mentioned was that the PDSS targets encouraged facilities to monitor certain previously neglected diseases, with hospital workers referring to active surveillance and follow up of TB patients. The PDSS had a set criterion for indigent community members (e.g., people who are blind, handicapped, homeless, widowed, and old). Neighborhood chiefs and RECOs were responsible for establishing lists of indigent community members, who received free treatment at HCs and paid a reduced percentage (20%‒40%) for hospital care. In Kikobe, the list included nine people. Both ITs reported treating on credit as part of the strategy to encourage sick patients to seek rapid care. However, other informants suggested that ITs were reluctant to accept credit from people who were traveling from distant areas and were unfamiliar. In Kikobe, we were told that family members of patients who cannot afford treatment can sign a contract and pay later or work for the HC in exchange for treatment costs. Although no community emergency funds existed, family members generally pooled resources to pay for the treatment of sick relatives; however, informants added that many poor people did not seek care due to concerns about the costs. Hospital informants emphasized that patients seeking care in the reference hospital were never refused due to lack of money, and that only transfusions must be paid up front and in full, with allowances for other payments made incrementally and after treatment. We were told that some patients fled the hospital in the night without paying for mandatory costs at the time of discharge or before an agreement was made about the payment schedule. Health mutuelles (community-based insurance schemes) were not available, with some informants noting that the underlying concept—paying for something before it happens—was in opposition to cultural norms and might even be viewed as an invitation to bad luck and poor health, and that they would not work in the Bunkeya context. Mining companies used a voucher system to cover healthcare costs for employees and their families, with payments made at the end of each month. No other financing initiatives were in place. Although the MCZ praised the notion of decentralization and the national health strategy, he complained that the zone lacked adequate financial support to operationalize workplans and activities. He said, The needs are enormous, really huge, and all we get is partial support. The needs in terms of infrastructure, we have seen that the government is reviewing how to support infrastructures, but there are so many infrastructures. You have seen health centers made of local bricks. There are needs in terms of equipment. It’s appalling. If you go to the field, you will see that equipment needs are urgent. There is a desperate need for essential and generic drugs. Facility-Based Human Resources The MCZ was transferred to Bunkeya in 2019, eight months before the evaluation; there were also an AG; a zonal nurse supervisor (Infirmier Superviseur [IS]); nurse supervisors for nutrition, HIV/AIDS, and TB; a pharmacist, and an AC (health zone officer in charge of community activities), although this person worked in the hospital fulltime. Both the MCZ and IS lived in Kolwezi, which was more than three and one-half hours away by vehicle, and informants reported that they only visited Bunkeya to participate in special meetings or activities. During our 12-day stay in Bunkeya, the MCZ appeared only on the last day of our evaluation, due to a measles vaccination campaign. The IS was absent the entire period. Baseline Report 135 In each HA, there was an IT (both A2 level) and an ITA (one A1 and one A2 level). Both HCs had A3 staff providing treatment, but they were locally recruited and had received only informal training. The MCZ indicated that government regulations now stipulated that A3 were not qualified to provide treatment. He also said that recruitment of new staff was extremely difficult, with the DPS unable to identify nursing staff to work in rural HAs. After initiating work, nurses typically worked in one health zone, rotating from one HA to another. The MCZ mentioned that individual performance evaluations were rare, and that action was not taken if health personnel performed poorly. The governor was the only one who could fire government workers, which was never done. The MCZ also discussed the rampant practice of hiring family members, which was carried out by people in powerful positions, indicating that this practice also undermined performance in the health sector and in other sectors. Training Informants reported a recent and significant decrease in training, which appeared to coincide with the departure of the predecessor PROSANI project. Topics covered during training sessions formerly offered by PROSANI included HIV/AIDS treatment, reproductive health, protection of mothers and children, health worker participation in community healthcare, FP, infant and child feeding (for CHWs), and gender. Training in 2019 included a regional workshop on the PDSS attended by hospital clinicians, and a provincial-level training on HIV/AIDS offered to ITs. CODESA members and RECOs had not participated in any training for more than three years. The MCZ called for the urgent need to strengthen the capacity of health personnel providing primary healthcare. Supervision ITs stated that supervision of the HCs was frequent (two to three times a month in Kalwa)—although no set schedule was mentioned—and was conducted by the MCZ, the provincial health inspector and the AG. The MCZ mentioned that the lack of transport funds prevented supervision, indicating that the health zone carried out 15 percent to 16 percent of the scheduled supervision visits. The DPS also conducted periodic visits to Kikobe, located on the main road to Bunkeya. Other supervision visits took place, but HA informants could not identify the people involved. Informants indicated that BCZS staff followed specific objectives and that supervision entailed a review of HC registers, forms, and records. At the end of the supervision visit, which typically took several hours, supervisors entered results and made recommendations in a notebook. Supervision was generally limited to a review of services offered at the HCs, with little focus on community activities and CHWs. Informants reported that supervisors never visited communities or households or talked to community members, although PDSS evaluations involved interviews with community members. The MCZ mentioned that supervisors were not adequately trained, mentioning limited knowledge in certain topics, such as pharmaceutical management and primary healthcare. The MCZ said, Primary healthcare has been lacking since 2007 or 2008. There are people who have not been trained for more than ten years. When we send a supervisor to support the facilities and to improve primary healthcare, he too does not know what to do ... [laugh]. Even routine vaccination, the last training in Lualaba dates to 2013. So, you can understand that in terms of quality, what the supervisors do, leaves something to be desired. This is why I am trying to organize training, a briefing for my agents on supervision… We have a lot of people in the health zone who are not capable. Supervision of the reference hospital was frequent and carried out by the different technical programs (HIV/AIDS, TB, malaria) of the DPS and the inspection office. Representatives of the Catholic Church’s Bureau Diocésain d'Œuvres Médicales (Diocesan Office for Medical Affairs or Activities) conducted supervision less frequently. Hospital workers said that coordination was poor, with technical program staff often descending on the hospital at one time, and that this can be followed by long periods devoid of supervision. 136 The Impact of USAID’s Integrated Health Program in the DRC The MCZ was positive about the role the office of inspection played ensuring that health regulations were followed. However, he emphasized that its work was hampered by underfunding, preventing inspection workers from carrying out regulatory supervision as scheduled. Information Available The findings showed that ongoing access to health and medical information was extremely limited. At the HA level, reference books and other written material were not available. ITs reported receiving health-related information through the radio or telephone, and from feedback provided by supervisors. RECOs and CODESA members stated that training was their primary source of information, with virtually no mention of coaching by the IT. They may also get information during HC meetings; however, in the lower performing HC, their participation was limited. CODESA members and RECOs appeared uninformed about the purpose of meetings in the BCZS or the training that ITs had received, suggesting that the ITs shared little or no information. One village chief complained that the head nurse failed to report to him and the general population about special activities, such as training. The reference hospital did not have Internet access, and although the BCZS had a modem and was supposed to receive monthly Internet credit, hospital staff did not use the BCZS system. An association of doctors had recently been established to share case studies and other medical information through the Internet, but the lack of Internet at the hospital made it virtually impossible to participate. The hospital had a small library with dictionaries and electronic manuals. Hospital informants reported that research on numerous tropical diseases had been conducted in conjunction with universities, mentioning studies on trypanosomiasis, filariasis, and schistosomiases, but no studies were done at the HA level. In general, informants viewed research, evaluation, and supervision as valuable methods to increase knowledge and improve their work capabilities. Informants uniformly expressed a need and desire for regular access to updated information. Health Provider Attitudes We administered a series of questions about health worker attitudes and whether provider behaviors changed according to the socioeconomic background, ethnic background, age, sex, or health condition of the patient, especially if the patient had a condition that could evoke stigma, such as HIV/AIDS, TB, adolescent pregnancy, or fistula. Informants working in HAs categorically stated that facility-based health providers were welcoming, sensitive to patient needs, and treated all patients with respect, never showing discrimination toward patients, with some adding that free care was provided to indigent community members. Some informants referred to specific cases when health workers demonstrated compassion by paying for healthcare, providing food, or caring for community members with special needs. Several informants admitted that negative interactions occasionally arose, but attributed these instances to patients who lacked respect or who disregarded HC protocols. Both ITs stressed that it was in the best interests of HC staff to maintain good relations with community members, emphasizing that the aim was to increase the use of the HC and revenue. The Kikobe IT said, Here in the village, if you display inappropriate behavior towards parents [of sick children], you risk that these parents will no longer use the health center. You must know how to collaborate with the community. If not, the community will run away from us. The payment that we receive here in the center, 50 percent is for operations and the other half goes to the health agents. If community members don't use the facility, we won't receive that anymore. The RECO in Kalwa said, Bad behavior is not good. It decreases the number of people. But good behavior attracts people and encourages patients to follow care. Informants said that community members often first communicated complaints of inappropriate behavior to CHWs. Subsequently, a meeting was held with the CHWs and health staff to discuss the complaint and identify needed corrections. Although these meetings were held on an as needed basis in Kikobe, we learned that they Baseline Report 137 had been discontinued in Kalwa. Examples of inappropriate behavior in Kikobe included prioritizing family members to receive treatment quickly or refusing to treat on credit, especially people coming from far away. In Kalwa, CHWs reported that staff sometimes charged patients additional fees beyond the official schedule. In Kalwa, birth attendants had previously been accused of using harsh treatment while attending mothers during childbirth, which the IT claimed had been addressed. None of our informants had ever participated in training related to health worker behavior. Hospital informants stated that staff were respectful and highly sensitive to the needs of the population, emphasizing that the influence of the Spanish nuns bred compassion to ensure that health services were accessible to all and that patients were treated with respect, notwithstanding their background. We were told that the nuns took special measures, such as waiving or covering payments community members could not afford. Although informants admitted that there were occasional negative interactions, they stated that transgressions were quickly addressed. The hospital routinely conducted individual performance evaluations, which entailed an examination of interpersonal interactions. Staff also held regular meetings at which negative interactions and appropriate corrections were discussed and addressed directly. Our observations of health worker-caregiver interactions confirmed that facility-based providers were generally respectful and empathetic. An exception included an interaction between a doctor and a 14-year-old mother seeking care for her infant, during which the provider conveyed distain toward the young mother. Focus group participants agreed that the health workers were welcoming and followed appropriate behavior, unless the patient was unable to pay, as several Kikobe mothers mentioned. Health Worker Motivation HA informants routinely indicated that health workers were primarily motivated by monetary compensation. None of the HC workers received a government salary, with all relying almost exclusively on monthly revenue, and more recently, PDSS supplements. The Kikobe IT said that monthly revenue was about US$480, with half allocated to salaries. In Kalwa, monthly revenue was from US$300 to US$425, with the IT claiming to use only 20 percent for healthcare worker salaries and five percent for CHW payments. One informant stressed the importance of regular drug supplies, highlighting that healthcare workers can only provide treatment and generate revenue when medications were available. As indicated, the Kikobe HC staff received additional funds through the PDSS, and health workers also received US$15 to US$20 during periodic vaccination campaigns, and per diem when they participated in training. The MCZ reported that only eight percent of the zonal health workers received a government bonus, referred to as a prime. Most hospital staff were not on the government payroll, nor did they receive a government bonus. Only two of the four hospital physicians obtained a government salary and a prime. Bonuses were paid in FC, which was experiencing rapid inflation. At the time of the study, physicians getting a bonus received about US$700, whereas nursing staff were getting less than US$100. Hospital staff were also provided support from monthly hospital revenue, which was US$700 for physicians and US$400 for nurses. In addition, the PDSS was supplementing health worker payments, adding from US$300 to US$500 to the nurse’s pay each trimester. Other benefits included free hospital care provided to hospital staff, their spouses, and their children up to age 12, with healthcare benefits limited to the first wife and her children. The hospital also supported education for one child in primary or secondary school. It should be noted that in 2018 and 2019, the Governor of Lualaba instituted a special bonus for government health workers each trimester, with an extra amount given to personnel working in rural areas. The Governor provided the bonus three or four times; our informants said that it was around US$50 to US$150 for physicians and US$20 to US$45 for nursing staff. At the time of the evaluation, this bonus had not been provided for more than four to five months. Clinician informants uniformly expressed discontent about the poor compensation but stated that helping the population provided satisfaction. Knowledge gained through training increased performance and motivation. When asked about other forms of motivation, such as scholarships or transfers to urban centers, HA informants 138 The Impact of USAID’s Integrated Health Program in the DRC indicated that these did not exist, suggesting that there were no opportunities to advance. Although hospital workers were aware of scholarships for advanced education, such as a Master’s in Public Health, they complained that a fair selection criterion was not followed. Informants recommended that the best way to improve health worker motivation was for the government to pay salaries. One physician informant stated, It’s really good to be in our profession, which is noble, but we also need to satisfy our needs. There are certain needs that we cannot do without. Some of my friends from medical school went to other countries, they are recognized for their work, they have bought vehicles and some built houses. Transport is a need, it is not a luxury, communication is a need, and it is not a luxury. These are among the needs of doctors. If we must steal to get a vehicle, then that is a shame. I don't know what the government is thinking! Imagine, a doctor at my age, I have spent how many years in medicine? Twelve years and I haven’t yet built a house, I do not have a vehicle. After how many years? A doctor! It does not make sense. There are those who went to Zambia. The first month they already had a vehicle loan. After two years they started building a house. I will be satisfied when my needs are satisfied. It's very complicated, very, very complicated! I can’t be dissatisfied, I know it’s good, I am very happy when I see someone who was sick in the morning and gets better, I am very happy. But when I see that from year to year, I cannot make investments, you speak only in terms of public health, in terms of hospital treatment. Work without savings, without investment is zero! It is not just the doctors, I’m speaking on behalf of all the staff. Everybody is worn out. This informant pointed out that doctors in urban centers had opportunities to work in several hospitals simultaneously and could increase their earnings, whereas in rural settings, clinicians did not have such opportunities. Health Workers’ Perceptions of Health Service Quality HA informants generally praised the quality of health services and the work of personnel, reporting that many people, even those from outside the zone, frequented the centers and that most patients recovered from illness. Obstacles to providing care included insufficient medications and the fact that the HCs were small, did not meet HC norms, and could not appropriately accommodate patients, with some deploring the mixing of males and females and adults and children. Lack of basic supplies and materials (beds, registers, mattresses, mosquito nets, etc.) was also cited as a limitation, as was the fact that health facilities did not offer transport for referrals. Both ITs stated that staff needed more training, but were unable to specify on what. The Kalwa IT mentioned that he worked with two A3 workers who had limited capacity and that the HC needed more qualified staff. Hospital respondents reported that deficits in laboratory and operating room equipment prevented them from offering critical services and forced personnel to continually seek outside funds. One hospital clinician emphasized that many hospitalized children were acutely malnourished, making it difficult for children to respond well to medical care, and that the hospital lacked supplies to treat malnourishment. Another obstacle related to the distances people had to travel over difficult terrain, especially during the rainy season, often delaying care seeking and causing patients to arrive when they were in a critical state. The widespread use of traditional remedies was also mentioned and that it interfered with effective biomedical treatment. Direct Observations (of Treatment) Direct observations of parents of sick children seeking healthcare showed that the waiting time to see a health provider was minimal. In the Kikobe and Kalwa HCs, parents and sick children waited in an area with other sick patients and where patients were being treated. In Kikobe, patients also waited on the veranda, with many forced to stand due to the lack of seating. In the hospital, patients sat in the corridor outside the consultation area, where benches and chairs were available. We did not observe triage at any of the facilities. Fee schedules were posted in the local language, and in Kikobe and at the reference hospital, also in French. Other information about the waiting area is available in Table 4.3. Baseline Report 139 Table 4.3. Results from observations of health provider-client interactions Kikobe: Higher performing HA 3 providers; 7 interactions observed Kalwa: Lower performing HA 2 providers; 3 interactions observed Reference Hospital 4 providers; 4 interactions observed Average age of providers observed 37 26 36 Title A1, A2, A3 A2, A3 A1, A2 (2), physician Average number of years working at the facility 2 5 5 Training received on childhood illnesses post formal education 2014, malaria (1) 2018, diarrhea (1) None (1) 2014, malaria (1) None (1) 2016, measles, polio, yellow fever, tetanus (1) 2017, IRA, diarrhea, malaria (1) None (2) Waiting area and triage At the hour that treatment services opened, were caregivers waiting to see the health worker? The HC was open 24 hours a day and accepts patients at any time. The HC was open 24 hours a day and accepts patients at any time. The hospital received patients 24 hours a day. Is there a designated waiting area for caregivers and sick children? No. Patients waited in a room where sick patients were hospitalized or on the veranda. Both had benches for seating. No. Patients waited in a small room where sick patients were hospitalized or in the IT’s office. Yes, patients waited in the corridor outside the consultation area, where benches and chairs were available. Was this area separate from the area where well baby services are carried out? CPS was held in a hangar near the health center, which was small and could not accommodate all CPS participants. CPS was carried out in a hangar in the HC compound. At the time of the evaluation, CPS had not been carried out for several months. CPS was held in the HC next to the hospital. Were there seats available and were there enough seats for all caregivers and patients? No, there were only three benches, which could not accommodate everybody, with many caregivers having to stand. No, there was only one bench that seated four people. Others had to stand. Yes, there were adequate plastic chairs and benches outside the consultation area. Was triage carried out to ensure that more serious cases were examined first? Not generally, but in one instance, we observed that a baby with a high fever was examined before others who had arrived earlier. Otherwise, No, health workers saw patients according to the order in which they arrived. Our observers did not see any triage taking place, with patients seen in the order in which they arrived. However, triage may have been done 140 The Impact of USAID’s Integrated Health Program in the DRC Kikobe: Higher performing HA 3 providers; 7 interactions observed Kalwa: Lower performing HA 2 providers; 3 interactions observed Reference Hospital 4 providers; 4 interactions observed patients were seen in the order that they arrived. when patients first entered the hospital. Did caregivers have to wait more than 15 minutes before the child was seen for treatment? No, health workers were available to see patients quickly. No, health workers were always present and if many people were waiting, another worker would assist with consultations. No, patients were seen quickly after they arrived. Was the waiting area clean and orderly? The veranda at the entrance was extremely active. It was generally dirt, with food and paper on the ground. Yes, the waiting area was clean. Yes, the waiting area was clean and orderly. Were there educational materials, such as posters on the walls of the waiting area? Posters on handwashing and a calendar for vaccinations were posted, as were the consultation fees. Some education materials were posted in the treatment area, which also served as the waiting area. Yes, there was a poster on vaccinations in the waiting area. Were sick children (other than those who were seen earlier due to the seriousness of their condition) seen in the order that they arrived at the health center? They generally used a number system to follow the order of arrival, although one day during our observations, numbers were not used. Some parents complained that the order was not always followed. No system was in place to ensure that patients were seen according to the order in which they arrived. One father of a child that others referred to as “commandant” insisted to be seen before other children who had arrived earlier. Caregivers were supposed to follow an order based on when patients arrived. However, there was no way that this system was being formally implemented. Were any caregivers/children sent away without being treated? No, all children were seen. No, all children were seen. All children were seen, even children of patients who did not have money. Health Provider Interactions with the Caregiver and Child Did the health worker greet the caregiver? Yes, all providers greeted the caregivers. The ITA greeted the caregivers, but the A3 provider did not. The three nurses all greeted the caregivers, but the doctor did not. Did the health worker ask for the name of the child? Yes, the health workers asked for the child’s name when completing the patient form. Yes, the health workers asked for the child’s name when completing the patient form. Yes, the health workers asked for the child’s name when completing the patient form. Did the health worker ask questions about the history of the illness, including when it started and In six of seven cases In two of three cases In two of four cases, the health provider only asked when the illness started, but did not ask Baseline Report 141 Kikobe: Higher performing HA 3 providers; 7 interactions observed Kalwa: Lower performing HA 2 providers; 3 interactions observed Reference Hospital 4 providers; 4 interactions observed the signs and symptoms? other questions about initial signs and symptoms. Did the health workers ask questions about the child’s eating behaviors? In four of seven cases; however, in cases where questions were asked, responses were minimal, with little additional probing. In two of three cases No questions were asked about the child’s eating behaviors. Did the health worker ask questions about the presence of fever and the duration? Yes, in all seven cases Yes, in all three cases Yes, in all four cases Did the health worker ask questions about the presence of cough and the duration? Yes, in all seven cases In two of three cases Yes, in all four cases Did the health worker ask questions about the presence of diarrhea, its frequency and duration, and whether there was mucus or blood in the stool? In six of seven cases Yes, in all three cases In three of four cases Did the health worker ask questions about the treatment given before bringing the child to the health facility? In five of seven cases Only in one of three cases In two of four cases Did the health worker determine the age of the child? Yes, in all seven cases Yes, in all three cases In three of four cases Did the health worker determine the weight of the child? Yes, all seven children were weighed. Yes, all three children were weighed. Yes, all four children were weighed. Did the health worker physically examine the child? Yes, in all seven cases, the children was examined. In two of three cases, the children were examined. Yes, in all four cases, the children were examined. 142 The Impact of USAID’s Integrated Health Program in the DRC Kikobe: Higher performing HA 3 providers; 7 interactions observed Kalwa: Lower performing HA 2 providers; 3 interactions observed Reference Hospital 4 providers; 4 interactions observed Did the health worker follow a treatment protocol when diagnosing the illness and providing treatment recommendations? In, four of seven cases a protocol was followed. A treatment protocol was never followed. In two of four cases a protocol was followed. Did the health worker explain the diagnosis? In three of seven cases the diagnosis was explained. Yes in all three cases Yes, in all four cases Did the health worker explain the treatment regimen? In two of seven cases the treatment regimen was explained. In one of three cases the treatment regimen was explained. Yes, in all four cases Did the health worker provide medication? In four of seven cases medication was provided. In the other cases, a syringe was provided and the mother was asked to return later in the day. None of the children received medication; in one case, it was a follow-up visit and the mother already had medication. In only one of four cases did the health provider give medication. In the case of diarrhea, did the health worker provide ORS sachets and zinc tablets? Only in one of three diarrhea cases; the other children did not have diarrhea. N/A N/A Did the health worker provide counseling on feeding the young child? Yes, in all seven cases counseling was provided. In one of three cases counseling was provided; in this case the child suffered from severe malnutrition. In one of four cases counseling was provided. Did the health worker request that the caregiver return to the health facility immediately if illness signs worsened? In three of seven cases In one of three cases All children were under observation in the hospital. Did the health worker ask questions to confirm that the caregiver understood the diagnosis? In two of seven cases In one of three cases In three of four cases Baseline Report 143 Kikobe: Higher performing HA 3 providers; 7 interactions observed Kalwa: Lower performing HA 2 providers; 3 interactions observed Reference Hospital 4 providers; 4 interactions observed Did the health worker ask questions to confirm that the caregiver understood home treatment? In two of seven cases No, in none of the cases did the health worker confirm that the caregiver understood home treatment. In one case; the other children were being treated and observed in the hospital. Did the health worker ask the caregiver whether s/he had any additional questions? In one of seven cases No, in none of the cases No, in none of the cases Did the caregiver ask the health provider any questions? Three of seven caregivers asked questions about the treatment. One of three caregivers asked questions about the treatment. One of four caregivers asked questions about the treatment. Did the caregiver appear to understand the information provided by the health worker? Yes, in all cases. In two cases, the health worker asked whether the caregiver understood. Yes, in all three cases All caregivers were asked whether they understood and responded affirmatively. Did the health provider request that the caregiver bring the child back for a follow-up visit? In six of seven cases, the health worker requested that the caregiver return for a follow-up visit. No, in none of the cases Only in one case, where the child was not hospitalized and was residing at home. Was the session interactive? In three of seven cases the session was described as interactive. In the other cases, the health worker spoke to the caregiver and the caregiver listened. In all cases the caregiver mostly listened. In three of four cases the session was described as interactive. Was the health worker respectful toward the caregiver and the child? Yes, in all seven cases the health worker was described as respectful. Yes, in all three cases the health provider was polite and respectful. In three of four cases the health worker was described as respectful. In the final case, the doctor was judgmental, asking 144 The Impact of USAID’s Integrated Health Program in the DRC Kikobe: Higher performing HA 3 providers; 7 interactions observed Kalwa: Lower performing HA 2 providers; 3 interactions observed Reference Hospital 4 providers; 4 interactions observed the 14-year-old mother why she had a child so young. Was the health worker nice when interacting with the caregiver?* In five of six cases the health worker was described as kind; in one case the health worker expressed disapproval that the child was not wearing underwear. Yes, in all cases the health worker exhibited kindness. Again, in one instance the health worker (doctor) expressed disapproval toward a teenage mother. Did the caregiver appear to be happy with the consultation? In five of six cases the caregivers appeared to be happy with the consultation. In one case the mother was anxious about her child’s health and apparently was not satisfied with the consultation. Two mothers appeared to be happy with the consultation; one mother appeared to be unhappy and extremely worried about her child. In two cases the caregivers appeared to be happy, including one family who did not have money but the child was treated anyway. The teenage mother was not happy. In addition, parents who were told that their child needed a transfusiontransfusion were not happy. Was the consultation carried out in a setting where privacy was maintained? During the consultation, people were talking to the health provider through an open window. The consultation area is curtained off and other people were entering the area during the consultation. In two cases the consultation took place in the area where patients were hospitalized and in the presence of other parents and sick children. In addition, people in the health center courtyard peered through an open window during the consultation. In the third case, the consultation was carried out in the ITs office and in privacy. Yes, only the health provider, caregiver, and child were present in all cases. Was the consultation carried out in a clean environment? In three of six cases the consultation room was described as unclean and disorderly. The space where consultations were conducted was small and difficult to keep orderly. In two cases the environment was described as dirty and disorderly. For the consultation carried out in the IT’s office, the environment was described as clean. The consultation room was described as clean and orderly. Was the consultation carried out in a quiet environment? No, noise from outside the HC was coming through the open window (people were even asking the nurse questions during the consultations). There was also noise coming No, there was lots of noise and it was often difficult for the caregiver to hear the nurse. In three cases the consultation was carried out in a quiet environment. In the final instance, there was a lot of Baseline Report 145 Kikobe: Higher performing HA 3 providers; 7 interactions observed Kalwa: Lower performing HA 2 providers; 3 interactions observed Reference Hospital 4 providers; 4 interactions observed from inside the HC. In three of six cases, the nurse was playing the radio during consultations. noise coming from outside the consultation room. How much was paid for the consultation? Each caregiver paid 2000 FC Each caregiver paid 2000 FC In two instances, the family paid 2000 FC. In one case, the family did not have any money but agreed to pay later. No data were collected on the final case. Providers greeted all caregivers and always asked for the child’s name. Most providers collected background information about the inception of the illness and signs and symptoms, although information on feeding practices was generally not collected, or when collected, was limited. All providers asked about fever, and most inquired about cough and diarrhea. All providers determined the age and weight of the child; only one healthcare worker failed to carry out a physical examination. The observations also identified numerous deficiencies, notably that many health providers failed to: ask questions about prior treatment; follow a treatment protocol; explain the recommended treatment regimen; request that the caregiver return to the facility if signs of illness worsened; provide counseling on child feeding; and ask the caregiver whether she understood the diagnosis and home treatment or if she had any questions. In Kalwa, health workers did not recommend a follow-up visit. Consultations in the HCs were generally not interactive, with most caregivers only participating when asked a question. All caregivers seemed to understand the information provided and most appeared to be satisfied with the consultation. The few exceptions included caregivers who were anxious about their child’s health or the prescribed treatment, and in one instance, a teenage mother who was chastised by the health provider. Privacy was not maintained during consultations in the HCs. Research assistants commonly described the consultation room as unclean, disorderly, and noisy, with health workers in Kikobe listening to the radio during consultations. By contrast, consultations in the reference hospital were done in a private, clean, and generally quiet setting. More information on health worker-caregiver interactions and the consultation environment is presented in Table 4.3. Community Health Services Infrastructure Health Areas According to informants, routine services involved sensitization on health problems and associated prevention and treatment, which were carried out during household visits and community discussions. Household visits were made to assess the health of community members, and to identify and refer sick patients to facilities especially children. Other community activities included stratégies avancées—outreach involving well baby consultations and vaccinations, which were supposed to be carried out one to two times a month in HA villages. Informants also mentioned vaccination campaigns and mosquito net distribution. Radio programs conveying health-related messages by reference hospital health workers were transmitted regularly and designed to 146 The Impact of USAID’s Integrated Health Program in the DRC supplement the community approaches. Religious leaders and CHWs shared health messages during church services. Community health workers included CODESA members and RECOs, who respondents generally described as having similar, interchangeable functions. Critical roles included providing oversight of the management of essential medications, interpersonal relations with patients, and the functioning of the HC. CHWs were responsible for sharing complaints submitted by community members about health treatment or personnel, and to work with facility-based health workers to resolve allegations raised by community members. Although CHWs were supposed to engage in regular community-based interventions, our data showed that there was no set schedule of activities at the HA level. Informants reported that facility nurses also engaged in community activities, such as stratégies avancées, community discussions, and vaccination campaigns. Both ITs claimed to conduct household visits (especially when a sick patient refused to go to the HC and needed additional persuasion), but other informants denied that these visits occurred. In both study HAs, ITs reported holding weekly sessions to share health-related messages with community members and HC patients and caregivers. The HCs, especially in the higher performing HA, had stacks of flip charts covering a variety of topics (e.g., prevention and treatment of childhood illnesses, FP, sexual behavior, and HIV), which they claimed to use when administering health education sessions. However, the flip charts were covered in dust and were clearly not being used. Although CHWs in the higher performing HA reported receiving flip charts to convey messages, they stated that these materials were torn and no longer useable. CHWs in the lower performing HA did not have official educational aids; rather, they used pieces of paper, which had photos of children, to convey messages on such topics as bed net use and hygiene. There was consensus that messages and visual aids, all developed in Kinshasa, had not been changed for many years. Informants expressed concern that messaging lacked innovation and failed to stimulate the adoption of improved behaviors. Informants had never received information through text messaging. CHWs and health workers disseminated health messages during discussion sessions attached to CPN and CPS sessions, although at the time of the evaluation, CPS had not been offered in the lower performing HA for several months. In the higher performing HA, CHWs reported holding informal meetings with groups of mothers or community members in villages during which they shared information on such topics as prevention of childhood illnesses, portable drinking water, handwashing and good hygiene, the importance of using a mosquito net, vaccinations, and critical nutrition practices. They indicated that the IT also periodically participated in community discussions. Although CHWs in the lower performing HA reported previously holding community discussions, they suggested that it was difficult to sustain this type of work on a voluntary basis, and that community meetings were held only during disease outbreaks. Our data suggested that routine activities, especially in Kalwa, were relatively dormant, except when special events, such as disease outbreaks or vaccination campaigns occurred, which often involved monetary incentives for the CHWs. During such events, we were told that information was shared rapidly and extensively, with RECOs using megaphones to disseminate messages. Messages were also shared during church services, either by religious leaders or CHWs, where it was possible to reach large and captive audiences. During epidemics, CHWs also went house to house to identify sick household members and bring them to the HC. Teachers also conveyed health-related messages to school children, who were encouraged to share the information with their parents. The radio was also an important way to share information, especially in the evening when families listened to the radio. Focus group participants said that they obtained health-related information during treatment consultations and CPS, mentioning information about common childhood diseases, hygiene, handwashing, potable drinking water, vaccinations, and child feeding. They confirmed that household visits occurred periodically, often during specific months (probably in preparation for vaccination campaigns) and at times of an outbreak or for a census, but were more frequent in the past. In Kalwa, mothers stated that “foreigners” had recently come to their homes to ask about treatment provided by facility health workers, which we suspect was related to ongoing PDSS validations. Mothers also mentioned that health workers shared messages by megaphone, especially during epidemics. They stated that community discussions occurred on rare occasions, especially when doctors from outside the zone visited during special events, such as during a recent cholera outbreak. Information was also Baseline Report 147 disseminated in churches and schools, and by radio, but they confirmed that radio reception from Bunkeya can only be received in Kikobe and most participants did not have a radio. No participants had a telephone. Both health workers and caregivers reported that there were no community-based organizations involved in healthcare in either HA. Respondents were unaware of the CAC— the village-level community structures designed to organize and provide oversight of community activities as part of the national community health strategy. Champion communities had never been implemented in the Bunkeya zone. Mechanisms to allow community members to participate in the oversight of health services, such as the bulletin communautaire, also did not exist. Health workers said that community members had never been invited to facilities to learn about the availability of services, which was confirmed by focus group participants. Although informants in the lower performing HA mentioned inviting members to visit the newly constructed buildings, the invitation proved problematic because of the expectation that community members would receive some form of compensation. Informants described iCCM posts as sites created for populations living in remote areas with inadequate access to healthcare, at which childhood illnesses, such as malaria, diarrhea, and ARI, were treated for free. However, not all CHWs were aware of the iCCM sites. The higher performing HA had one operational iCCM site located 25 km from the HC. In the lower performing HA, there were two iCCM sites, with one 45 km from Kalwa and relatively close to Bunkeya. Informants reported that iCCM sites were operated by RECOs, who received instructional and practical training at the HCs. RECOs in charge of iCCM were mandated to treat childhood illnesses and make rapid referral of sick children to the health facilities. However, informants suggested that due to the distance, RECOs often saw and treated sick children multiple times, and only if the illness persisted, referred them to the HC. To carry out the work, RECOs were given basic materials, such as a basin for handwashing, a chair, a scale flip charts to convey messages, and essential medications, including paracetamol, SRO, mebendazol, zinc, and ACT for the treatment of childhood illnesses. One IT said that the iCCM sites had antibiotics, such as amoxicillin, but the second IT denied that RECOs were permitted to treat with antibiotics. RECOs overseeing iCCM were supervised by ITs and relied on HCs for supplies, with informants reporting that they received small quantities of medications and frequently confronted stockouts. Drugs were difficult to restock due to the long distances and the fact that HCs may experience stockouts of the same medications. Lack of transport for both CHWs and community members presented a major obstacle to the provision of community health services, especially during the agricultural season when people lived in isolated areas. Kalwa informants admitted that during the rainy season, CHWs did not carry out or reduced household visits, also pointing to the fact that RECOs resided near their fields during the rainy season. The Kalwa village chief said, The problem is where they go, that is what causes the work to stop, especially during the rainy season, the work stops. The IT says the norm is for RECOs to carry out 50 household visits, but during the rainy season it is reduced to 10‒ 15 households. It is difficult to go to the fields to make household visits, somebody who is not paid, you cannot prevent him from going to his fields. He also must work his fields. Reference Hospital Clinicians reported having a formal relationship with CHWs, who in conjunction with the niche HC, worked directly with the reference hospital to carry out active screening for TB and HIV/AIDS, monitor treatment of HIV/AIDS cases, monitor a peer education approach, share information about illnesses identified as problematic and on preventive measures, and target cases identified in the hospital that could benefit from follow-up visits. Identifying TB cases and tracking people with HIV/AIDs falls under the PDSS, with assistance from PROSANI (for TB), and RECOs received compensation for active screening (10,000 FC for identifying a positive TB case, 3000 FC for screening and referring a suspected case that tests negative). We were told that drug resistant cases of TB had been identified in neighboring zones, raising serious concerns about TB in the health zone. Hospital staff analyzed epidemiologic data weekly to identify immediate health problems and to determine key messages to share with parents and family members during bi-weekly meetings in the hospital compound. The information also guided messages conveyed by RECOs during visites à domicile VAD [household visits]. 148 The Impact of USAID’s Integrated Health Program in the DRC Hospital sessions often focused on the prevention of prevalent childhood illnesses, such as diarrhea, malaria, ARI, and typhoid, and the importance of rapid care seeking. In conjunction with the HC, the hospital also held monthly meetings to assess treatment provided to indigent community patients. The hospital was involved in broadcasts of health-related information by radio, which was accompanied by an interactive question-answer session. However, the radio transmission could not be received in the study HAs. As a result, activities implemented less regularly, such as community outreach, were missed because the health workers failed to receive the information shared on the radio. Hospital informants claimed that the most effective way to introduce information quickly was through the RECOs and radio. The hospital’s approach to involving the community was to engage family members of patients in the upkeep of the hospital compound, such as cleaning the pediatric ward or ensuring that water was available in the latrines. Hospital staff held meetings with family members to explain how they expected them to assist. One physician noted a public health saying, which stated, “If you do something without the community, you have accomplished nothing.” We were told that community members had been invited to the hospital on two occasions to learn about the services that were available. System Design Role of CODESA Members ITs and CODESA members reported that the CODESA committee was responsible for supporting the development of the HC; providing community oversight of HC services and the use of materials and medications; and creating a bridge between the population and the health workers. In reality, CODESA members appeared to carry out activities officially delegated to the RECOs, such as sensitizing community members about health issues and carrying out household visits to identify and refer sick community members, especially children and during times of outbreaks. Other activities included participating in the construction of health facilities, weeding around the HC compound, and cleaning the HC. CODESA members in the higher performing HA participated in monthly CODESA meetings and zonal meetings. CODESA members in the lower performing zone, where the CODESA president was also the village chief, appeared less organized and active. One informant suggested that the village chief’s involvement complicated the relationship with the HC and general activities. The MCZ contended that most CODESAs were not functional. In the higher performing HA, there were nine active CODESA members, including one woman who served as treasurer, with one informant mentioning that women were more reliable managing money. In the second HA, the CODESA committee consisted of four members, with no females. One informant explained that few female residents were literate, and women were therefore not eligible. CODESA members had not been formally evaluated and had not participated in any training for several years. Only the HC linked to the reference hospital recently held CODESA elections. We were told that CODESA members received occasional coaching by the IT, especially if an event, such as an outbreak or a new activity, was introduced. Role of RECOs Broadly, the RECO’s mandate was to support health services at the community level and to help manage the HC. Informants described their main role involving household visits to monitor the health of community members and identifying and referring (in more serious cases, accompanying) sick members and pregnant women to the HC, with some describing RECOs as the “entrance” to the health system. One clinician said, You know, community members will not listen to us, but they listen to the RECOs. They are the link between us and the community. They are the ones who can explain things well so that the community listens. In the HAs, RECOs shared health-related messages about hygiene and handwashing, water treatment and storage, the importance of using a mosquito net (especially for children), FP, exclusive breastfeeding, complementary feeding, and malnutrition, at the community level, during CPN and CPS, and in churches. They were also expected to identify children who had missed vaccinations. RECOs were especially active during times Baseline Report 149 of outbreak, when they carried out household surveillance to identify and refer cases, although no follow-up was done post treatment. Materials given to RECOs were minimal; the MCZ noted that RECOs even lacked bags and an umbrella. As indicated, RECOs assisted with periodic vaccination campaigns and, in the high performing HA, they participated in outreach. RECOs were also involved in the distribution of FP methods. Another role was to maintain lists of indigent community members. They also participated in projects designed to upgrade the health facilities, such as making bricks, constructing hangars where educational sessions were held, and building latrines. RECOs were responsible for maintaining notebooks of their ongoing activities and submitting activity reports to the IT at the end of the month. Reports included the number of household visits and sick children identified and collection of surveillance data during epidemics. Information about the number of households that RECOs followed varied from 30 to 50 in the high performing HA, to 10 to 50 in the low performing HA, with each household supposed to be visited at least twice a month. There was general confusion about how many households RECOs should follow and how often household visits should take place. In the higher performing HA, there appeared to be a push to increase household visits, which is a PDSS indicator. Our data suggested that CHWs in the lower performing area were generally inactive, except during special events when money can be made, or during outbreaks when they carried out household visits to identify sick children. The MCZ described all RECOs in Bunkeya as inactive, claiming that they did not even participate in monthly reporting. Informants reported inconsistent numbers of active RECOs, with ITs suggesting higher numbers than other informants. In the higher performing HA, informants indicated that there were anywhere from five to nine active RECOs, of which three were women, whereas in the lower performing HA, there were three to twelve RECOs, with two to five women. Neither HA had RECOs in each HA village, as stipulated by government policy. In Kalwa, our RECO-informant said that when she joined, there were 20 female RECOs, but of the 20, she was the sole remaining female. The IT explained that when they learned that there was no remuneration, they quit. The MCZ claimed that there were more women RECOs than men in the zone. In both HAs, RECOs had not been elected, but had volunteered to participate. Informants stated that RECOs had not received training for several years, with some mentioning since 2016, but had previously participated in training on breastfeeding and alimentation du nourrisson et du jeune enfant [infant and young child feeding], FP, diarrhea treatment, and good hygiene. Although regular briefings on activities and coaching were conducted in Kikobe, in the lower performing HA, the IT appeared to have little interaction with the RECOs. Informants applauded the relations that RECOs had with communities, generally describing RECOs as sensitive to the needs of community members and well respected. The characteristics of a successful RECO included being sociable, non-confrontational, understanding, and a community role model, with one stating “like a pastor,” suggesting that their messages would not be accepted otherwise. In the higher performing HA, we were told that health workers held sessions to talk about the appropriate behaviors that RECOs should have with community members, and if anybody displayed inappropriate behavior, they were counseled. RECOs said that they occasionally interacted with unreasonable community members. When needed, the IT intervened during CPN and CPS sessions by sensitizing community members on the role of the RECOs and the respect they deserved. Informants agreed that RECOs required additional training, with one IT indicating that some did not understand their roles. When asked why the ITs did not train the RECOs, one IT explained that if formal training was held, the RECOs would expect per diem. Coaching had become an alternative, but the MCZ claimed that nobody had received training on how to coach. He stated, In my monthly report for October, we realized that we had deficits in what is called coaching. Well, personally I have never been taught how to coach. I have been taught how to support, how to supervise, but how to coach, I have never been trained. It seems to be a special theme. As we are expected to coach the structures on this or that theme, we thought that maybe what we are doing is not good. We asked the DPS, who are the only ones who have been 150 The Impact of USAID’s Integrated Health Program in the DRC formally trained on coaching, for help. But for three, four, five months we have not been visited by the DPS, we feel abandoned. RECOs claimed that community members believed that the RECOs received payment for routine work and large sums of money when they participated in special activities. CHWs said that they needed uniforms, flip charts, bags, and umbrellas to better carry out their work. Specific Services Offered Mosquito Nets HA informants reported that insecticide-treated bed nets were supposed to be distributed during the first CPN visit and after children attending CPS received the measles vaccination, which was given at nine months of age. Although the ITs claimed that they had been out of mosquito nets for about one month, we learned through other sources that bed nets had not been available for much longer periods (since 2016 in the higher performing HA). In the lower performing HA, health workers distributed bed nets to pregnant women only because CPS was not held. Campaigns involving bed net distribution to households had not been conducted since 2016, and when campaigns had been held, informants said that the number of nets was always insufficient. Informants reported that the bed nets that community members used were old and had holes, and that poverty prevented people from purchasing them. The fact that households in Bunkeya, which are often made of canvas material, were extremely small, made it challenging to set up and use the nets, especially during the rainy season when people live in temporary housing. The RECO in Kalwa said that she did not feel comfortable entering households to check whether bed nets were used. Informants stated that people understood that mosquitos posed a major health threat and recognized the importance of using a net for protection. Focus group participants in both HAs claimed that bed nets had not been distributed for about three years, with most stating that they no longer used a bed net because they were torn. Mothers reported that bed nets had previously been distributed during CPN, CPS, and after delivery, and there was also mention of campaigns. Although most participants claimed that residents who had received bed nets used them, there were some reports of bed nets being sold to fishermen. Hospital informants indicated that 75 percent of all patients tested positive for malaria, and that malaria was especially lethal during the rainy season, when parents brought their children for care when they were dangerously sick and required a blood transfusion. Hospital informants stated that the niche HC regularly distributed nets during CPN and CPS, and that RECOs working in the town of Bunkeya conducted household visits to assess whether and how bed nets were used, especially when epidemiologic data showed surges of malaria. Vaccinations Vaccinations were provided routinely and during campaigns. In the high performing HA, routine vaccinations were given in the HC once or twice a month during CPS for residents living near the HC and in three surrounding villages. Twice monthly, health workers provided vaccinations during outreach activities in four HA villages. Because CPS was not being held in the lower performing HC, vaccinations appeared to be offered only sporadically during special sessions when health workers requested community members to bring their children to the HC, with CHWs reporting that it was difficult to mobilize residents who resided next to their fields. The Kalwa IT claimed to carry out outreach to HA villages occasionally, depending on the availability of transport money, stressing that some villages were extremely far away and only accessible by motorcycle, which had to be rented for 25,000 FC per day. Informants from both HAs reported that vaccination campaigns were held at least twice a year when children were vaccinated for vaccin antipoliomyélitique oral [oral polio vaccine]and vaccin contre la varicelle [chicken pox vaccine], with the aim of covering the entire HA. RECOs and CHWs played a critical role, dividing into teams to cover different parts of the HAs. Given the limited number of active CHWs in Kalwa and the large geographic Baseline Report 151 area, vaccination teams could not reach all community members, explaining the low vaccination coverage in Kalwa. During campaigns, the administration of the oral VPO was primarily done by CHWs, whereas facility￾based health providers, including hospital nurses, administered the measles vaccine, which was injected. The CHWs oversaw the mobilization of community members with the help of village criers. CHWs reported traveling long distances (more than 20 kms) by foot to villages. In Kalwa, they also used bicycles for transport. Vaccinations were always free and stockouts were not reported. The MCZ said that IPs had decreased funding for campaigns, impacting on the motivation of health workers and coverage. The informants emphasized that there was extensive community resistance to vaccinations, especially among people living in more isolated areas who were less educated and less exposed to information. We were told that villagers, more commonly men, contended that vaccines caused illness, with some reporting that campaigns were part of a strategy to increase the use of the HC and HC revenue. The fact that vaccines were free appeared to raise skepticism. One village chief said, They refuse, stating that the vaccines given for free cause illness in children and push people to come to the hospital. They describe it as a business, claiming that the vaccines manifest fever in children, forcing them to come to the hospital to pay for health services. Several informants said that village residents often fled when campaign teams appeared. A Kalwa RECO said, When we arrive in villages, as soon as they see that it is the vaccine team, they flee to the fields or tell the children to hide by the river. We can remain standing for a long time, and if we ask questions, parents or their children will answer that we and our children will not be vaccinated because vaccines are where there are diseases, and what causes diseases are vaccines. Most of them eventually come out, but the others who say it is the vaccines that give disease, flee. When we return to the health center, we inform the nurse that so and so fled, and refused vaccination. Informants gave accounts of men physically assaulting CHWs or chasing them away. They reported that certain churches, including the Protestant denomination called Black Church (église de noir) and Jehovah’s Witnesses systematically opposed vaccines. In Kikobe, a RECO talked about trying to educate church goers, stating, We go to church. Everyone is there, all RECOs, to talk about vaccination. Most people refuse to have their children vaccinated, they say that our children are not sick, and we come with our things [vaccines] to make our children sick. We really have difficulties; they call us the soldiers of the nurses who give us drugs to give to our children. I personally have been hit, it was in a field and by a father who did not want his child vaccinated. A CODESA member from Kalwa said, Sometimes during vaccination campaigns, some parents chase us, saying that vaccines cause disease. Despite this, we educate them about these things. And, if the community supervisor intervenes and the parent continues to resist, he will take the child by force. This is how vaccination takes place. Mothers participating in focus groups said that vaccination campaigns were held, with women in Kikobe better informed about eligibility, frequency, and the type of vaccines administered. Participants indicated that vaccines were given to protect against illness and to decrease the virulence of disease. Caregivers confirmed that some people refused vaccines by hiding their children, and sometimes they insulted the health workers, claiming that vaccines caused fever and sickness. Other reasons for refusal were that churches opposed vaccines or parents did not understand the importance or held opposing belief systems. A woman from Kalwa said, Some people hide their children due to the belief that if a child gets vaccinated, he will experience blood loss and the parents will not have money to pay for hospital treatment. Another respondent said, There are times that community members insult the health providers and even grab them by their clothes, stating ‘I refuse that [the vaccination] for my children and the others refuse also.’ A Kikobe mother said, 152 The Impact of USAID’s Integrated Health Program in the DRC There are some parents who refuse that their child be vaccinated, saying that if they are vaccinated their temperature will increase and the child will get sick. During campaigns, vaccines were administered in households, which was considered more convenient and safer because children were not forced to interact with other children and possibly be exposed to disease. Caregivers from the higher performing HA mentioned that routine vaccinations were also provided during CPS, but women living in the lower performing HA claimed that the frequency of CPS was irregular, and even when held, many women did not attend, especially after the child received the measles vaccination. Women from the higher performing HA reported that CPS was offered once a month and attendance appeared regular up to the time the child was two years of age. Community Worker Motivation Most CODESA members and RECOs stated that they were motivated by the small sums of money they received from participating in periodic vaccination campaigns, typically US$5 to US$15. Training also presented opportunities for remuneration, but because PROSANI was no longer active, no training had been held for years. Even when training was conducted, our data showed that the CHWs did not receive official per diem rates. Under the PDSS, CHWs were supposed to be paid US$1 per household visit, but the RECOs stated that they were not appropriately compensated, with one IT giving a few dollars to RECOs for VADs at the end of each month. The IT in the second HA claimed to give a small monthly payment, but the CHW informants maintained that they were not remunerated. In Kikobe, when RECOs participated in outreach, they were not given the official transport money. Informants emphasized the injustice of not being paid. One RECO said, Honestly, we are treated badly, you cannot imagine! During vaccination campaigns, we are on foot, walking distances up to 20 km. At the same time, CHWs mentioned a desire to assist community members, as one RECO explained, I am not motivated in terms of money, but what pushed me to become a RECO is compassion for the community and the desire for their well-being. Living around death all the time is not good. Another RECO said, We are there to serve our people. It is not for money. What drives us is to see the community in good health. We are there for the people. We do not work for money and we are not formally engaged. Another maintained that holding a recognized job brought pride and esteem compared with manual labor, such as farming or pounding grain. This RECO expressed the “love” she had for her job: There is not so much (financial) motivation, but I am used to that and I like this work. We work like this, if there is a benefit, I will receive it, and if not, it is the work that I love. We know that it is voluntary, if we will gain something or if we gain nothing, we work only out of love. One RECO highlighted increased knowledge about disease as another benefit and source of motivation. Another CHW believed that she would be compensated sometime in the future for her years of hard work. USAID Projects in the Area Informants reported a disruption in the provision of medications between the present USAID IHP project and the predecessor project, locally referred to as PROSANI. At the time of the study, most informants indicated that the current project was only supporting the provision of medications, especially for malaria, HIV/AIDS, and TB, and for supplies, such as testing kits for malaria or to assess blood type. At the HA level, there was confusion about USAID IHP’s role and its relationship with Camelu. From the BCZS standpoint, PROSANI had not started. The MCZ stated that support had been far less than given in the 2019 operational plan, which involved extensive input from USAID IHP staff. We learned that many promised activities were not being executed (e.g., support for monthly transport of medications from the Baseline Report 153 zone to the HA, iCCM, evaluation and revitalization of CODESA members, reestablishment of community participation, training) or that support was much less than planned (e.g., payment for one supervision visit rather than regular supervision, fewer special campaigns and activities for FP, CPN, and TB). Along with government failure to fulfill promises, this made it difficult to implement the operational plan. The MCZ also expressed frustration that USAID IHP did not provide money and that each expenditure had to be justified. He indicated that during DPS meetings, the MCZ interacted with partners, but once they were in their zones, there was no direct communication, making it difficult to understand what assistance to expect and to plan for activities. The MCZ added that it would be helpful if partners, including USAID IHP, had more of a presence in the field. Informants reported that the previous provision of medications and medical supplies had been more regular and that larger quantities were given. Many noted that PROSANI activities had dramatically decreased compared with prior years when diverse training for facility-based workers and CHWs was offered. Other key activities, such as supervision, M&E, provision of medical equipment, forms and registers, WASH, and financial support, had been discontinued in 2018. One hospital informant stated, This year PROSANI is not active as in the past. It’s as if PROSANI experienced a metamorphosis. If I’ve heard about PROSANI, it’s regarding drug supplies, that’s all for this year. When we had PROSANI, PROSANI was here every day. There were PROSANI supervisions to monitor HIV cases, they monitored assessments. We went to trainings, this year not at all. Now it is as if PROSANI is not there, PROSANI seems to have disappeared, we haven't heard about PROSANI for a while. Some referred longingly to former USAID projects, especially ACCESS, which assisted with the construction of HCs and distributed equipment, such as refrigerators and solar panels. Most informants appeared to be unaware of the USAID IHP mapping exercise, which had been administered by health zone staff and was likely assumed to be part of routine supervision. 154 The Impact of USAID’s Integrated Health Program in the DRC Qualitative Baseline Report, Health Zone of Kasaji Background Information This research was conducted between November and December 2019 in Kasaji health zone, which is comprised of 312 villages and 26 HAs, of which 24 HAs were functional, having medications and trained staff available to provide basic treatment. The health zone is isolated and difficult to access, especially during the rainy season when holes several feet deep form on the main road between Kolwezi (the capital of Lualaba) and Kasaji, making sections of the road impassable. The zone is vast, with the most distant HA 133 km from the central zonal offices. Data were collected in two HAs—Methodist and Lueu. In each HA, we conducted in-depth interviews with an IT (head nurse), a member of the CODESA (health development committee), RECOs (CHWs), and a village chief or a village chief representative. We also carried out in-depth interviews with a physician working in the reference hospital and the AG (health zone financial and administrative manager), and the acting health zone medical officer (MCZ) in the central zonal office (BCZS). At the time of the evaluation, the MCZ was in Kinshasa working on a master’s degree at the Kinshasa School of Public Health. We therefore administered the key informant interview to the MCZ in Kinshasa. We also conducted observations of the health facility infrastructure and interactions between health workers (4) and caregivers of sick children (13). The average age of in-depth interview informants was 46 years and the majority (10 of 11) were men. Both ITs had A2 level training. CODESA members, RECOs, and village chiefs had 17 years of schooling, on average, whereas the two doctors had 18 years of training. Informants had eight years of work experience in the same position, on average. All informants participated in other work, most often agriculture. Some worked as instructors at a primary school or a university or as religious leaders. The informants were Protestant (8) and Catholic (3). Informants had seven people living in their households, on average. We carried out focus group discussions in each site, with 12 caregivers of children under five years of age participating in each session. A summary of the characteristics of in-depth interviews and key informants is shown in Table 4.4. Table 4.4. Background information collected from in-depth interviews and key informants in Kasaji Variable In-depth interview informant (11) Key informant (1) Average age (years) 46 45 Average years of education 17 18 Sex - Male - Female 10 1 1 - Religion - Catholic - Protestant 3 8 1 - Average years of experience in their roles 8 7 Participation in other work* - Farming - Teaching - Nun - Preacher 5 3 - 1 - 1 - - Baseline Report 155 Variable In-depth interview informant (11) Key informant (1) - Judge for the village chief - None - 2 - - Average number of household members 7 7 Description of Facility-Based Services Infrastructure The Methodist HA HC was in the city of Kasaji and was less than one km from the BCZS. Although originally affiliated with the Methodist Church, the HC was government operated. The HC was built between 1984 and 1985. It was a small, narrow building that included a maternity and a laboratory. Due to limited space, treatment for adults and children was given in the same consultation area. Informants reported that the HC required extensive renovation. The Methodist IT said, Our health center needs more than a simple renovation. Everything is old. You can see it for yourself. We don’t have a waiting room, we don’t have a pediatric room, and we have many kids here, we have many sick patients. The maternity ward is not big enough. We don’t have any beds, no mattresses, we should have bed sheets, and we don’t have a clean room for the lab. The pharmacy itself does not meet required standards, and there is also the roof–there are many problems. Our needs go beyond renovation. A month before our assessment, the HC electrical cables were stolen, and at the time of our evaluation, electricity was not available. Before this, electricity was limited and based on a government schedule. The HC had a well that provided water for sick patients and women delivering in the maternity. HC staff included an IT (A2), an ITA (A2), and a nurse (A3) who managed the pharmacy, and a laboratory technician and five traditional birth attendants. Lueu HA had a reference HC located seven kms from the Kasaji BCZS and two health posts, 11 and 8 kms from the HC. Only one village in the HA was outside the city limits of Kasaji. The health facility was originally constructed in the 1940s when it served as a hospital for Belgian missionaries. When the missionaries left, the Catholic Church took over hospital management. The HC was comprised of four buildings, including a maternity and two operating theaters. It had the capacity to hospitalize 100 patients. Child consultations were carried out in a separate treatment room. Solar panels functioned and provided a regular source of electricity; Lueu also had access to electricity from Kasaji. Although the center was government run, the Catholic Church provided emergency funds during times of crisis. Many informants suggested that the building was in disrepair, indicating that the only renovations carried out in the past five years involved repairs to the roof of the main building. A CHW said, There was some renovation, but it wasn’t enough. You can see how dilapidated the building is. Although we changed the roof tiles to metal sheets, there are cracks everywhere. We put some metal sheets here and there, but there is no ceiling. There are gaping holes that allow dust and hot air in. It would be good to have a ceiling. Also, we no longer have windows and there are no functioning latrines. However, the MCZ described the Lueu center as well equipped and organized and that it could be used as a referral hospital. Staff included physicians (2), one of whom was the managing director, an IT (A2), an ITA (A2), a laboratory technician, a nun in charge of the pharmacy and the cash register, an AG, and several nurses. The reference hospital was a large hospital built between 1943 and 1945 and comprised of 10 to 11 buildings, including pavilions for internal medicine, surgery, gynecology, obstetrics, pediatrics, and childbirth. The hospital also included a HC (centre niche), a private clinic, a laboratory, and a chapel. Electricity was available 24 hours a 156 The Impact of USAID’s Integrated Health Program in the DRC day. Originally a denominational hospital of the Ngarengaze church, it became recognized over time as a government referral hospital, although the facility was still linked to the church. The MCZ mentioned that the hospital had received extensive support from the provincial government, which had recently provided two new vehicles, funds to build a mortuary, and support for vulnerable patients. Hospital buildings had been renovated in the last five years and the MCZ said that the facility’s structures were in good condition. The reference hospital employed 102 workers, of which 80 percent had administrative or support roles. Clinical staff included 20 nurses and two physicians. The MCZ indicated that the hospital needed more physicians, but that the recruitment process was difficult due to the involvement of the Ngarengaze church, which must approve the hiring of medical doctors. Additional information on the infrastructure at the facilities is presented in Table 4.5. Table 4.5. Facility infrastructure, supplies, and medications related to childcare based on observations in Kasaji Methodist Health Center Lueu Reference Health Center Reference Hospital Has the structure been renovated in the past 5 years? The health center was generally in good condition, but needed some renovation. The building was very narrow and did not accommodate patients. The health center was renovated in the past five years. Renovations involved painting the facility, replacing the main door, and roof repairs. Some buildings were renovated in the past five years and walls were painted. Is electricity available in the health center? The health center power cables were stolen about one month before the evaluation; the HC did not have any electricity. The refrigerator was run on solar power. The center had electricity, but there were frequent power cuts. Electricity was available 24 hours a day. Is there a separate area of the facility where child health services are provided? There was no specific area dedicated to services for children. The health center was small and narrow, and did not have adequate space to have a specific space for child services. There was a whole room dedicated to child health. There was a pediatric ward. Is an infant treatment table available? No, there were no funds and insufficient space for an examination table for babies. Yes, in the room where child services were provided. Yes, in the pediatric ward. Is there a baby scale available? No, there was not a functional baby scale. Yes, there was a newborn scale in good condition. Yes Is there a Salter scale with trousers available? Yes Yes Yes Is there a munie scale available? No There was one in one of the operating theaters. Yes, and there were a few in stock in the pharmacy. Baseline Report 157 Methodist Health Center Lueu Reference Health Center Reference Hospital Is there a height measure available? No Yes, in the consultation room for children Yes, in the outpatient consultation area and maternity ward. Are growth monitoring kits available? There were no growth charts available; the provider was unaware of how to use growth charts. Yes, growth charts were in the examination room. There were growth monitoring charts, height measures, and weighing scales. Is sterilizing equipment available? Yes Yes There was an autoclave sterilization system. Are there key instruments, such as a stethoscope, thermometer, and timer available for the health providers to use? Yes, and all were functional. Yes, and all were functional All these instruments were available in each ward. Are essential medications available, such as zinc, ORS, amoxicillin, ACT, and other antimalaria medicines? Yes, but not in insufficient quantities. All medicines were available in the HC pharmacy. Available in sufficient quantities in the pharmacy. Are insecticide￾treated mosquito nets available? Yes, but in small quantities Yes, but mosquito nets were insufficient. Bed nets were used on all patient beds. There was a stock of nets. Each hospital bed was covered with a net. Are there rapid diagnosis kits for malaria available? Yes, but in insufficient quantities. Yes Yes, and in sufficient quantity. Are all essential vaccines available? Yes, but they were stored in the BCZS. There were none at the HC when we conducted the survey. They were stored in the BCZS and needed to be collected two days before the next CPN/CPS consultation. All vaccines were available, except for the vaccine for rotavirus because it had not yet been introduced. Is there a refrigerator which is functioning? No. They used the BCZS refrigerator to store vaccines. Yes There was a solar￾powered refrigerator, but it had not been working for a month. 158 The Impact of USAID’s Integrated Health Program in the DRC Methodist Health Center Lueu Reference Health Center Reference Hospital Are fee schedules posted? Yes, they were displayed in the waiting room and the examination room. Yes, they were displayed on the veranda where patients wait for care. Yes, they were displayed in the waiting room. Are there any educational materials, such as posters, displayed on walls? No No Yes, there was a poster on TB transmission. Are there BCC materials/aides for the health workers to use? Yes Yes No, because it was a reference hospital focusing more on curative care. Is there evidence of activities to discourage fraud and increase transparency, such as a hotline or a complaint box? No No There is a suggestion box. Services Offered Health Centers Health services in the Methodist HC included outpatient treatment, 24-hour observation of patients with more serious conditions, basic surgery, and preventative care. As a reference HC, Lueu offered a broader package of complementary activities, including treatment for severe acute malnutrition and surgery, which was carried out in two operating theaters. Each HC had a laboratory and the facilities were open 24 hours a day. Providers followed treatment protocols introduced by a former World Bank initiative called Projet d'Appui au Renforcement du Système de Santé (PARSS [Health System Support and Strengthening Project]). The evaluation team found treatment protocols available in both HCs, with abbreviated versions displayed on walls where child consultations were conducted. However, both ITs said that health workers frequently failed to consult the protocols when administering care. One IT stated, You understand that humans make mistakes, at times some providers do not follow the protocol. Instead of following the norm, a provider may do something else. Let me give you an example; a sick child comes in, he does not meet criteria for acute malaria, but the provider sees that the child is fatigued, and he decides to give a blood transfusion or he gives quinine, which should only be given if the child exhibits specific danger signs. At times, the treatment protocols are ignored. Baseline Report 159 Although the health zone officials emphasized the importance of following treatment protocols during supervision visits, the MCZ indicated that many health workers lacked basic training, which he linked to deficiencies in treatment practices. Nurses reported that treatment for simple cases of malaria involved the drug Coartem. For severe malaria, quinine was administered and sometimes blood transfusions were necessary, which was an old treatment protocol. As for diarrhea, zinc was provided, and if the child was dehydrated, oral rehydration solution was given. In addition, children with diarrhea were tested for worms, and if worms were detected, mebendazole was provided. In cases of pneumonia, injectable antibiotics were typically administered. Challenges in treating children mentioned included drug shortages; poverty, which prevents families from paying for certain treatment and drugs; low quality treatment regimens provided by the national malaria program; and resistance to malaria drugs. When talking about treatment provided by the national malaria program, one IT said, We can’t cure malaria with Coartem only. A good treatment must be given with other products. Right now, the treatment against malaria does not include all the medicines that should be administered [only Coartem]. A child who has malaria is given Coartem with amoxicillin. But other medicines must be paid for. Because the population is poor, they cannot afford to pay for the other medications. Which means that the next time the child has malaria, they will think that the treatment was not effective and the parents won’t come back. Preventive services mentioned included CPN, CPS, CPON, distribution of mosquito nets, and vaccinations. Informants reported sharing preventive messages with caregivers during treatment consultations for children related to diarrhea prevention, healthy diets, the importance of maintaining a clean compound and latrine, handwashing with soap, and the use of insecticide-treated bed nets. Focus group participants said that HCs provided treatment for common childhood diseases, such as malaria, diarrhea, cough, and typhoid fever, blood transfusions and intravenous therapy, labor and delivery services, and laboratory examinations. They also mentioned that CPN, CPS, and vaccinations were offered and that centers distributed mosquito nets impregnated with insecticide. Reference Hospital The reference hospitaloffered a wide range of basic curative care for children, including care for HIV-positive patients. The reference hospital had a treatment unit for severely malnourished children requiring emergency care and hospitalization. The physician interviewed indicated that the Kasaji population was extremely poor and had limited access to healthcare, stating that sick patients often delayed care-seeking until the situation became complicated and dire. He said that patients often arrived at the hospital without money, and that they often could not procure everything needed for effective treatment, stating, First and foremost, is the lack of resources. Antibiotics are expensive. A small bottle of Amoxi (amoxicillin) costs 1,000 FC, and maybe you need 7 or 8 bottles; you ask the mother to go pay, and she doesn’t have the means. In the case of pneumonia, you must do an x-ray to see how the lungs are affected, and the x-ray costs 15,000 FC. She’ll tell you, ‘I don’t have that.’ Sometimes, we must make do with what we can do in the clinic. If you carry out an exam and the results suggest that it is pneumonia, then you conclude that is what it is. Why? Because they can’t afford to get an x-ray to confirm the diagnosis. Preventive care was offered in the niche HC, which was under the same administrative and medical structure, and was located in the hospital compound. The niche HC offered a minimum package of curative care and a range of preventive services, including CPS, CPN vaccinations, FP, and treatment for children with moderately severe malnutrition. When sick patients went beyond the HC capabilities, they were referred to the hospital. Equipment Methodist HC had equipment to provide basic child services, including benches for sick patients waiting for treatment and a table used to assess patients during treatment consultations. The center had salter scales with 160 The Impact of USAID’s Integrated Health Program in the DRC trousers but did not have a newborn scale; the height measure was broken. At the time of the evaluation, the center did not have growth charts. There was sterilizing equipment, a stethoscope, thermometers, a timer, registers, kits for the rapid diagnosis of malaria, and bed nets, but in small quantities. In the maternity ward was a delivery bed and three beds for women in labor and after delivery. The HC did not have a refrigerator and, as a result, vaccines were stored in the BCZS, which was only 800 meters away. A water well was next to the HC. Informants reported that the HC lacked a lot of materials, emphasizing the need for a patient observation bed and a refrigerator. The Lueu reference HC had benches for patients waiting to be consulted, beds for sick patients under observation, a cabinet to store drugs, and registers to monitor treatment and preventive services. There was a treatment table for babies, an infant scale and salter scales with trousers, a height measure, and growth monitoring kits. Other basic equipment included sterilization equipment, a stethoscope, a thermometer, a timer, kits for the rapid diagnosis of malaria, and bed nets, which informants stated were in insufficient quantities. In the maternity, there was a delivery bed, and several beds for women in labor or postpartum. A solar operated refrigerator provided by the Global Vaccine Alliance in 2017 was used to store vaccines. Informants in Lueu could not recall the last time they had received medical equipment, underlining the need for an operating room table, sciatic lamps, surgical kits, and another refrigerator. The reference hospital had the same basic materials needed for child consultation as found in the HCs, but in larger quantities. Unfortunately, the evaluation team did not collect detailed information on more sophisticated equipment. The MCZ indicated that the hospital had benefited from support from the provincial government, which provided an ambulance (although it had been out of order for several months), a new Land Cruiser, and a mortuary with a functioning cold room, and extensive funds for ongoing hospital operations. Regarding hospital needs, the MCZ mentioned that the hospital lacked an intensive care unit. Our hospital informant reported a need for beds, mattresses, wheelchairs, stretchers, an x-ray machine, an ultrasound machine, an electrocardiogram machine, and an aspirator. When asked how the facilities procured equipment, most informants reported that requests were made to the health authorities, including the BCZS, provincial DPS, or the national MOH, or the provincial governor. Facilities with religious affiliations can requested religious authorities to procure equipment. Administrators can also reach out to national and international development partners to assist. A reference hospital doctor noted, When we need equipment, the first thing we do is write a memo to the Ministry of Health, or the provincial governor, since it is a public facility. If we work with partners, people who support us, we don’t hesitate to ask them for help. Another informant from Lueu said, We buy equipment here and there. When we arrived here, we found that a lot was lacking. But with the help of PDSS [the World Bank-supported PBF project], we are currently buying equipment here and there, but it’s not enough. We need a lot more. If another partner can help, it will be welcome. Informants said that equipment repairs did not present a challenge if the facility had the financial means or human resources available. Otherwise, the facility had to wait until it received support from a partner. Respondents agreed that maintenance of equipment was challenging. A hospital clinician stated, Repairs depend on the resources we have; if the cost of a repair is not very high, we use our petty cash, we repair it, and we move on. But if it goes beyond the scope of our skills, for instance the ambulance over there, which has four flat tires, a broken engine, and a broken injection pump‒-this is out of our control. We wait to see if we can miraculously find a donor. Medications Up to 2014, the health zone received medications from PARSS, an implementing partner supported by the European Union. Between 2014 and 2017, the MCZ reported that the health zone did not have a partner to assist with the provision of medicines. Around 2017, the national malaria program began to supply health zones Baseline Report 161 with free antimalarial medications and supplies, and some antipyretic drugs from the CDR. At the time of the evaluation, the MCZ mentioned that the health zone was still receiving bed nets; antimalarials, such as fansidar; and antipyretic drugs, which the BCZS distributed to the HAs. The MCZ stated that during the 2019 PAO review, USAID IHP agreed to provide the health zone with other essential drugs. According to the MCZ, this had not happened, and the health facilities continued to purchase essential medications using their own resources and revenue. The MCZ added that Lualaba health zones located closer to Kolwezi were receiving medicines from USAID IHP; before decentralization, these zones were in a health district receiving assistance from the US government, and according to the MCZ, these health zones continued to be favored by USAID. In 2019, the DRC government started to provide the BCZS US$10,000 monthly to obtain materials and for salary support. The BCZS was asked to use a portion of the money (US$2,000) to purchase drugs, which HCs located in the zone could subsequently buy from the BCZS. The MCZ indicated that the approach aimed to guarantee an ongoing supply of medicines, ensure standard drug prices, and reduce trips that ITs had to take to Kolwezi to restock drugs. Another advantage was that the HC could buy the drugs on credit. The MCZ reported that the BCZS obtained the medications from retailers in Kolwezi, but there was no way to confirm the quality of the drugs. Despite these efforts, the MCZ stated that drug stockouts continued to be a problem, indicating that HCs did not have adequate means to purchase large quantities of medications. The MCZ stated that during stockouts, most health workers gave prescriptions; however, he added that many facilities were in remote areas where people had no access to pharmacies. He also mentioned a decline in the quality of malaria treatment, reporting that Coartem had replaced ACT. Health facility informants confirmed the frequent stockouts of essential children’s drugs, mentioning zinc, ORS, ACT, mebendazole, artesunate, rapid diagnostic tests, paracetamol in syrup, amoxicillin in syrup, and F75 and F100 milk formulas for treatment of malnutrition. Informants from Lueu said that stockouts lasted a long time, with one health provider stating, Stockouts are frequent. Sometimes they can last six, seven, or even eight months. Many Methodist HC informants reported that during stockouts, they purchased medicines from Kasaji or Kolwezi, insisting that they did not rely on government supplies and that it was critical to replenish medications quickly. One CODESA member declared, When there is a stockout, we try to obtain medicines because otherwise the population will not use the facility. They will start going elsewhere. From the money we receive from treating patients, we first try to motivate the nurses a little, and then we take some to buy medications to keep our clients. A RECO said, When there is a stockout, we make a lot of effort to obtain the medicines, even in private pharmacies, with our own resources, because when there are no medicines, people leave. They will start going elsewhere, and this will decrease utilization of the health center. Around here, it’s not easy to get people to come back to something they’ve rejected. This is why facilities do whatever possible to keep their clients. Fewer Methodist informants stated that they gave prescriptions, indicating that patients opposed this practice. By contrast, Lueu informants indicated that prescriptions were frequently provided, as a CODESA member declared, Nurses provide prescriptions; sick patients buy the medicines in private pharmacies and bring them back to the health facility to be treated. During a stockout, the population suffers a lot. Some informants mentioned that stockouts were especially frequent during the rainy season when poor roadways created a major obstacle to the delivery of medications and other essential supplies to the health zone. 162 The Impact of USAID’s Integrated Health Program in the DRC During the rainy season, some HAs can only be reached by motorcycle, and others cannot be reached at all. One village leader said, Right now, it is the rainy season. There may be stockouts because reaching Kolwezi is a big challenge, and coming here from Kolwezi, because of road conditions, is really difficult. Supplies get blocked between Kolwezi and here. Service Utilization Most informants mentioned that poverty was the main obstacle to seeking care at facilities for sick children, with some focus group participants reporting that they faced major challenges when they were required to pay before receiving treatment. Other barriers mentioned included distance to the health facility, parental negligence, resistance by parents to use biomedical care, and self-medication and/or the use of traditional medicine. One Methodist nurse stated, The first thing is poverty–people do not have enough money to bring a sick child to the hospital. Second, around here, people prefer using traditional treatment first. When it fails, they come here. This is why the rate of transfusions is very high, because people bring their children only after receiving other care and when they are in serious condition. Third, people make their own decisions about treatment. Pharmacies have been transformed into health centers and everyone is a doctor or a nurse! When they come here [to the health facility], they say, ‘Give me quinine, give me paracetamol.’ So, 99% of patients coming for care have already started a home treatment. When it fails, they come to see us. All types of informants reported that traditional beliefs constituted a major obstacle to care-seeking from health structures. One informant explained that when a simple condition, such as fever, rapidly became serious, or when patients showed inexplicable signs and symptoms, family members often assumed that the condition was human-inflicted. In these cases, care was sought from traditional healers, such as a soothsayer or witch doctor. Management and Governance Coordination Informants said that there were biannual meetings with the board of directors, which was chaired by the director of the provincial health division (DPS) or a DPS representative, with the MCZ acting as secretary. Other participants involved local politico-administrative authorities, such as the mayor of Kasaji, members of civil society, representatives of large local churches, HCs, CODESAs, and the central zonal management team. The MCZ explained that, although CODESA members were from the community, they were linked to HCs and were part of the health framework. He stated that it was critical to include civil society leaders, who represented multifaceted sectors of social life and did not necessarily understand how the health system worked. The first annual board meeting was designed to share the annual operational action plan (PAO) developed by the health zone. Meeting participants were requested to critique and adopt the plan. During the meeting, major health zone decisions were discussed, such as dividing a large HA into two areas. If the board accepted, the DPS would present the proposal to the MOH at the central level. Board meetings offered a forum to discuss other important issues, such as community resistance to vaccination. Meeting participants were expected to report meeting decisions to work affiliates, with the overall goal of informing all sectors working in the health zone about ongoing operations of the health sector. Monthly meetings were also held with the management committee or COGE. These meetings involved members of the board of directors, except for a DPS representative, with the MCZ acting as president. The purpose of these meetings was to ensure the accountability of health zone operations, with health zone representatives reporting to the politico-administrative authorities, representatives of religious denominations, and members of civil society about achievements, HA performance, challenges faced, and proposals for future activities. During monthly meetings, members examined the execution of the PAO, which was broken down into quarters, Baseline Report 163 months, and weeks, and prepared for subsequent activities according to the PAO. They also reviewed ongoing activities, such as preparations for meetings, the development of reports that must be transmitted to health authorities, epidemiologic data submitted by HCs, and disciplinary actions to be taken against health workers. Health zone finances were likewise presented, including money raised, expenditures made in HAs, and staff changes. There were monthly monitoring meetings during which head nurses from all HAs submitted their monthly SNIS and other reports. These meetings were also attended by key BCZS supervisors, who were required to stay informed about data from each HA. During these meetings, monthly reports of HA service data were presented in relation to monthly targets, with specific services assessed and recommendations made based on the evidence. When the HA failed to meet targets, discussions ensued to analyze the situation, to identify the reasons for the gaps and strategies to prevent this from happening in the future. Health provider informants reported that these meetings provided a platform for ITs with more experience to share field experiences and practices, and to coach less capable head nurses. The central office held separate weekly meetings with the core management team, the entire central office staff, and the team monitoring epidemiologic data. The executive or core team was made up of the MCZ, the AG, the director of the reference hospital, the director of nursing, and the IS (zonal nurse supervisor). The BCZS pharmacist and a medical director of another hospital also participated. These core team members represented the steering committee responsible for making decisions to ensure the smooth functioning of the health zone. The central office staff included all workers in the health zone offices, the medical director of the reference hospital, and the director of nursing. During the epidemiological meeting, core team members, along with zonal supervisors, reviewed the data collected weekly by HC head nurses. The data provided information on the number of illness cases treated during the week and allowed for the identification of potential outbreaks. HAs far from the BCZS either submitted the epidemiologic data by phone or delivered a paper form with the weekly epidemiologic data to another IT working in a HC where a phone connection was available. The MCZ reported that, generally, no fewer than 18 of the 24 HAs provided the weekly epidemiologic data, although promptness was a problem. CHW coordination meetings, called CODESA meetings, were supposed to be held monthly in the HAs. The MCZ indicated that in each HA village, there should be a CAC comprised of RECOs associated with the HC. At the end of each month, a CAC representative traveled to the HC to present health information from their village, and ongoing activities, such as vaccinations administered or bed net distribution, to the head nurse who was in charge of the HA. CODESA members also participated. These sessions provided critical village-level information, which allowed the IT to understand the HA performance, including strengths and weaknesses, and to identify possible solutions to problems. Problems that the IT was unable to resolve were supposed to be reported to the community animator in the BCZS. CODESA members and RECOs said that they were occasionally included in BCZS meetings to discuss community health activities. Other BCZS meetings were convened when a problem needed to be addressed or a special event or intervention was to be implemented. All health providers stressed the importance of the meetings, indicating that they allowed health providers to share information and helped improve the provision of health services. Specifically, we were told that meetings with other health professionals served to strengthen and reorient community outreach; identify capacity building needs; and identify ways to strengthen the functioning of health facilities. Accountability Mechanisms The MZC reported that suggestion boxes were included as an indicator in the PDSS approach, and that during PDSS validations, PDSS evaluators checked whether a box was posted. Despite this, the evaluation team found that a suggestion box was only posted in the reference hospital, and not in the HCs. The clinician working in the hospital was the only informant familiar with a mechanism used to signal negative practices. The MCZ attributed the failure of HCs to post a suggestion box to negligence, and concerns about being accused of wrongdoing. The MCZ also noted that illiteracy was high and community members were unaccustomed to 164 The Impact of USAID’s Integrated Health Program in the DRC putting grievances in writing. Rather, the MCZ stated that community members typically shared complaints among themselves, intentionally avoiding direct confrontation. If CODESA members or RECOs heard about complaints, they were supposed to share community concerns with the head nurse during monthly meetings. Despite these obstacles, the MCZ viewed the suggestion box as a potentially effective tool to identify difficulties from the perspective of community members and to resolve service delivery problems. The PDSS also provided funds for HCs to purchase a telephone, which was to be kept by the head nurse. The telephone number was supposed to be posted, with the goal of encouraging community members to report complaints or problems with health services. The MCZ reported that 24 of the 26 HAs in the Kasaji health zone had already purchased a phone, indicating that the final two HAs were lower performing and had not yet received PDSS funds. The two HCs we evaluated had purchased phones with PDSS funds, but the evaluation team did not observe a phone number posted. The MCZ was not aware of the hot line or green line approach, which involves the installation of free phone lines for community members to use to submit complaints. The MCZ emphasized the importance of instituting mechanisms designed to ensure health worker accountability, stating that the only way to improve services and to increase service use was to involve community members. Referral Systems Informants indicated that the decision to seek health services was guided by a patient’s family members. RECOs may intervene when a community member was seriously ill and their family resisted or did not know how to seek formal healthcare. Families were responsible for transporting sick family members to the HC, which was typically done by motorcycle or bicycle, or if the distance was not far, by foot. In serious cases, or when the sick person was considered vulnerable, such as in the case of pregnant women, young children, or the elderly, RECOs and community members may assist by providing transport or resources for transport. According to our informants, when a sick person was brought to a HC, the families did not expect referral to another facility. Referrals were made between institutions, with lower level facilities making the decision to refer to institutions capable of providing more advanced care. One provider from Lueu described referrals as follows: Referrals are done either between health posts and health centers, or between health centers and reference health centers or hospitals. The health facility manager, such as the head nurse in charge of a health post or HC, or a doctor in charge of a reference HC, decided which cases to refer. Health provider informants followed two guidelines, including making referrals as early as possible and using referral forms called “transfer forms” (fiches de transfert). The transfer form was completed by the person initiating the referral and included clinical observations and the reason for making the referral. The form was carried by the patient’s family and submitted to the higher-level facility on arrival. Once again, transportation was the family’s responsibility, and if the distance was not long and the patient was able, the patient might walk to the facility. Some informants mentioned that RECOs sometimes accompanied the family. All participants stated that referrals were conducted when the facility did not have the capacity to provide appropriate care to a patient. Common reasons for referrals included pregnancy complications, acute malaria, conditions requiring surgery, prolonged coma, medication stockouts, and the need for blood transfusion or intravenous treatment. Health provider informants reported that the facility receiving the referred patient was supposed to send a patient health status report back to the referring facility. However, some admitted that the follow-up report was not always sent. Informants mentioned two major obstacles to referrals. First, if health personnel needed money, they might avoid referring patients. The second obstacle was that family members might refuse that patients be transferred due to cost concerns. One physician explained, Baseline Report 165 Since we work in a system where everyone is trying to increase revenue, this means that even when it is time to refer a patient, the facility does not let them go because they will lose money. They prefer to keep them in the facility to generate more money. This is the biggest obstacle. The other obstacle comes from families themselves. If you indicate you want to transfer a patient, the family says ‘no, if you send us there, they will ask us for a lot of money. Seeing a doctor is too expensive. At the hospital, the bills are huge. We cannot go there. We do not have enough funds.’ Informants reported that a facility with the capacity to provide care for a given condition cannot refuse a referral, even if the patient arrived late. Regarding training, both nurses and community health workers had received training on referrals. Health professionals suggested that they learned about referrals during formal studies and did not need additional training. Healthcare Financing PDSS activities began in October 2017. It was the major funding partner of this health zone. The project involved PBF, with the aim of lowering service fees and increasing healthcare access. Evaluations were carried out every three months and health facilities were rewarded according to a performance template, which combined service provision and health facility improvements over time. The MCZ indicated that DPS staff evaluated the BCZS, reference hospital, and referral HCs, whereas PDSS staff evaluated the HCs. At the start of the PDSS, each HC was asked to develop a management plan, which included facility needs. The PDSS signed contracts with 24 HCs, which were given start-up funds of US$2,500. Health workers, CODESA members, and PDSS staff worked together to determine affordable service fees that would cover treatment costs. Treatment of simple illnesses was set at 1,500 FC, with the costs for minor and major surgery established at US$5 to US$10, and US$20, respectively. The MCZ felt that providing a standard amount for start-up was inappropriate, claiming that HC management plans were not followed. The MCZ also noted that service fees had not previously been standardized, and it was important to establish fixed fees. Initially, the reduced flat rate pricing fees decreased HC revenue, making it difficult to obtain essential supplies and to pay workers. In response, nurses reverted to fees used before the PDSS, causing what the MCZ described as a chaotic situation. The MCZ stated, In the first month, all the structures, and I mean all the structures, whether it was the BCZS, the referral hospital, referral health centers or other health centers, everything was paralyzed. How was it to continue? We government representatives were supposed to make sure that the approach was followed and successful in improving accessibility to healthcare. We received a lot of criticism. At the same time, PDSS could not meet their own commitments. After this debacle, nurses reinstated original service fees to recoup losses. It was total chaos. The population deserted the facilities, and the health indicators, which had seemed to be going up, fell much lower [than before the project]. These are the difficulties that we experienced with PDSS. After this initial experience, some facilities refused to implement reduced fees. The PDSS also regularly provided late payments, sometimes by more than six months, diminishing the ability of HCs to function. Another challenge mentioned was the failure of USAID IHP to provide support, especially for the provision of medicines, which the MCZ maintained would have reduced the negative effects of late PDSS payments. The MCZ reported that health facilities that performed well received large payments that could be used to improve infrastructure and services, and to motivate health workers, indicating that 50 percent of payments were intended for operational costs and 50 percent were reserved for health provider bonuses. For example, the MCZ said that the BCZS could receive US$8,000 per quarter. The money was used to purchase motorcycles for supervision, construct a large depot to store medications, rehabilitate offices, and obtain critical office equipment. The reference hospital received US$15,000 to US$20,000 per quarter, which was used to buy essential medical equipment; and HCs received up to US$6,000 each quarter. However, the MCZ stressed that the initiative continued to be hampered by late payments, making it difficult to convince head nurses about its 166 The Impact of USAID’s Integrated Health Program in the DRC merits. The MCZ also mentioned that the PDSS prohibited the BCZS from receiving a small percentage of the health facility revenues, an approach previously used to purchase critical supplies, such as fuel to carry out supervision visits, money to repair motorcycles, or small bonuses for staff. As to the payment of health service fees, the MCZ stated that the zone promoted the provision of treatment, even if patients lacked money. When treated on credit, patients may be asked to leave material goods, such as clothes, as a guarantee of payment. The MCZ maintained that treating on credit, combined with late PDSS payments, prevented health workers from purchasing medications and supplies needed to provide care. He stated, Our philosophy is to first treat and save lives and the money comes after. We hoped that the PDSS would quickly compensate in situations when people cannot pay. We expected them to tell us to treat first and that we would regularly get reimbursed. But this is not the case. Head nurses are forced to pay for drugs, and when the PDSS makes payments, it is too late. There are periods when nurses are unable to purchase the drugs and supplies needed to treat other patients. The health providers become blocked [cannot purchase medicines or supplies] because the payment is too late. The MCZ claimed that the PDSS was impeding the proper functioning of the health zone and influencing the decline of key indicators. Health worker informants provided somewhat different information, with Methodist HC and referral hospital informants indicating that before the PDSS, higher service fees deterred care-seeking from facilities and that fee reductions had increased the use of healthcare. When talking about the PDSS, one hospital provider said, It’s effective because the project compensates health workers. Health facilities, in turn, lower the cost of care. As a result, patients are coming to facilities. Now they easily accept to see a doctor for 1,500 FC. It’s not too much of a burden, compared with paying 20,000 FC to see a doctor in larger cities. Lueu reference HC appeared to follow different standards, reverting to higher fees, and forcing patients to pay before consultations. Health provider informants from the different health facilities mentioned that the PDSS provided funds for the care of vulnerable population members. Regarding PDSS service acquisition, several providers raised objections, maintaining that some services were not included, and in some cases, compensation was inadequate. Other weaknesses commonly identified by our health worker informants included late payments, lack of flexibility, and the fact that the approach favored better skilled workers. Informants reported the PDSS had developed deterrents for corruption, indicating that a facility accused of fraud was penalized with a 50 percent reduction in PDSS funds. Additional transgressions were considered a breach of the contract, and the person in charge of the facility was replaced. Aside from the PDSS, there were no other health financing initiatives. The MCZ mentioned efforts to implement health mutuelles for health clinicians, but participants were unwilling to contribute regularly and the approach was unsustainable. Facility-Based Human Resources The MCZ started working in the health zone in 2013. The BCZS also had an AG, an IS, a pharmacist who is not formally trained, and supervisors for several technical areas, including vaccinations, nutrition, leprosy, TB, and malaria. Including support staff, such as drivers and security guards, the office had 16 staff, exceeding the government standard of 10 staff per zonal office. The MCZ explained that it was impossible for the IS to cover all technical areas in a timely and quality manner, forcing the BCZS to hire ITs working in the zone as technical supervisors. The MCZ added that the DPS sometimes assigned people to the zone, duplicating positions already filled and making it difficult to adhere to the government norm of 10 staff per zonal office. In each HA, there was an IT (A2), an ITA (A2), a laboratory assistant, and five traditional birth attendants. In Lueu, there were two physicians. At the Methodist HC, there was a nurse (A3) who took care of the pharmacy, and in Lueu, there was a nun in charge of the pharmacy and the cash register. Neither HC met government Baseline Report 167 standards for an urban HC. As mentioned earlier, the reference hospital had 104 staff, with about 20 nurses and only two physicians. Training Health professionals and CHWs recalled receiving training in recent years on revitalizing CPN, immunization, FP, TB, leprosy, and HIV/AIDS. Village leaders were unaware of the training that health providers had received. Informants noted that training on CPN revitalization and HIV/AIDS treatment were particularly useful, strengthening their capacity to provide comprehensive CPN activities and up-to-date HIV/AIDS treatment. Informants also mentioned that in 2018, the CODESA received training as part of efforts to revitalize community-based activities. Informants reported receiving recent training on treatment of severely malnourished children, nutrition surveillance, and awareness raising. All informants mentioned the urgent need for capacity building of health personnel, with several noting that many health workers were not adequately qualified. One health provider from Methodist stated, We need a lab worker who is competent. Building capacity for our lab worker is the first thing we need. Moreover, the midwives in the maternity ward need training; if possible, we would like qualified personnel instead of matrons without any qualifications. The maternity offers a critical service. The MCZ reported that in 2019, health workers received training sponsored by the Neglected Tropical Disease program on treatment of filariasis. He also mentioned that in 2018, PROSANI Plus trained head nurses, members of the BCZS, and the DPS on the iCCM sites. Supervision The MCZ said that HAs were supposed to be supervised by BCZS core team members once a month. Emergency visits can also be made during outbreaks. Supervisors followed general objectives, with more specific terms of reference established monthly according to information presented during monitoring meetings. During HA supervision visits, supervisors may visit iCCM sites or health posts, or make unannounced visits to other facilities en route to HCs. Although the BCZS team was responsible for supervising nurses and their teams, the head nurses supervised the CHWs. However, if community activities were facing problems, the BCZS team might talk to RECOs. Before the PDSS, BCZS staff supervised the reference hospital. However, because hospital administrators were also part of the BCZS executive core team, supervision responsibilities were transferred to the DPS. We were told that DPS supervision visits were irregular, sometimes occurring after six months, which prevented the BCZS from staying informed about hospital activities and delaying the submission of PDSS evaluation reports and performance-based payments. The MCZ indicated that central-level MOH representatives and technical program officials also conducted supervision. IPs might accompany them, but IP staff cannot supervise. All facility-based informants reported being supervised by BCZS staff once or twice monthly, stating that supervision entailed a review of registers and documents related to curative care, laboratory services, CPN, CPS, immunization, and finances, followed by a series of questions posed to health providers. Health providers explained that supervision visits informed recommendations given at the end of the supervision visit. Our health worker informants confirmed that supervision carried out by the BCZS did not involve community health activities. Rather, they said that RECOs were supervised by the IT, who carried out home visits to validate whether RECOs visited households and conveyed messages effectively. We were told that the ITs used this information to strengthen community activities. Information Available All informants reported that they were able to obtain information that strengthened their ability to work. Information sources mentioned included television, radio, newspapers, health-related documents, and data 168 The Impact of USAID’s Integrated Health Program in the DRC collected during outreach activities carried out by RECOs. Informants also mentioned receiving results of research examining the provision of health services in Lualaba province. All informants stated that research on health service delivery was helpful because it identified care-seeking challenges and problems that health facilities confronted in the provision of healthcare. They emphasized that the healthcare system was dynamic, and that updated information was essential to improving the quality of care. Health Provider Attitudes All informants reported that health providers were generally respectful toward patients and provider-client relations were good. However, several informants mentioned that some health providers occasionally exhibited anger or annoyance with patients, which was shown by refusing to answer questions, responding curtly, interrupting before a patient had finished asking a question, or providing routine responses that failed to address the specifics of the situation. Some informants suggested that providers frequently made assumptions about caregiver or patient knowledge and behavior. Informants mentioned fatigue, insufficient medical training, lack of training on health provider-patient interactions, poor remuneration, and difficult work conditions as the main reasons for negative interactions with patients. Most informants indicated that all patients were treated the same, irrespective of their health conditions and socioeconomic backgrounds. The majority said that gender had no influence on the provider-client relationship. Although most informants reported no differences between male and female providers, some stated that men complained less and that women were less adaptive to difficult work conditions. Age was believed to affect provider-client interactions, with young health providers viewed as inexperienced and, therefore, less capable of providing quality care. Informants emphasized that health providers should be humble and compassionate when interacting with caregivers and patients, indicating that negative behaviors led to a decrease in the use of health services. Some noted that the causes of bad behavior or attitudes, such as those associated with health worker education or motivation, were difficult to change. Most informants had never participated in a training or formal discussions about the importance of health provider attitudes and behavior when administering care. Focus group participants, who were caregivers of children under five years of age, maintained that HC personnel were welcoming, friendly, and respectful, and provided rapid and effective treatment. Health Worker Motivation The sources of motivation mentioned were income generated through monthly revenue, bonuses (primes), an understanding of the population and their health needs, and passion for their work. One hospital clinician said, If I tell you what we get, it’s nothing, really nothing. What motivates us primarily is our passion. When one is passionate about the work they chose, even when they get little money, they stay (with the work). Some of our colleagues who are doctors flee shortly after they have arrived. Here, the consultation fee is 1,500 FC. How many sick patients do you have to see to make a living? You need to be passionate about your work to stay here. The MCZ stated that only four of the 400 health workers in the health zone received a government salary, explaining that these health providers were older and were enrolled decades earlier in the national payroll. The MCZ indicated that all other health workers, including himself, received only a government bonus. Due to inflation, government bonuses had been significantly reduced, with most workers receiving small sums (the MCZ mentioned US$5 or less). Because health workers did not receive regular payments, they were forced to rely on facility resources and monthly revenue made through user fees, which the MCZ reported was about 500,000 to 600,000 FC in the Lueu reference health center, and 200,000 to 300,000 FC in most HCs. Facility￾based informants said that ITs and ITAs received about 45,000 FC and 30,000 FC, respectively, with other nurses receiving 15,000 to 25,000 FC monthly. Health personnel argued that it was not possible to replace salaries with health facility funds, which they described as insignificant. A hospital informant said, Baseline Report 169 What we make for a visit is low. Only if we carry out many surgeries, which gives us a little breather and increases the resources. If you are 28 personnel and at the end of the month you have 500,000 FC, you divide that by 28. You give 20,000 FC, 18,000 FC, 15,000 FC, what is somebody going to do with that? These workers have children. They need to pay their bills, their rents, they need food and clothes. When there are real deficits, we take from the medicine account to compensate workers’ pay a bit, to motivate them. Otherwise the system would not function. All informants reported being underpaid, stating that their work performance would only improve with an increase in renumeration. The MCZ emphasized that physicians working in rural areas suffered because they were unable to generate supplementary income by providing advanced care, indicating that they frequently left their posts to seek more profitable work in cities. He stated, Particularly us, the chief doctors, the doctors working in the bush, we suffer a lot. When you are in a city you can do a cesarean, and you get money, you can get three hundred dollars. You do another procedure like an appendectomy, I don't know, you can get two hundred or I don't know how many dollars, but here in the village, if somebody has appendicitis, you many receive twenty thousand [FC] or you are given God’s blessing as compensation. How can you make ends meet at the end of the month? Why must we suffer when we studied to lead good lives. Virtually all informants said that they lacked opportunities for career advancement and that there was an urgent need to strengthen health worker capacity. Health Workers’ Perceptions of Health Service Quality Informants from both HCs reported multiple factors that negatively influenced health service quality. The most common obstacle was ongoing drug stockouts, especially during the rainy season when nurses must travel by motorcycle to Kolwezi to obtain medications. Lueu informants said that some nurses had been injured making the journey. Methodist health workers indicated that the lack of electricity affected their ability to provide quality care. They also mentioned having inadequate personnel to deal with recent increases in the use of healthcare. Lueu providers reported insufficient skilled workers and a decrease in the use of healthcare and facility revenue, which affected the quality of care. Our data showed that Lueu charged high service fees; the HC also required patients to pay before treatment and to purchase medications. According to informants, these factors were pushing patients to go to Methodist, where services were cheaper and the quality of care was perceived to be better. To improve the quality of care, some Lueu informants recommended the renegotiation of the PDSS health service fees and not requiring payment before treatment, which would increase health service use, increase revenue, and impact monthly wages. Informants from Methodist HC recommended that the Congolese government provide support to purchase a refrigerator and reinstall electricity. Direct Observations (of Treatment) The Methodist HC and the reference hospital had waiting rooms for sick patients, whereas in Lueu, a veranda served as a waiting area. Only the waiting room in the hospital was described as clean and tidy. The hospital had educational materials, such as posters displayed on the waiting room walls, whereas the other facilities did not display such materials. Most child guardians arrived at the facility before the health provider. In the HCs, sick children were consulted on a first-come, first-serve basis, whereas in the hospital, children were prioritized according to the severity of their conditions. No sick child was sent home without receiving treatment, but one caregiver in the Methodist HC was required to leave a piece of cloth (pagne) as a guarantee that she would pay for treatment later. Providers greeted all parents, asked for the children’s names, and collected background information on the illness inception and signs and symptoms. Although all providers asked about fever, questioning about cough and diarrhea was inconsistent, especially by the medical doctor in the reference hospital. HC nurses did not always inquire about treatment that had been given before the child came to the health facility. All providers determined the age and weight of each child, did a physical examination, and followed a treatment protocol to diagnose the 170 The Impact of USAID’s Integrated Health Program in the DRC disease and prescribe treatment. Health providers did not consistently explain the diagnosis, and in two instances, the nurse did not explain the treatment. Half of the caregivers observed in Methodist HC and most parents observed in the hospital were given a prescription after the consultation to purchase medicine. Most health workers provided counseling on appropriate child feeding and instructed parents to return immediately to the health facility if symptoms worsened. In several instances, the health providers failed to confirm whether the parent understood the diagnosis and the home treatment. Only the doctor asked whether the parent had any further questions. Although only one of the 13 caregivers asked questions of the provider, all parents appeared to understand the information provided. Providers were observed to be respectful, engaged, and friendly. All consultations at Methodist and at the reference hospital and half of the consultations at Lueu were carried out in privacy. The consultation rooms in Methodist and Lueu were described as unclean, whereas at the hospital, the environment was observed to be clean. Fee schedules for health services were displayed at the facilities. Other information about the waiting areas is provided in Table 4.6. Table 4.6. Results from observations of health provider-client interactions Methodist HA 1 provider; OBSERVATIONS Lueu HA 1 provider; OBSERVATIONS Reference Hospital 1 provider; OBSERVATIONS Average age of providers observed 45 56 35 Title A2 A2 Medical Doctor Years of experience in this position 15 28 06 Training received on childhood illnesses beyond formal training 2017 and 2018 on acute respiratory infections, diarrhea, and malaria 2018, without specification of what childhood illnesses None Waiting room and triage At the hour that treatment services were open, were caregivers waiting to see the health worker? In 3 of 4 instances, parents had to wait for the provider. In all cases, parents had to wait for the provider. In 4 of 5 cases, parents had to wait for the provider. Is there a designated waiting room for caregivers and sick children? Yes, there was a waiting room for sick children. No, there was no designated waiting room for sick children. Yes, there was a designated waiting room for sick children. Was this area separate from the area where well￾baby services are carried out? Yes, the waiting room was separate from the well-baby visit area. No, the waiting room was not separate from the well￾baby visit area. Yes, the waiting room was separate from the well￾baby visit area. Were there seats available and were there enough seats for all of the caregivers and patients? Yes, there were seats available and enough seats for all caregivers and patients. Yes, there were seats available and enough seats for all caregivers and patients. Yes, there were seats available and enough seats for all caregivers and patients. Baseline Report 171 Was triage performed to ensure that most serious cases were examined first? No triage was performed. No triage was performed Yes, triage was performed. Did caregivers have to wait more than 15 minutes before the child was seen for treatment? No, the parents did not have to wait. No, the parents did not have to wait. No, the parents did not have to wait. Was the waiting area clean and orderly? No, the waiting room was not clean. The veranda where people wait is clean. Yes, the waiting room was clean and tidy. Were there educational materials such as posters on walls of the waiting area? No, there were no educational materials displayed on the walls in the waiting room. No, there were no educational materials displayed on the walls in the waiting room. Yes, there were educational materials such as posters displayed on the walls in the waiting room. Were sick children (other than those who were seen earlier due to the seriousness of their case) seen in the order that they arrived in the health center? Yes, the four children observed were seen on a first come, first serve basis. Yes, the four children observed were seen on a first come, first serve basis. Yes, the five children observed were seen on a first come, first serve basis. Were any caregivers/children sent away without being treated? No, all children were consulted. No, all children were consulted. No, all children were consulted. Community Health Services Infrastructure Informants stated that CODESA members and RECOs were the CHWs who provided critical links to community members. We were told that RECOs carried out home visits to identify sick children and to sensitize parents about preventive care for immunization, use of bed nets, hygiene and sanitation, handwashing, and appropriate dietary intake. Some informants suggested that CHWs were mostly involved in campaign activities, such as vaccinations, and failed to perform routine community outreach activities. The MCZ said that in the past three years, UNICEF had done training as part of efforts to restart community activities, with the goal of ensuring that one trained RECO was posted in each village. Despite this, many villages lacked RECOs, with the MCZ explaining that poor motivation led to high attrition. In general, informants maintained that CHW skills were lacking and that extensive capacity building was needed to improve their performance. Some informants mentioned that CHWs were not fully aware of their roles and responsibilities. Informants also mentioned that poor motivation discouraged CHWs from fulfilling their roles. Informants reported that IT involvement in community activities included supervision of RECOs, awareness raising on specific health issues, especially during times of epidemics, and monitoring activities during 172 The Impact of USAID’s Integrated Health Program in the DRC vaccination campaigns. Informants mentioned that nurses trained and oriented the RECOs, but they were not aware of ongoing monitoring of community-based activities. CHWs reported sharing health-related information during CPN and CPS visits; most indicated that the messages did not change. Although aids, such as flip charts, were available in the HCs, CHWs said that some materials were in poor condition. One hospital informant highlighted the need to translate messages, which are in Kiswahili, into the local language, stating, To improve messages, we should translate them into local languages. Many people do not speak Swahili. They speak Kikatchokwe and Runde. For them to understand, one needs an interpreter. So, in remote places like here, these messages should be translated in the local language, so they can read them on their own. That’s what I hear a lot, especially from women. Half the informants, including the RECOs, reported never participating in community discussions. The MCZ maintained that community discussions were not held. Those who had been involved mentioned that discussions focused on the treatment of diarrhea, vaccination of young children, and placing children under a mosquito net at night. The data collection team did not gather information on community-based discussions from participants. Focus group participants stated that they received health information from nurses and CHWs during CPN and CPS visits and treatment consultations, and from RECOs during household visits. Some mentioned receiving health messages from the radio or television or in church. Informants stated that community-based organizations did not exist; no informant had heard of champion communities. The MCZ indicated that in February 2018, and under the auspices of the PROSANI predecessor project, the health zone installed iCCM sites in 15 HAs, with all sites located at least five kilometers from health facilities. The MCZ explained that the approach was designed to facilitate the administration of first aid treatment to sick children before they could reach a health facility. The MCZ stated that two RECOs were trained to manage each site, with RECOs responsible for monitoring the health of children between two months and 59 months of age living in proximity to the iCCM site. The project provided medicines, such as antimalarials, antipyretics, ORS, and zinc, and basic supplies, such as a thermometer, a scale, height measure, jerry cans, a basin for handwashing, and small cupboards for storage of medications. The MCZ also mentioned that RECOs were responsible for producing ongoing progress reports, which they submitted to the DPS. With the end of PROSANI Plus, the distribution of drugs and supplies, including reporting forms, to the iCCM sites decreased. The pressure to perform well and submit reports also declined, leading to the discontinuation of many iCCM sites. During our evaluation, many HA informants were unaware of the iCCM approach and its sites. Those informants who knew about iCCM activities mentioned frequent stockouts of medicines, lack of transport to get sick children to HCs, insufficient training of RECOs to carry out the work, and high turnover and absence of RECOs to oversee the iCCM sites. System Design Role of CODESA Members Informants reported that CODESAs were a bridge between the community and the health facility, and that it provided oversight of health facility activities. One CHW informant from Methodist explained, In my opinion, CODESAs have an essential role in knowing whether the center receives a lot of sick patients, how they [the patients] are treated, and for how much. Is the official price applied? The CODESA President is like the president of a parents’ committee. Between parents and the school, there is a president, since parents cannot go to school, they must go through the committee. This is how it works with CODESA. Fewer informants stated that CODESA members supervised RECOs. Surprisingly, one RECO did not know the role of CODESA members in his HA. Regarding training of CODESA members, most informants could not recall any recent training, indicating that CODESA members received minimal technical support. Informants from the Methodist HA mentioned regular Baseline Report 173 monitoring meetings involving CODESA members and RECOs to assess community activities. In Lueu, meetings designed to monitor community activities appeared to be infrequent. In the HAs evaluated, we identified one female CODESA member who was the CODESA president in the Methodist HA. Role of RECOs Informants indicated that RECOs were responsible for carrying out household visits to identify sick children, pregnant women, and people with disabilities, and to refer community members to the HCs for treatment and CPN. Their work also involved awareness raising around key health topics, such as maintaining good hygiene and disease prevention, assisting with routine vaccination and mass immunization campaigns, and informing facility workers about health behaviors and care-seeking practices of community members. Informants’ estimates of active RECOs were inconsistent; informants from the Methodist HA reported 22 to 40 RECOs, whereas in Lueu HA, they mentioned anywhere from 12 to 33 active RECOs. Of the active RECOs, Methodist informants said that more than one-quarter were female, whereas most Lueu informants suggested that there were no female RECOs. One Lueu informant explained that the low number of female RECOs was influenced by spousal opposition, which he described as follows: It’s due to their husbands. You see, villages are different from cities. In villages, men are so complicated. A man cannot let his wife move around freely or go to other villages to conduct outreach. This can end a marriage. This is why we are not very interested in enrolling more women. As for recent training, capacity building workshops in Methodist HA and a training on nutrition for Lueu RECOs were mentioned. All informants reported that RECOs maintained positive relations with community members, with some noting that, as community representatives, RECOs must be respectful and supportive. Several mentioned that the RECO work activities, which involved identifying sick patients in their homes, accompanying them to HCs, and mobilizing community members who were reluctant to seek care in facilities, demonstrated their compassion for the community. Some emphasized that maintaining trust and respect were central to the RECO role, pointing out that, otherwise, community members would disregard awareness activities and efforts to improve health behaviors. Informants said that community members followed the advice provided by RECOs, which they considered a testament to the trust established. One village leader from Methodists HA said, I think the RECOs have a good relationship with communities. I have seen, on multiple occasions, that people give the RECOs gifts [out of appreciation for their work]. One thing I see a lot is that certain church goers don’t like them very much, particularly the Apostolo and Kitawala groups, who refuse mass vaccination. Apart from them, RECOs are well appreciated. Specific Services Offered Mosquito Nets Most informants reported involvement in the distribution of bed nets, which were distributed monthly during CPN and CPS visits and in the maternity. The last mass distribution of bed nets was conducted in 2016, and although the MCZ said that a bed net campaign was carried out in other provinces in “grand” Katanga in 2018, Lualaba was excluded. The MCZ noted that mass distribution was important to ensure widespread bed net coverage. Regarding eligibility, study informants reported that they were pregnant women attending CPN, women who delivered in the maternity, and children under five years of age attending CPS. The major challenge mentioned to achieving high coverage was that mosquito nets were insufficient to meet the population’s needs. One Methodist village leader informant said, 174 The Impact of USAID’s Integrated Health Program in the DRC This is the problem–even if the State provides mosquito nets, they don’t get to everyone. The population is very large. There will always be a challenge. Some will get it, others will not. You will see that in one household, there may be 10 people. The nets are insufficient, which triggers a lot of negative rumors in the community. All informants said that mosquito nets were set up in households and used correctly, as confirmed by RECOs during home visits. Some maintained that recently distributed bed nets were made of a lighter material, preventing people from using them as fishing nets. Unfortunately, the evaluation team did not ask focus group participants when they last received mosquito nets. All focus group participants from the Methodist HA claimed to have a mosquito net, which they used to protect their children from malaria. Vaccinations Informants reported that vaccinations were administered during routine services offered to pregnant women and young children at health facilities or during outreach services in remote villages. HAs were supposed to carry out at least two routine vaccine sessions each month, one in the facility and one in an HA community. The MCZ said that only some HAs had a refrigerator to maintain the cold chain, which they shared with neighboring HAs. He also indicated that high vaccination coverage required four monthly sessions, noting that vaccine coverage in the zone was low due to limited outreach activities, adding that outreach was easier in urban HAs, such as Methodist. Vaccines were also administered through campaigns that were held twice a year, generally in April and September, for specific diseases affecting children, such as measles or polio. Mass campaigns were dependent on donor support and involved all levels of the health system, with RECOs playing a key role in vaccine administration. Challenges to vaccinations mentioned included stockouts, the fact that many community members lived in hard to reach areas, and resistance by parents. The village leader from Lueu said, The challenges we face are with parents. Some do not like to send their kids [for vaccines]. They may say that vaccines trigger disease, or that the child’s blood will be drained out. Others ask us if we ourselves vaccinate our children. Some say vaccines are a white man’s strategy to exterminate our children, as Herod did in the Bible. Many of those who resist are practicing Kitawala and Apostolo parishioners. Other informants reported that resistance was declining, as this health worker from Lueu said, Because vaccinations are free, the community had a bad perception. Thankfully, [this perception] is slowly disappearing due to epidemics such as measles. Before people thought that the vaccine would cause disease in children. When we had a measles outbreak recently, everyone understood its [the vaccine’s] importance. Children without the vaccine developed complications from measles and some even died. Only a few reluctant religious groups such as the Apostolo and Kitawala are still resisting vaccination. The village leader from Methodist said, There were many challenges before, but now people are starting to adapt and bring their children to get their shots. Before, there was resistance. People would say that the vaccine causes disease. But thanks to outreach efforts, some people have started to vaccinate their children and they can see for themselves that, even if their vaccinated child gets sick, it is not as serious as it is in children who aren’t vaccinated. So, people understand that vaccines are a good protection. Focus group participants mentioned that they were required to pay a fee for the CPN and CPS form before receiving vaccinations. Unfortunately, the data collection team did not collect other information from focus group participants on vaccinations. Baseline Report 175 Community Worker Motivation CHWs unanimously reported that love for their work and dedication to serving their communities motivated them, with some noting that from the outset, they knew the work was voluntary. Several maintained that continued involvement in the work signified that they were satisfied, whereas those who did not enjoy the work quit. CHW informants mentioned that training and educational materials would enhance their effectiveness as community agents. The MCZ stated that stipends provided during vaccination campaigns motivated the CHWs. He also indicated that some ITs gave free healthcare to RECOs. The MCZ said that some projects provided benefits, such as bicycles (a project funded by the U.S. President’s Emergency Plan for AIDS Relief) and small stipends, which the PDSS gave to CODESA presidents for their involvement in the management of HC funds. USAID Projects in the Area In 2016, the health zone received some assistance from the PROSANI predecessor project for the treatment of HIV-positive cases. According to the MCZ, antiretrovirals were going to expire and PROSANI leadership needed to distribute the medications quickly. In 2018, PROSANI Plus provided training for the installation of iCCM sites and materials, including kits and drugs that RECOs needed to manage the sites. Starting in 2019, USAID IHP began funding meetings, such as monthly monitoring reviews and one biannual board of directors meeting. USAID IHP also supported work on HIV treatment and the organization of data evaluation and validation meetings on malaria. USAID IHP was also providing funding for the PBF activities. However, the MCZ claimed that USAID IHP had not fulfilled many promises, especially for the provision of medications, which they had promised to deliver from the CDR to the health zone. Other promised activities, such as TB treatment work, likewise never came to fruition. The MCZ believed that USAID IHP was mostly supporting health zones in proximity to Kolwezi and neglecting more remote zones. The MCZ said, PROSANI amazes me. If we had met in Kasaji, I could have shown you the commitments that USAID IHP made, which included provision and transport of drugs. There were a lot of other things, but I'm most interested in the medicine. Before I left, they had only supported the board meeting and the data validation meeting as part of PNLP [National Malaria Control Program]. This is good, but it does not solve the biggest problem, which is drug supply. Other informants were not aware of previous USAID-supported activities in the health zone. In Methodist HA, health providers and community agents had heard of the USAID IHP project, with some indicating that the IT participated in a briefing about the project in the BCZS. However, they were generally unaware of the nature of future activities or whether interventions had begun. A nurse said, Indeed, I have heard of the PROSANI Project, but I don’t know the reality. I have never participated in any meetings. In our health area, it is the very first time we’ve spoken with the project’s delegates [meaning the evaluation team]… I’ve heard that a project arrived, they conducted a briefing in the central offices with the head nurses, but I did not attend. One RECO suggested that the project supported nutrition activities and provided malaria supplies, stating, We received a delegation that came to train the head nurses in nutrition. After the training, we received some products for nutrition activities. After that, we received rapid malaria tests, and the injectable artesunate. We were told it was PROSANI that was supporting us with these medicines. In Lueu, informants had heard of a USAID-funded project that would provide support to their facility, but they had not attended any meetings about the project and were not even aware of the project name. Hospital informants affirmed that they knew about a USAID-funded project called "PROSANI,” which had supported training in nutrition and provided supplies for the treatment of malnourished children. 176 The Impact of USAID’s Integrated Health Program in the DRC Summary and Implications Summary The purpose of the study was to assess baseline levels of health governance and leadership, service readiness, and service quality as part of an evaluation of USAID IHP. The empirical results were based on surveys of provincial health offices (n=6), health zone offices (n=113), HCs (n=328), hospitals (n=110), and health workers (n=1,213). We also used data from Abt Associate’s baseline household survey to characterize the care seeking experience and level of participation in health services among women of reproductive age. The empirical findings were supplemented by interviews with 20 key informants at central and provincial levels, including MOH representatives (n=7), USAID IHP senior staff (n=6), USAID staff overseeing USAID IHP (n=3), and collaborating partners (n=4), to understand their perspectives about the project and the various health systems challenges that need to be addressed in the DRC. We also carried out an in-depth, qualitative investigation in two health zones in the Lualaba province, one that had received extensive support from the USAID IHP predecessor project and another that had received sup In each health zone, we administered a mix of qualitative research methods including key informant and in-depth interviews, observations, and focus group discussions. Data collection involved in-depth interviews with health center nurses (4), members of the health development committee (4), community health workers (4), village chiefs or village chief representatives (4), reference hospital physicians (2), hospital or BZCS managers (2), and an acting health zone medical officer; key informant interviews with the MCZ (2); observations of health infrastructures (6) and health worker-client interactions (40); and focus group discussions with caregivers of children under five years of age (4). port for specific activities such as iCCM. Both the empirical and qualitative analyses suggest that the government-run health system faces myriad challenges at both the health zone and facility levels to effectively plan, implement, and monitor services. At the health zone level, respondents mentioned several problems with communication and supervision that constrain the ability of health zone offices to effectively coordinate and manage health activities. Communication problems include the limited and interrupted availability of electricity, lack of office cell phones, and limited Internet availability and credit. Health zone office respondents reported the lack of telephones and Internet to be the two most important barriers to the timely reporting of disease surveillance (MAPEPI) data and to receiving timely reports from HCs. In terms of supervision, MOH guidelines stipulate that health zone offices should receive external supervision from the provincial level at least once every three months, but only half of the offices mentioned being visited by a representative of a provincial health office in the three months before the survey. Health zone offices are also supposed to provide external supervision to hospitals through visits at least once every three months, but fewer than half of the hospital respondents reported being visited by a representative of a health zone office in the three months before the survey. At the facility level, most HCs do not offer the full minimum package of services because a relatively low percentage of facilities offered FP services, CPS, and minor surgery at HCs. For those services that are offered, we uncovered several issues with service quality and financial accessibility. Service quality problems include stockouts of medicines and supplies, the frequent use of unregulated medicines, lack of basic equipment, very low availability of electricity (especially at HCs), and many providers having inadequate knowledge about how to properly offer the standard of care for treatment of diarrhea and the provision of ANC and FP services (demonstrated through the data collected using provider vignettes). Qualitative data also show that HCs are hiring local community residents who have not received formal training to provide services. In terms of financial access to services, indigent fee schedules were found to be lacking, and facilities reported using other strategies when patients cannot pay, including having family members work at the health facility and allowing families to pay at a later date. Health workers rely on user fees as a source of income; therefore, staff may not be incentivized to reduce or waive fees. Problems with reporting SNIS data were also common, with Baseline Report 177 qualitative data suggesting that the lack of phone credit and access to the phone network force facilities to deliver hard copies of surveillance reports, which causes delays. Lack of paper forms and transportation were also mentioned as barriers to reporting on time. Problems with the availability and motivation of health workers at hospitals and HCs were also identified. Most health facilities have staffing levels that are well below MOH guidelines, and most health workers interviewed reported not being satisfied with their jobs due to low remuneration and poor working conditions. The findings from the key informant interviews at the central level and in Lualaba province provide rich insights on several leadership and governance issues in the health system. Based on their wealth of experience developing and implementing health service delivery programs, and in HSS, informants consistently described the national health system strategy as well-conceived and organized, and encompassing key elements. However, there is universal agreement that many facets of the health system are not being implemented as planned. Underfinancing, mismanagement of resources, and poor governance are seen as the primary obstacles contributing to low use and quality of health services. When asked to describe the USAID IHP approach, key informants from USAID and USAID IHP mentioned that malaria programs will be executed across all health zones (178) in USAID IHP-supported provinces, but that the implementation of the other programs will vary according to the availability of resources, local needs, and ongoing activities of IPs. The project will concentrate efforts on geographic areas to increase synergy with other USAID programs and improve overall project impact, increasing its potential to improve the quality and content of health service delivery across health zones. The funding of USAID IHP through a contractual agreement between USAID and Abt Associates is intended to provide USAID with ultimate decision-making power and technical oversight of project development and implementation, and the use of funds, whereas Abt Associates is responsible for the execution of activities and ensuring that indicators are followed and results are achieved. However, some informants raised concerns that the restrictive nature of the contract and USAID IHP’s approach will limit the ability of government personnel to lead project development and implementation, and to take ownership of HSS. Abt Associates representatives and USAID IHP collaborating partners also expressed frustration that they are simply playing the role of implementer but have little decision-making power. Informants described USAID IHP as a complex project comprising many interventions and actors, with several informants commenting that the project is overly ambitious in relation to the multiple challenges plaguing the DRC health system. Commonly cited problems include many of those mentioned above: poor health information systems affecting data quality and reporting, irregular provision of quality medications, low motivation of health workers linked to poor remuneration and work conditions, and rampant corruption and mismanagement of funds that infiltrate all levels of the health sector. Some respondents also mentioned problems with the DRC’s community strategy, notably that CODESAs and RECOs are not performing their roles and that cultural practices and beliefs among community members are major obstacles to seeking care at formal healthcare facilities. Findings show that MCZs are frequently physically absent from their posts. Staffing of health personnel does not meet government standards, with health centers often using untrained workers to provide medical care. Health centers are often small, and facilities are frequently poorly constructed, in disrepair, and lacking key equipment and supplies. Stock-outs of essential medications are common, forcing health workers to restock often with unregulated drugs or to give patients prescriptions to purchase drugs in local pharmacies. A growing number of private pharmacies selling unregulated drugs allows people to self-treat before seeking professional care. Community members often seek treatment with traditional healers who provide remedies that can interfere with medical care and cause harmful health effects. Health workers rely primarily on monthly facility revenue for compensation, although performance- based financing activities are providing important bonuses to facilities that meet standards. Despite low and irregular 178 The Impact of USAID’s Integrated Health Program in the DRC remuneration, health providers expressed gratitude for being employed. Informants described health workers as respectful of and courteous towards clients, which was confirmed during health worker-client observations. Observations illuminated that health workers fail to follow components of treatment protocols during patient consultations, which often take place in unclean, noisy environments, where privacy is not maintained. According to the individuals interviewed, no community-based organizations involved in health existed in the zones studied. Community outreach activities are not functioning according to the national strategy. Community health workers require training, supervision, and supplies to fulfill roles. Lack of oversight and supplies does not allow iCCM posts to operate as planned. Poor motivation causes high attrition of community health workers. Community health workers are predominantly male, with men controlling positions of leadership. Programmatic Implications Although many of the health system challenges identified through the study—such as limited access to electricity and phone connectivity among health workers and problems with health worker remuneration—are outside the control of the project and lie instead with the government, they are nevertheless critical constraints that must be considered when designing and implementing USAID IHP-supported interventions. Obvious examples are the lack of electrical power and limited phone and Internet connectivity and Internet credit, which can affect efforts to improve service readiness, data collection, and SNIS reporting. However, other challenges identified can be partially addressed through project-supported strategies and interventions. Below are recommendations based on the study results: • The design of both facility- and community-based service delivery interventions should be informed by formative research on community perceptions and needs. This recommendation is based on the qualitative research findings that highlight the importance of geographical and cultural differences in the DRC, and the need to design program approaches to better coincide with contextual conditions. More experimental interventions should also be tested to assess the role of cultural context on their effectiveness. • The program should explore alternative approaches to improve the remuneration of health workers, possibly through coordination with the MOH’s PDSS program, to improve health worker motivation and, in turn, service quality and availability. This recommendation is based on both quantitative and qualitative research findings that suggest that a large percentage of health workers do not receive a salary. We found that this affects health workers’ motivation and leads to a heavy reliance on income received from household out-of-pocket payments. • The program should place more emphasis on the supervision of local health officers and health workers to improve service quality, and the collection, availability, and use of routine data beyond the data available in the DHIS2. This recommendation is based on survey findings indicating that external supervision is not carried out as frequently as MOH guidelines stipulate, and that providers need support for adhering to provider practice guidelines. • More emphasis should be placed on continuous education programs to improve adherence to provider practice guidelines that can improve adherence to standards of care. Again, this is based on the findings from the clinical vignettes, showing that providers often did not have the knowledge to correctly diagnose and treat certain health problems. • Stockouts of essential medications plague the health facilities, undermining health services and their use, and jeopardizing the revenue needed for health facilities to function. Stockouts force health workers to use unregulated drugs and provide prescriptions to sick patients. To ensure the provision of Baseline Report 179 regular and quality care, efforts are needed to guarantee that medications meet drug orders and are delivered in a timely and routine fashion. • Community health activities are not functioning as described in the national community health strategy. CHWs require ongoing training, supervision, and materials to successfully carry out activities, including their roles in the execution of iCCM activities. Community activities would benefit from the oversight and support of community-based organizations and development committees. Mechanisms to motivate and retain CHWs need to be explored. Limitations There were several limitations to the evaluation. Although the full set of limitations is presented in the methods section above, the limitation that relates to the interpretation of the baseline results is presented below. The results are based on the surveys conducted in USAID IHP-supported provinces only and, as a result, comparisons between project provinces and non-project areas were not feasible. Moreover, the D4I and USAID IHP surveys that are used as the source of baseline data were administered several months after USAID IHP started operations (in July 2018). This timing would be problematic if USAID IHP’s implementation activities had already begun because it would bias the performance evaluation component of the study. However, USAID IHP was engaged in its initial planning phase when the baseline survey was conducted and had not yet started implementing its approach in the provinces. In fact, based on previous meetings with USAID and the USAID IHP management teams, one of the key purposes of the USAID IHP surveys is to provide evidence that can be used to comprehensively assess the health system needs in the nine provinces. References Adam, T., Hsu, J., de Savigny, D., Lavis, J., Rottingen, J., & Bennett, S. (2012). Evaluating health systems strengthening interventions in low-income and middle-income countries: Are we asking the right questions?. Health Policy and Planning, 27(suppl 4), iv9-iv19. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/23014156 Baseline Report 180 Appendixes Appendix 1: Evaluation Protocol Research Protocol Evaluation of USAID’s Integrated Health Program for the Democratic Republic of Congo 13 June 2019 Data for Impact University of North Carolina at Chapel Hill 400 Meadowmont Village Circle, 3rd Floor Chapel Hill, NC 27517 USA TEL: 919-445-9350 FAX: 919-445-9353 http://www.cpc.unc.edu/measure Baseline Report 181 Contents Abbreviations............................................................................................................................................182 1. Purpose and Audience .....................................................................................................................183 2. Background.......................................................................................................................................185 3. USAID IHP Description......................................................................................................................188 4. Theory of Change .............................................................................................................................192 5. Research Questions..........................................................................................................................194 6. Evaluation Design.............................................................................................................................195 7. Limitations and Threats....................................................................................................................205 8. Quantitative Data Requirements, Collection, Management, and Use ............................................207 9. Qualitative Data Requirements, Collection, Management, and Use ...............................................210 10. Ethics and Informed Consent...........................................................................................................217 11. Data Management ...........................................................................................................................218 12. Evaluation Team and Stakeholder Roles..........................................................................................219 13. Timeline and Deliverables................................................................................................................220 References ................................................................................................................................................223 182 The Impact of USAID’s Integrated Health Program in the DRC Abbreviations ANC Antenatal care ARI Acute respiratory infection CBO Community-based organization CHW Community health worker CODESA Comités de Développement de l’Aire de Santé (Health area development committee) CSO Community-supported organization D4I Data for Impact DHIS2 District health information system 2 DHS Demographic and Health Survey DID-PSM DRC Difference-in-differences with propensity score matching Democratic Republic of Congo HZ Health zone IHP Integrated Health Program KSPH Kinshasa School of Public Health LiST MNCH Lives Saved Tool Maternal, neonatal, and child health MOH Ministry of Health MOU PICAL Memo of Understanding Participatory Institutional Capacity Assessment and Learning SBC Social and behavior change TB Tuberculosis USAID USAID IHP United States Agency for International Development United States Agency for International Development’s Integrated Health Project in the Democratic Republic of Congo USD United States Dollar WASH Water, sanitation, and hygiene Baseline Report 183 1. Purpose and Audience As part of its strategy to improve health outcomes in the Democratic Republic of Congo (DRC), the United States Agency for International Development (USAID) funded the Integrated Health Program for the DRC (USAID IHP) in 2018. The program began operations in July 2018 and is being implemented by Abt Associates and several partner organizations. The purpose of USAID IHP is to strengthen the capacity of Congolese institutions and communities to deliver quality, integrated health services to sustainably improve the health status of the Congolese population. The specific health, population, and nutrition areas that will be the focus of the project include maternal health; neonatal, infant, and child health; tuberculosis (TB); malaria; child nutrition; water, sanitation and hygiene (WASH), and family planning. USAID IHP’s overall performance objectives to reach this goal include: ● Strengthen health systems, governance, and leadership at the provincial, health zone (HZ), and facility levels in target provinces (Objective 1) ● Increase access to quality, integrated health services in target provinces (Objective 2) Increase adoption of healthy behaviors, including use of health services, in target provinces (Objective 3) USAID IHP will work in nine contextually diverse provinces in the regions of Eastern Congo, Katanga, and Kasai and will include a wide array of interventions. Given the breadth and depth of the planned interventions, the USAID/DRC mission has requested Data for Impact (D4I) to conduct an independent third-party evaluation of the performance and impact of USAID IHP on key health systems-related outcomes, including the uptake of family planning and health care services; health systems functioning (i.e. improved disease surveillance, the availability of essential commodities, and health worker motivation); and the practice of key healthy behaviors. The purposes of the evaluation are to investigate the following: • The progress of USAID IHP over time in achieving Objectives 1, 2, and 3, as listed above: strengthened health systems, governance, and leadership at the provincial, health zone, and facility levels; increased access to quality, integrated services; and increased adoption of health behaviors. • The impact of USAID IHP on several proxy indicators of healthy behaviors, including treatment for childhood illnesses, maternal health care use, and contraceptive method use. • The factors that enabled or limited the success of USAID IHP, including the design and scope of the project, implementation factors, and external and contextual factors. To achieve these evaluation objectives, both a performance evaluation and an impact evaluation will be carried out. The performance evaluation will incorporate several study components based on data from USAID IHP provinces only, including District Health Information System (DHIS2) data, population￾based household survey data; health facility, health zone office, and provincial health office survey data; and qualitative data. This component of the evaluation will investigate whether the USAID IHP targets were achieved for all three USAID IHP performance objectives – strengthened health systems, 184 The Impact of USAID’s Integrated Health Program in the DRC governance, and leadership; increased access to quality, integrated services, and increased adoption of health behaviors. To the extent possible, the empirical data will be triangulated with qualitative data to explore whether changes over time in the adoption of healthy behaviors were associated with changes in strengthened health systems and increased access to quality, integrated services, and to determine why expected changes were or were not observed. Finally, to explore the health impacts of the changes in intervention coverage, the Lives Saved Tool (LiST) will be used to estimate how changes in the adoption of healthy behaviors translated to changes in the number of lives saved. The impact evaluation will be carried out by comparing DHIS2 data of several proxy indicators of health behaviors in project areas and non-project areas using a difference-in-differences with propensity score matching approach. Because it will not be possible to administer the population-based household surveys and the health facility, health zone office, and provincial health office surveys outside of the nine USAID IHP provinces areas, it will not be feasible to assess the impact of USAID IHP on population-based service coverage rates, or on indicators of strengthened health systems, leadership and governance and increased access to quality, integrated services. Key audiences for the evaluation include USAID, the Ministry of Health (MOH), and other international health organizations and agencies. It is expected the results of the evaluation will be used to improve the design and implementation of future health systems strengthening activities in the DRC. It is important to note that D4I has only been contracted to carry out baseline evaluation activities thus far. The protocol is structured as follows. Following this introduction, Section 2 provides background information on the country and health system context. Sections 3 and 4 describe the USAID IHP and the USAID IHP Theory of Change. Section 5 presents the research questions that will be investigated, and Section 6 describes the evaluation design and methods. Section 7 describes the limitations of the study. Sections 8 and 9 present the quantitative and qualitative data requirements and details on how the data will be collected, managed and used. Section 10 describes the ethics and informed consent considerations and procedures. Section 11 describes data storage and security issues, as well as data sharing and knowledge management considerations. Section 12 provides a description of the evaluation team. Finally, Section 13 describes the study timeline and the deliverables. Baseline Report 185 2. Background a. Country setting The DRC has experienced decades of sporadic conflict and widespread and extreme poverty. The conflict along with substantial weaknesses in governance have had devastating effects on the economy, institutions, and households in the DRC (World Bank, 2017). In the period from 1998 to 2007, an estimated 5.4 million people died as a result of the conflict and protracted humanitarian crisis (Moszynski, 2008). Millions more were pushed into poverty due to displacement and the loss of their economic livelihoods. By 2012, most of the country had returned to relative peace, although in eastern DRC, a crisis involving the M23 armed group as well as other armed factions persisted. Militia activities and ethnic tensions continue to erupt periodically in regions of the country. In addition to armed conflict, the DRC has experienced longstanding political tensions. Both the 2011 and 2018 presidential and legislative elections were marred by violence and disputes about the results, which reflected deep political divisions and the fragility of DRC’s electoral institutions. While the DRC is endowed with extraordinary natural resources, it is also one of the least developed countries in the world. It ranks at the bottom of the International Food Policy Research Institute’s World Hunger Index, making it the hungriest country in the world, with only 10 percent of its agricultural potential exploited (USAID, 2014). The country is also tied for last on the United Nations Human Development Index. Gender inequalities are also prevalent, as the DRC ranks 148 out of 157 countries on the Gender Development Index. Poverty is also widespread. In 2011, gross national income per capita was 190 USD and 71 percent of Congolese lived below the poverty line of $1.25 per day. Individuals living in rural areas were more likely to be impoverished than individuals in urban areas (75 percent vs. 61 percent) (World Bank, 2017). The DRC is one of the world’s top five contributors to maternal mortality (846 deaths per 100,000 live births) and child mortality (104 deaths per 1,000 live births). The Total Fertility Rate has remained at around 6.6 since 2007. Malaria, diarrhea, and pneumonia are significant causes of death for children under five years of age. The prevalence of malaria among children under five years of age was 30.8 percent at the time of the most recent Demographic and Health Survey (DHS), and national health statistics indicate that 40 percent of outpatient visits are for suspected malaria (Ministère du Plan et Suivi de la Mise en œuvre de la Révolution de la Modernité (MPSMRM), Ministère de la Santé Publique (MSP) et ICF International, 2014). Malnutrition continues to be a serious issue among children under five years of age. Over 40 percent of children are malnourished as indicated by high rates of stunting (low height-for-age), with more than half of these children falling into the severe range. Malnutrition and child survival rates are closely linked to breastfeeding, and water, sanitation, and hygiene (WASH) practices. WASH indicators have improved little, with the majority of the population still lacking access to improved water and sanitation facilities. TB also is a serious problem and the DRC is ranked ninth among the 22 highest burden countries for the disease (Ministère du Plan et Suivi de la Mise en œuvre de la Révolution de la Modernité (MPSMRM), Ministère de la Santé Publique (MSP) et ICF International, 2014). b. Health systems context 186 The Impact of USAID’s Integrated Health Program in the DRC The DRC’s government-run health system is organized on three levels: central, provincial, and peripheral (zone de santé or HZ, the equivalent of a district in most countries). The MOH at the central level is responsible for overall policy direction and stewardship, and the management of national programs. At the provincial level, the provinces are responsible for providing technical support and supervising HZs. At the peripheral level, each HZ covers an average population of 110,000 and includes a central HZ office and at least one general reference hospital. The HZ is divided into health areas (aires de santé), each of which contains, in theory, at least one health center or health post that is responsible for providing a minimum package of services as defined by the government. The government has received significant technical and financial support from major bilateral and multilateral partners, including USAID, the United Kingdom’s Department for International Development, the European Union, the World Bank, and the United Nations, including the World Health Organization, UNICEF, and the United Nations Population Fund. The poor health outcomes described above are linked to low levels of effective coverage of reproductive, maternal, child, and adolescent services. In 2015, per capita health spending in the DRC was only 22 USD, one-sixth of the average in sub-Saharan Africa and one fourth the amount necessary to provide basic health services to the population (Meheus & McIntyre, 2017). Private household health spending is substantial, with households bearing 40 percent of total health expenditures, 93 percent of which is made up of out-of-pocket expenditures, which contributes to inadequate financial access to services. Donor spending is also a key driver of total health expenditures, making up 40 percent (World Bank, 2017). Inadequate health spending levels also contribute to inconsistent and often poor service quality (World Bank, 2017). While there has been extensive decentralization of financing and management responsibilities in the health system, inadequate provider payment contribute to poor health worker motivation (World Bank, 2017). In addition, the creation of 15 new provinces has led to challenges that result from low managerial capacity in the health sector. While community-based interventions are critical to providing support to health facility services, poor organization and lack of motivation of community health workers (CHWs) undermine community activities (Bertone et al, 2016). c. Project description In response to these health systems challenges gaps, USAID funded the USAID IHP, which is being implemented by Abt Associates and several partner organizations. The purpose of USAID’s IHP is to achieve sustainable improvements in the health of the Congolese people through: (1) strengthened health systems, governance, and leadership at the provincial, HZ, and facility levels, (2) increased access to quality, integrated health services, and (3) increased adoption of healthy behaviors, including use of health services. The project will be implemented in nine provinces in the regions of Eastern Congo, Kasai, and Katanga, and the USAID IHP management team plans to use a province-specific tailored approach to achieve the three above-mentioned objectives of the project. d. Motivation for the evaluation Despite the need for a more responsive health system, the evidence base for improved policy decision￾making in the DRC is weak, due to limited research capacity, lack of transportation and communications infrastructure, security challenges, and limited funding for health systems research. Baseline Report 187 Beyond the DRC, there is also a paucity of information in low-income countries, particularly fragile states, regarding the efficacy of interventions to address system-level gaps and capacity shortcomings. In a 2012 systematic literature review of evaluations of health systems strengthening programs, few evaluations were found to be comprehensive across multiple health system building blocks and few included evaluation designs that considered the complex nature of the programs (Adam et al., 2012). This evaluation represents an effort to help fill the present knowledge gaps by supplying critical information about the impact of USAID’s IHP, a large, complex health systems strengthening project. By contributing to the country’s evidence base of successful health systems strengthening activities, and those that are not successful, the proposed evaluation can be used to help design future projects intended deliver high quality family planning and health services that meet the needs of the Congolese people. 188 The Impact of USAID’s Integrated Health Program in the DRC 3. USAID IHP Description a. Project name, goals and objectives, key indicators and outcomes The overall purpose of the USAID IHP is to improve the health status of the Congolese population. The program aims to achieve this goal through strengthening the capabilities of health systems, government institutions and communities to support the delivery of high-quality, integrated and sustainable health services, which will, in turn, increase access to improved health services. Another important component relates to promoting the adoption of healthy behaviors including increased utilization of health services, as well as health-related behaviors at the individual, household and community level. The program includes three objectives to reach the main goal of improving the health status of the Congolese population, as follows: ● Objective 1: Strengthen health systems, governance, and leadership at provincial, HZ and facility levels in target HZs. ● Objective 2: Increase access to quality, integrated health services in target HZs. ● Objective 3: Increase adoption of health behaviors, including health services use in target HZs. The program components and interventions are designed to influence measurable improvements in several health outcomes. By the end of the project, USAID IHP is intended to achieve positive changes in the following: ● Proportion of children under five years of age for whom treatment/advice was sought for acute respiratory infection (ARI), diarrhea, and fever; ● Proportion of children under five years of age who slept under an Insecticide-Treated Net; ● Proportion of married women using any modern method of contraception; ● Proportion of children who received all eight basic vaccinations; and ● Proportion of pregnant women who attend four ANC visits. The program is based on the assumption that improvements in these and other health-related outcomes will contribute to positive changes in impact-level indicators, including maternal mortality ratio; neonatal, infant, and under-five mortality rates; TB case notification rate; malaria prevalence rate; contraceptive prevalence rate; and acute and chronic malnutrition rates. b. Project components, specific interventions, and delivery mechanism Project components and interventions are broken down to address the three program objectives as described below. Objective 1: Strengthen health systems, governance, and leadership at provincial, HZ and facility levels in target HZs. Programmatic approaches related to Objective 1 aim to support provinces, HZs, and communities to be empowered stewards and effective managers of health system functions, via tailored needs-based interventions guided by results of Participatory Institutional Capacity Assessment and Learning (PICAL) Baseline Report 189 evaluations and human-centered design techniques. At the province and HZ levels, the PICAL tool will be applied to foster a culture of self-assessment, enhance institutional capacity building, and guide the development and implementation of performance improvement action plans to support improved governance, leadership, and accountability. To inform activities geared towards health systems strengthening, capacity-building needs identified during PICAL assessments will also be used to facilitate targeted technical assistance, coaching, and leadership training in (1) public financial management; (2) analysis and use of data for improved disease surveillance and facility-level data reporting; (3) management of Human Resources for Health, taking gender into consideration in the recruitment and deployment of staff; and (4) use of the performance dashboard tool to equip provincial and HZ managers with real-time, data-driven, decision-making capabilities. Further, USAID IHP will optimize use of existing methods such as results-based financing; employ mobile phone-based surveillance technologies; and strengthen supply chain activities to support quantification, forecasting, and timely inventory replenishment. At the community level, USAID IHP will use the recently-developed MOH Community Dynamics strategy to improve stakeholder coordination and oversight functions. By facilitating collaboration of provincial, HZ and community stakeholders, this strategy aims to strengthen the capacity of Comités de Développement de l’Aire de Santé (CODESAs), civil society organizations (CSOs) and community-based organizations (CBOs) to be true partners in addressing social and behavior change (SBC) and mobilizing the demand for and uptake of improved health services. Activities to support community-level monitoring of health system performance will include streamlining community scorecard approaches; launching a toll-free fraud and complaints hotline number for reporting corruption, abuse or similar allegations; and providing rights-based education to communities. Capacity-building of CODESAs, select CSOs or CBOs will also take place through a Grants under Contract program. Together, this enhanced coordination capacity and multi-level collaboration will support more effective community stewardship of the health system, while demanding accountability of both local and provincial authorities. Objective 2: Increase access to quality, integrated health services in target HZs. Programmatic approaches related to Objective 2 focus on increasing health service demand, access, and quality in the program regions. A primary component entails scaling up health facilities that can provide essential, integrated and high-quality health services. Facility-based activities include renovating health infrastructures, equipping health facilities with drugs and medical supplies, and building knowledge and capacity among health workers so that health personnel can provide a package of integrated services related to Maternal, Neonatal, and Child Health (MNCH), nutrition, family planning, and reproductive health, WASH, malaria and TB. Interventions will also focus on improving health provider attitudes and interpersonal communications. As part of this approach, the project will implement a fraud and complaints hotline and reporting system to enhance health worker accountability. Using a cluster model strategy, the project will first prioritize building capacity in a high performing facility in a HZ, and once strengthened, use that health structure to provide support and outreach to facilities in the same HZ. The project aims to strengthen other facilities located in more remote locations over the course of the project. Community-based health activities are considered critical to increasing utilization of facility services and improving provision of essential health services, particularly in remote localities. Interventions designed to strengthen community-based health services will include recruitment of new CHWs, particularly women, training of CHWs on health promotion (with a focus on WASH) and integrated community case management, and training of facility-based health workers on community outreach and provision of 190 The Impact of USAID’s Integrated Health Program in the DRC health services at the community level. Community activities will be scaled up over time, with an initial focus on remote communities with access to supported health facilities. Interventions will also involve strengthening of referrals from community platforms and health centers to referral hospitals. A general emphasis will involve building collaboration with government health structures, United States Government and other donors by supporting and actively participating in central level meetings during which learning experiences, needs, and priorities can be jointly identified and discussed, and policy influenced. Objective 3: Increase adoption of health behaviors, including health services use in target HZs. Interventions related to Objective 3 are geared to increase the adoption of healthy behaviors and utilization of health services in the targeted provinces. The strategy aims to raise community awareness and knowledge of health care services and address barriers to optimal health care seeking, as well as to strengthen community engagement and social support which will enable healthy behaviors. Specific interventions will include a healthy family campaign comprised of a multipronged, educational program involving a family drama series focusing on common health problems and issues related to accessing facility and community-based health services, the care received, and satisfaction derived. Storylines disseminated through radio and text messaging will highlight sociocultural barriers that inhibit access to services and the practice of healthy behaviors and ways they can be overcome. Radio listening sessions will be organized to facilitate community discussions and reactions to scenarios presented during the drama series at the local level. Messages conveyed through the drama series will be complemented by interpersonal communication carried out by CHWs and CODESAs and supported by women’s organizations and other community-based groups through mobilization events. Open houses will be held to showcase improvements in health facilities and encourage utilization. The Champion Community model will be implemented to prioritize health areas and target audiences and develop work plans and monitor activities in the targeted areas. Mini-campaigns focused on addressing health problems will also be carried out according to specific and immediate needs. Efforts will be made to share lessons learned, harmonize strategies and improve approaches by collaborating and coordinating with other groups involved in SBC, including the following: key government institutions working on communications; government officials, implementing agencies and other stakeholders participating in coordination meetings (clusters, MCZ, head nurse) at the central, provincial and zonal level; and USAID staff and partners. The project aims to share SBC activity results to international audiences during academic conferences and through peer-reviewed, scientific manuscripts. At the more local level, coordination of SBC approaches will be carried out in conjunction with the HZ office, CODESAs and Cellules d’Animation Communitaire, with assistance provided to HZs during the development of their Operational Action Plans to ensure the overall goal of scalability of sound and effective messaging and activities that align with and contribute to the achievement of agreed-upon health goals. c. Targeting The project will be implemented in nine provinces located in three contextually different regions of the DRC including South Kivu and Tanganyika in Eastern Congo; Haut-Katanga, Haut-Lomami, Lualaba in Katanga; and Kasai-Central, Kasai-Oriental, Sankuru, and Lomami in Kasai. The war-torn region of Eastern Congo is relatively densely populated with a high population of internally displaced persons. Baseline Report 191 Despite strong humanitarian donor support over an extended period, Eastern Congo lacks adequate and sustainable health services. Katanga, the home of commercial mining, has extreme wealth that has had limited benefits to local populations. The northern part of the Katanga region has poor infrastructure. It should also be noted that Haut-Katanga, located in the southern Katanga region, recently experienced ethnic tensions and rebel attacks causing internal displacement. Transactional sex and internal mobility have caused increases in HIV and TB infection rates in the Katanga Region. Kasai is a vast, sparsely populated area with poor infrastructure. Recent militia activities have caused unrest and internal displacement within the region, contributing to an already fragile situation and causing additional strain on a weak health care system. The project will target all HZs located in the USAID IHP provinces offering services to residents living in the 178 targeted zones. Youth and gender equity will be an integral consideration in program implementation. The program is designed to be agile and results-focused, with project activities adapted according to ongoing measurement of the effectiveness of interventions and the particular context and health needs in each province. d. Project implementation plan The project started in July 2018 and will be implemented over a four-year period, with the possibility of a three-year extension. The project is led by Abt Associates with core partners International Rescue Committee and Pathfinder International. Seven niche partners with expertise in health programming, designing innovative approaches and research in fragile states, including DRC, will participate. The project team will work closely with government health officials at the central, provincial, zonal and health facility level in efforts to build capacity, leadership and ownership of the interventions, and sustainability. The project team will maintain ongoing coordination with USAID personnel, sharing progress updates of project activities and results during regularly scheduled meetings and reporting. 192 The Impact of USAID’s Integrated Health Program in the DRC 4. Theory of Change The USAID IHP theory of change addresses key areas of concern to influence and improve systems, services, and behaviors. The theory of change, which is presented in Figure 1, is being used by Abt Associates and its partners throughout the life of the program to validate assumptions, test the theory of change, and ensure that all activities and interventions effectively contribute to intended results. The theory of change extends the results framework to a more granular level, describing the relationships between proposed activities and interventions and these results. USAID IHP is intended to build the capacity of Congolese institutions and communities to ensure that the DRC health system effectively provides improved availability, access, and use of high-quality health services leading to better health of Congolese citizens. The program’s overall theory of change maintains that sustained capacity requires: ● An enabling environment supported by enhanced leadership and stewardship to ensure an accountable health system, and organizational systems and processes that assist managers and providers to carry out critical health functions at every level, coordinate activities between levels, and manage the delivery of high-quality health care; ● Skilled individuals at the facility and community level who have technical, managerial, and advocacy skills, and knowledge to ensure quality service delivery, people-centered approaches, and informed communities; ● Communities that have confidence and trust in the health system and are engaged partners; and ● Communities that are empowered to improve their own health through healthy behaviors and appropriate and timely use of high impact health services. According to the theory of change, the successful enhancement of systems and human capacity within DRC’s health sector will result in more effective stewardship of the financial, human, and programmatic resources in the health sector; enhanced capacity at provincial, health zone, facility, and community levels for management and delivery of quality services; and increased engagement of communities in mobilization, planning, and advocacy for health. By improving the MOH’s capacity (institutional and human) at all levels, the health sector will be able to more effectively and efficiently plan, monitor, and deliver programs and services. The improved availability of high-quality services – along with increased demand for care and improved confidence in the health system – will lead to greater service utilization and consequently better health outcomes and improved livelihoods for Congolese people. Baseline Report 193 Figure 1. USAID IHP Theory of Change. The theory of change emphasizes the complex nature of USAID IHP. Among the characteristics1 of the project are the following: • Multi-component and multi-level interventions (enabling environment interventions, service delivery interventions, health behavioral interventions) that will vary across provinces and time. • Multiple, changing processes (behavior of households and health workers, organizational structures and processes). • Multiple goals (strengthened health systems, improved quality of care, increased adoption of healthy behaviors). • Dynamic, heterogeneous contextual settings across the nine USAID-supported provinces. As will be described in Section 6 of this protocol, this complexity was considered in developing the research questions and evaluation methodology. 1 This approach for characterizing the complex nature of the project is based on guidelines for evaluation of complex interventions that were developed by the Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee (2019). 194 The Impact of USAID’s Integrated Health Program in the DRC 5. Research Questions The specific research questions that will be addressed in the evaluation are the following: 1. Did the expected changes in outcomes and impacts occur? a. Strengthen health systems, governance, and leadership at provincial, HZ, and facility levels in target HZs. i. Outcome: Enhanced capacity (institutional and individual) of provincial health offices, HZs, and facilities to plan, implement, monitor, oversee and supervise services ii. Outcome: Strengthened capacity of CSO’s and community structures to provide health systems oversight b. Increase access to quality, integrated health services in target HZs. i. Outcome: Increased availability of quality, integrated facility-based health services and commodities ii. Outcome: Increased availability of quality, integrated community-based health services iii. Outcome: Improved affordability of integrated health services c. Increase adoption of healthy behaviors, including health service use, in target HZs. i. Impact: Proportion of children under 5 for whom treatment/advice was sought for ARI, diarrhea, and fever ii. Impact: Proportion of children under 5 who slept under an Insecticide-Treated Net iii. Impact: Proportion of married women using any modern method of contraception iv. Impact: Proportion of children who received all eight basic vaccinations v. Impact: Proportion of pregnant women who attend four ANC visits vi. Impact: Proportion of newborns cared for with the Kangaroo method vii. Impact: Proportion of newborns placed on the breast within one hour of birth viii. Impact: Proportion of children under 6 months breastfed exclusively 2. If there were changes in healthy behaviors over the course of the study period, to what extent were these attributable to USAID IHP? 3. Did the project contribute to gender equity in health services and within the health system? 4. What factors enabled or limited the success of USAID IHP? a. Design/scope b. Implementation/management c. External environment/contextual factors d. Government decentralization Baseline Report 195 6. Evaluation Design Several issues were considered in developing the evaluation research questions and methodology. First, as is evident from the description of the project and the theory of change, USAID IHP is a large, complex health systems strengthening project that involves multiple components, interventions, and objectives. Second, the decisions regarding which types of interventions will be supported and where and when they will be introduced and scaled up will be made by USAID IHP management team in consultation with USAID, without any input from the D4I evaluation team. This effectively means that it will not be possible to randomize interventions supported by USAID IHP, which prevents the ability to estimate the impact evaluation of one or more specific health system strengthening interventions of interest. Third, while population-based household surveys will be conducted as part of USAID IHP, the surveys will be administered by Abt Associates as part of the project’s Activity Monitoring and Evaluation Plan, and not independently by D4I. In consultation with USAID, the USAID IHP management team has decided only to carry out the surveys in the nine USAID IHP provinces and not in outside provinces, which effectively removes the possibility that the household surveys can be used as a source of data for the D4I impact evaluation. Given these considerations, a decision was made in consultation with USAID to carry out two types of evaluation components for this study: a performance evaluation and an impact evaluation. As defined by USAID Evaluation Policy, performance evaluations incorporate before and after comparisons, but generally lack a rigorously defined counterfactual to control for factors other than the project or intervention that might account for the observed change, whereas impact evaluations assess the extent to which changes in health outcomes or service utilization over time are attributable to an intervention (USAID 2016). Table 1 below describes the study component (performance evaluation or impact evaluation) and sources of data that will be used to address each of the research questions. As described in Table 1, the performance evaluation aspect of the study will address: Research Question 1, which investigates changes over time in USAID IHP areas; Research Question 3, which examines the extent to which the project addressed issues of gender equity; and Research Question 4, which investigates factors that enabled or limited the success of the project. Data for this component of the study will come from multiple sources, including: the DRC’s routine health information system (DHIS2); household surveys; surveys of health care facilities, HZ offices, and provincial health offices; and key informant and in-depth interviews and focus group discussions. The impact evaluation aspect of the study will address Research Question 2 which investigates the extent to which changes in healthy behaviors are attributable to USAID IHP. The impact of USAID IHP will be based on a difference-in-difference with propensity score matching model, a non-experimental design, based on data from the DHIS2. 196 The Impact of USAID’s Integrated Health Program in the DRC Table 1. Overview of the D4I evaluation of USAID IHP Research Question Study Component Sources of Data 1: Did the expected changes in outcomes and impacts occur? Performance Evaluation DHIS2 USAID IHP Household Survey USAID IHP Health Facility Survey D4I Health Facility, Health Zone Office, and Provincial Health Office Surveys D4I Qualitative Key Informant Interviews, In-Depth Interviews, and Focus Group Discussions 2: If there were changes in healthy behaviors over the course of the study period, to what extent were these attributable to USAID IHP? Impact Evaluation DHIS2 3: Did the project contribute to gender equity in health services and within the health system? Performance Evaluation DHIS2 USAID IHP Household Survey USAID IHP Health Facility Survey D4I Health Facility, Health Zone Office, and Provincial Health Office Surveys D4I Qualitative Key Informant Interviews, In-Depth Interviews, and Focus Group Discussions 4: What factors enabled or limited the success of USAID IHP? Performance Evaluation D4I Qualitative Key Informant Interviews, In-Depth Interviews, and Focus Group Discussions a. Analysis of impact using a difference-in-differences model Model description: A doubly robust model that combines difference-in-differences with propensity score matching (DID-PSM) will be used to estimate the impact2 of USAID IHP on the provision of 2 It should be noted that, in the health evaluation research literature, the term ‘impact’ typically refers to the effects on health outcomes, such as lives saved, or disability-adjusted life years averted. However, in the health systems strengthening evaluation literature, “impact” often refers to the effects Baseline Report 197 maternal and child health care services and family planning services (as identified in Research Question 2). The unit of analysis will be the health zone. The treatment arm includes health zones in USAID IHP provinces (based on data from both hospitals and health centers/posts) in the pre-intervention (2017- 18) and post-intervention (2019-23) periods and the control arm includes comparable health zones without USAID IHP support. The DID-PSM estimates of project impact will include health zone￾level fixed effects to adjust for time-invariant factors including baseline differences between USAID IHP-supported health zones and control health zones as well as location and socio-economic characteristics. To the extent allowed by our data, which come from the health management information system, the DID-PSM model will also include time-variant controls for volume of cases across health care facilities, and socio-economic characteristics, and account for the staggered implementation of USAID IHP. To account for serial correlation in monthly health zone outcomes, standard errors will be clustered by health zone. By including health zone and month fixed effects, the DID-PSM model uses trends in outcomes in the control health zones as the counter-factual for what would have happened in the treatment group without USAID IHP. Sample: The health zones included in this component of the study will be: 1) health zones (n=178) in the nine provinces receiving technical assistance from USAID IHP (treatment arm); and 2) health zones (n to be determined) that are outside the nine USAID IHP provinces and NOT receiving assistance from a donor-supported health systems strengthening project comparable in scope to USAID IHP (control arm). Control health zones will be weighted to resemble treatment health zones using the propensity score matching technique. First, all health zones receiving donor-supported health systems strengthening interventions that are comparable in scope to USAID IHP between 2019 and 2023 and those that experienced an Ebola outbreak in 2018 or later will be excluded from the group of potential control health zones; all other non-USAID IHP health zones in the country will be retained as potential contributors to the weighted control group. The exclusion of health zones that received health systems strengthening interventions will be based on a list of donor-supported health projects compiled and periodically updated by the Ministry of Health for the purposes of the National Health Accounts. As of June 2019, examples of health systems strengthening projects that will be considered for exclusion from the study include the World Bank’s PDSS, the United Kingdom’s ASSP Project, and Belgium’s Memisa Project, which include that aim to strengthen the health system via improvements in health financing, management, and quality of primary health services. Next, the propensity score (i.e. probability of getting the treatment) will be calculated for each health zone based on a set of predictors that includes the mean values across the pre-intervention period (Table 2). These propensity scores will be used to generate weights for each health zone. We will weight each zone to recover the average treatment effect across the DRC by weighting by the inverse probability of treatment (1/propensity for the treated observations and 1/(1-propensity) for the control on service delivery or other aspects of health systems functioning (Adams et al., 2012). For the purposes of this evaluation, we use “impact” in the latter way, unless otherwise noted. 198 The Impact of USAID’s Integrated Health Program in the DRC observations). Sample balance will be tested on the array of available predictor variables averaged across the pre-period using these weights. The two samples will be considered balanced when the weighted means of control variables are within 10% of each other as measured by standardized differences. Propensity scores will be recalculated using higher order polynomials of predictor variables or interaction terms until this balancing condition is satisfied. Baseline Report 199 Table 2. Predictor variables for Propensity Score Matching. Category Predictor variable Basic characteristics • Health zone population* • Urban/rural setting • Old/new province • Performance-based financing program participant • Number of health centers/posts* • Number of hospitals* • Mean expenditures and receipts per health facility* Service delivery • Average length of stay of hospitalization* • Number of reported, facility-delivered newborns who received five components of Newborn Essential Care and three exams in the first six days of life* • Number of births assisted by qualified personnel at health facilities* Health status • Number of reported, facility-delivered newborns with low birth weight (<2,500 grams)* • Number of child clients 6 to 59 months of age with moderate acute malnutrition* • Number of child clients 6 to 23 months of age still breastfeeding* • Number of child clients under 6 months seen exclusively breastfeeding* *Indicates a DHIS2 indicator. The DHIS2 includes estimates of the total population for the catchment area of each health facility, but age- and gender-specific estimates are unavailable. DHIS2 indicators that measure service statistics are reported at the facility-level and will be aggregated to the health zone level for the purposes of the PSM modeling. 200 The Impact of USAID’s Integrated Health Program in the DRC Source of data: Data on services provided (treatment of ARI, diarrhea, fever, new family planning users, child vaccination, antenatal care, and skilled delivery assistance) will come from the DHIS2, an electronic database that compiles data reported from public and government-supported hospitals and health centers/posts. The DHIS2 was scaled up to all provinces in the DRC in 2016 and this study will use pre-intervention data over the period January 2017 to December 2018 and post-intervention data over the period 2019 to 2022 (and to 2025 if the project is extended for an additional three years). The DHIS2 data are available monthly for each participating health care facility and will be aggregated to the health zone level for three-month periods. For the purposes of the impact evaluation, the dependent variables will consist of the numbers of individuals who received services, as opposed to coverage rates. Ideally, coverage rates based on the number of individuals in need of services would be used as the dependent variables. Unfortunately, this is not possible, due unavailability of age- and gender-specific health zone populations in the DHIS2. b. Analysis of changes over time in project areas Facility-, health zone office-, and provincial health office-level changes. To evaluate the progress of IHP-supported areas on service readiness, service quality, and service utilization, a separate analysis component will use data from provincial health offices, HZ offices, health facilities and (health centers and hospitals), which we will collect, and data from baseline, midline, and endline facility and household surveys that USAID IHP is collecting independently. This is the performance evaluation component of the overall study. As USAID IHP is operating in all HZs within its nine provinces, it is not feasible to identify and survey a comparable control group of HZs within these same provinces. Comparisons will be made in selected indicators between baseline (Year 1), midline (Year 4), and endline (Year 7). To conduct the comparisons, the absolute changes for each indicator value will be compared between survey waves using t-tests or Chi2 tests, both overall and for the three regional sub-groups. We will also stratify results by sex and age when applicable. Table 3 lists the indicators that will be assessed. Reference sheets for each indicator can be found in Appendix A. Baseline Report 201 Table 3. Key outcome indicators by topic area and outcome measured Service quality 1 Increased availability of quality, integrated facility-based health services and commodities a Percent of health centers that offer select MOH Minimum Package of Preventive and Curative Activities b Percent of hospitals that offer select MOH Complementary Package of Activities c Percent of facilities offering a permanent method of family planning d Percent of facilities meeting minimum standards with regard to essential supplies and equipment to support provision of long-acting or permanent methods of contraception e Average provider attitudes and interpersonal skills f Percentage of health facilities that incorporate patient feedback by intensity of incorporation g Percent of health centers that do not require payment before treatment of emergency cases h Percent of health facilities with a private delivery suite i Percent of facilities with improved sanitation facilities j Percent of eligible women who were referred to a higher-level of care (i.e. hospital) for prevention of mother-to-child transmission (PMTCT) of HIV k Percent of facilities with adequate infection control equipment l Percent of facilities with electricity m Percent of patients who report passing the nearest IHP-supported health center or foregoing care due to quality of services. n Percent of patients who report long wait times for care o Percentage of women who are discharged at least 24 hours after a normal delivery p Percent of facilities that have selected tracer drugs in stock on the day of the survey q Percent of women giving birth who received uterotonics in the third stage of labor r Percent of facilities with adequate staffing numbers/mix according to government guidelines and patient volume s Percent of providers who respond correctly to clinical vignettes t Percent of health facilities with all basic equipment u Number of supported facilities offering a package of youth-friendly family planning services v Number of supported facilities offering a package of comprehensive sexual- and gender-based violence (SGBV) services 2 Improved affordability of integrated health services a Percent of health centers with a posted fee exemption policy for indigent patients b Percent of health centers that accept payment after treatment c Percent of facilities that detain patients until their fees are paid d Percent of patients who report having to borrow money or sell possessions to pay for services 3 Increased availability of quality, integrated community-based health services a Percent of health areas in which community health workers provide family planning information, referrals, and/or services b Percent of health areas with an Integrated Community Case Management (iCCM) site Leadership and Governance 4 Enhanced capacity to plan, implement and monitor services a Participatory Institutional Capacity Assessment and Learning (PICAL) participation and score b Leadership style (i.e. collaborative, autocratic or laissez-faire) c Autonomy of the health facility/office d Percentage of health workers receiving performance-based incentives 5 Improved transparency and oversight in health service financing and administration a Percent of USG-supported health facilities that were audited by health zone or provincial-level authorities in the past 3 months 202 The Impact of USAID’s Integrated Health Program in the DRC b Percent of USG-supported health zones that were audited by provincial or national-level authorities in the past 3 months c Percent of USG-supported DPS that were audited by central or national-level authorities in the past 3 months d Health worker supervision 6 Strengthened capacity of CSOs and Community Structures to provide health system oversight a Community monitoring and oversight b Percent of supported CODESAs that are woman-led 7 Improved effectiveness of stakeholder coordination at the provincial and health zone levels a Percent of health zones and provincial offices that have increased communication with key stakeholders 8 Improved disease surveillance & strategic information gathering & use a Percent of USG-supported provinces and health zones reporting Maladie à Potentiel Epidémique (MAPEPI) District Health Information System 2 (DHIS2) cases within 24 hours 9 Improved management and motivation of Human Resources for Health a Percent of health workers and managers receiving training in Human Resources Management using the iHuman Resources Information System (iHRIS) b Percent of health workers and managers who received training in Human Resources Management using the iHuman Resources Information System (iHRIS), and say they are using new skills/knowledge on the job c Health worker satisfaction d Health worker motivation 10 Strengthened collaboration between central and decentralized levels through sharing of best practices and contributions to policy dialogue a Percent of health zones and provincial offices that have increased communication with key stakeholders Data from facilities will include information on staffing, service delivery, infrastructure and equipment, information systems, and governance. Surveys at hospitals and health centers will also include medical record reviews to assess quality of care. Clinical vignettes on topics such as antenatal care, treatment of childhood diarrhea, and treatment of fever will be administered to clinicians (physicians, nurses, midwives as appropriate to the case) during the health worker survey. Responses to vignettes will be analyzed to assess health workers’ knowledge and practices relative to the standard of care. Vignettes will be re-administered during each survey wave, with changes in responses over time analyzed. One component of the hospital and health center surveys will be quality assessments of the data reported in the DHIS2 (i.e. comparing DHIS2 data to data recorded in facility records). If data is of high quality, this will support its use in the impact evaluation component of study. If data quality is poor, we will assess whether it is randomly or systematically poor and adjust the analysis accordingly. Household-level changes. The population-based household surveys that USAID IHP is administering will include data on the health care services and behaviors that the project is supporting, including family planning, antenatal care, skilled-delivery assistance, immunization, and treatment of childhood illnesses. In addition, questions on the exposure to the various interventions that will be supported by USAID IHP will also be included in the household surveys. As with the facility- and health-worker data, changes will be analyzed for the entire sample and by the three regional sub-groups. The Lives Saved Tool (LiST) (Walker et al., 2013) will be used to evaluate the health impacts of USAID IHP at midline and endline. The LiST estimates of lives saved will be calculated at program baseline based on the 2019 population-based household and service provision assessment surveys that will be Baseline Report 203 conducted by Abt Associates, and then again at midline and endline based on similar surveys to be carried out in 2022 and 2025. Actual changes in coverage between baseline and midline and between baseline and endline will be compared to the changes targeted. The LiST requires a number of data fields to calculate lives saved. Mortality rates and causes of death will be derived from the UN inter-agency group for child mortality estimation and the WHO maternal and child epidemiology estimation. Demographic projection data are derived from the most recent Demographic and Health Survey or its equivalent. However, in order to obtain estimates of lives saved for the USAID IHP targeted intervention areas, sub-national estimates for causes of death, health services coverage, population by age/sex, stunting and wasting, breastfeeding, and total fertility rate will need to be compiled. This will likely require drawing from the Abt-run surveys mentioned above as well as the 2017-18 DRC Multiple Indicator Cluster Survey, published literature, and any other relevant survey data generated from the USAID IHP consortium. The LiST will allow for estimates of lives saved by target population (mothers, newborns, and children <5) and target region based on the USAID IHP objective of increasing access to quality, integrated health services. Lives saved estimates will be independently calculated as a result of changes to coverage rates of curative care services for children, contraceptive use among females, childhood vaccinations, and attendance of antenatal care visits. This segmented approach will effectively yield intervention-specific impacts of the number of lives saved. Taking into consideration estimates of lives saved based on multiple scenarios of health services coverage rates, comparisons can be made for population-specific mortality rates. Line graphs will show mortality rates under scenarios where baseline coverage estimates are maintained and where IHP-DRC coverage targets are achieved. Additional plots will be made utilizing the interpolation function of the LiST given actual mid and endline health services coverage estimates as reported by the IHP-DRC and/or D4I teams. The D4I team will also explore the feasibility of conducting interrupted time series analyses on DHIS2-derived domain-specific monthly case counts for each of the targeted populations as a result of scaling up intervention coverage/use. c. Analysis of enabling or limiting factors To assess the perceptions of various stakeholders on the implementation of USAID IHP-supported interventions, qualitative research involving a mix of methods will be carried out during the baseline, midline and endline evaluations with the aim to complement the quantitative data analysis. During the baseline evaluation, we will carry out research at the central level (Kinshasa) and in one province in the Katanga Region to assess the status of health systems and to understand details of the USAID IHP interventions and plans for implementation. At the central level, key informant interviews will be carried out with government officials, donors, and health care implementers to learn about facilitators and barriers to health systems strengthening. We have selected the province of Lualaba to conduct an initial in-depth analysis of institutional capacities and health delivery systems from the provincial down to the facility and community levels. The selection of Lualaba was based on the fact that the province was newly formed in 2015 and some of the Lualaba health zones had received assistance from the USAID IHP predecessor project, implemented by Management Sciences for Health. During the evaluation, we plan to compare data collected in zones that had and had not received previous IHP support. In addition, Lualaba will be one of three provinces where the healthy family campaigns (HFC), a central component to the USAID IHP behavioral change strategy, 204 The Impact of USAID’s Integrated Health Program in the DRC will be launched from the outset of the project. In Lualaba, government officials and USAID IHP program implementers will act as key informants at the provincial level. Two HZs with comparable health indicators, including one where the USAID IHP predecessor project led by Management Sciences for Health was implemented and one where the predecessor project was not implemented, will be identified for additional data collection. In these HZs, the chief medical doctor (MCZ) or an assistant will serve as key informants. In-depth interviews with reference hospital clinicians and administrators, health center nurses, CODESA and CHWs and facility-based observations will be employed to assess health service delivery, particularly in regard to child health services related to treatment of malaria, ARI and diarrhea, distribution of insecticidal nets, and vaccination coverage. Finally, focus group discussions will be administered with caregivers of children under five years of age to learn about care seeking behaviors and treatment practices associated with the leading causes of child morbidity and mortality, availability and utilization of Insecticide-Treated Nets, and access to essential children’s vaccinations, as well as perceptions of the delivery of facility and community level health services. A more detailed description of the qualitative methods is presented below. During the midline and endline data collection, we anticipate using a similar mixed methods approach but will conduct research in all three regions (Eastern Congo, Katanga, and Kasai) targeted by IHP. Data collected at midline and endline will concentrate on the activities the project has carried out, perceptions related to strengths and weaknesses of the interventions, and the extent to which the project was implemented according to plan. We will also explore the positive and negative consequences of the USAID IHP project, as well as assess the USAID IHP project’s impact on the functioning of activities, particularly related to child health. During all phases, data will be collected from a wide range of respondents including stakeholders involved in the design and implementation of interventions, service providers, and beneficiaries, with the goal to use environmental, methodological, and data triangulation to validate and interpret the research findings. d. Use of evidence-based population estimates. The DRC has not conducted a national census since 1984. The official government population estimates are projections from that census. We will incorporate more evidence-based population estimates in the event study and difference-in-difference analysis; these estimates are based on micro-census data extrapolated to the population level using satellite imagery and will be available from Oak Ridge National Laboratory and FlowMinder in early 2019. Baseline Report 205 7. Limitations and Threats There are a number of limitations of the evaluation, as well as threats to carrying out the evaluation as planned. First, due to the data limitations described in the previous section, the impact evaluation component of the study will only investigate the impact of USAID IHP on proxy indicators related to service provisions, including treatment of childhood illnesses, contraceptive use, vaccinations, and antenatal care. Because data on health outcomes, service quality, and health systems governance and leadership will not be available from non-project areas, impacts on these aspects cannot be rigorously assessed. However, to descriptively explore these aspects, a performance evaluation will be carried out using both quantitative and qualitative data that will be collected in the nine USAID IHP provinces to explore changes in proxy indicators of all three USAID IHP objectives – health systems strengthening, quality, integrated health services, and healthy behaviors – as well the factors that enabled or limited the success of the project. Second, the impact evaluation of the study will be based on routine data from the DHIS2. While it is expected that the advantages of using a research design based on DHIS2 data (i.e. numerous, repeated health zone observations over extended periods and the real-time indicators of service coverage) allow for power and cost advantages over a research design based on intermittent population-based surveys, poor data quality nevertheless remains a threat to the evaluation, due to inaccurate data on counts of services provided (numerators) and the populations that are served (denominators). This could lead to two consequences. First, poor data quality could add spurious variability to the dependent variable. If it is random measurement error, it will add to the variation of the random error in the model with the consequence of larger standard errors in the estimated coefficients and increasing the chances of not finding significant effects when there is impact. This is a common problem in research studies based on data from management information system, and there is little that the evaluator can do to address this issue. Second, the measurement error could potentially evolve over time as data quality improves with the improvements in the data reporting systems. Since USAID IHP aims to improve DHIS2 data quality, these improvements could be different in treatment and comparison areas, which will create a type of endogeneity in the program variable of the model that varies over time, so it is not controlled by the fixed effects. This heteroskadasticity should be accounted for through the estimation of robust standard errors. In order to address routinely assess data quality, the D4I evaluation team plans to partner with BlueSquare, a USAID IHP partner that is responsible for compiling and assessing DHIS2 data in all nine USAID IHP provinces, to carry out the same assessment and adjustment procedures to DHIS2 data from non-project provinces. Moreover, data quality will be assessed as part of the D4I health facility surveys that will be carried out at baseline, midline, and endline, as will be described later in this protocol. Third, while it will not be possible to rigorously assess the impact of USAID IHP on health outcomes, the performance evaluation component of the study will include a LiST analysis to estimate the number of lives saved in IHP provinces as a result of changes in service coverage, based on demographic projections and health services coverage estimates from the USAID IHP population- 206 The Impact of USAID’s Integrated Health Program in the DRC based household surveys and service provision assessment surveys to be conducted by the project at baseline, midline, and endline. The LiST only provides estimates for lives saved and will not allow for estimates of other health impacts such as morbidity. Additionally, the LiST treats interventions as if they are delivered at full quality thereby assuming a degree of effectiveness that may not represent reality. However, the effect sizes for specific interventions are represented by global values from systematic reviews and meta-analyses, large-scale randomized controlled trials, and, in the absence of published data, from Delphi estimations (expert opinions). Further, there are over 60 indicators that can be calculated in the LiST software, but only about 20 are directly measured by large-scale, population-based household surveys (e.g. DHS/MICS). For example, antenatal care visits are directly measured, but relatively conservative coverage estimates for individual components of antenatal care are estimated from the parent indicator Fourth, the D4I and USAID IHP surveys that will be used as the source of baseline data will be administered no earlier than June 2019, eleven months after USAID IHP started operations (in July 2018). This timing would be problematic if USAID IHP’s implementation activities had already begun, as it would bias the performance evaluation component of the study. However, based on previous meetings with USAID and the USAID IHP management teams, one of the key purposes of the USAID IHP surveys is to provide evidence that can be used to comprehensively assess the health systems needs within the nine provinces, and that USAID IHP will withhold most of its interventions until after the survey data has been analyzed, estimated to be no earlier than July 2019. Fifth, the DRC is an unstable environment, and there is a possibility that both the implementation of USAID IHP as well as USAID IHP and D4I survey activities could be affected by political and social unrest over the project period. This is not likely to affect the impact evaluation component of the study, which relies on routine health information system data, but it could affect future surveys if the data collection teams cannot safely travel to sampled provinces, health zones, and facilities affected by unrest, should that occur. Baseline Report 207 8. Quantitative Data Requirements, Collection, Management, and Use a. Data sources Routine data: Routine data on service delivery will be obtained from the DRC’s DHIS2 system. The data are routinely updated onto a web-based platform, and D4I has been granted permission to access the data by the Government of the DRC. Primary quantitative data: D4I will partner with the Kinshasa School of Public Health (KSPH) to collect primary quantitative data (the scope of work and fieldwork plan can be found in Appendix B). Quantitative data will be collected through surveys of provincial health offices, HZ offices, hospitals, and health centers. Surveys of hospitals and health centers will include a medical record review and a module for health workers. Table 4. Sampling strategy Survey Module Sampling Frame Sampling Methodology e Size Provincial health office f Provinces All in selected provinces 6 HZ office f HZs All in selected provinces 121 Health center/post USAID IHP records Simple random sampling (3 per HZ) 363 General reference hospital Census Purposive sampling/census (1 per HZ) 121 Health worker: health center Duty roster of all physicians, nurses, and midwives responsible for providing health care services on duty in the selected facility the day of the survey* Select all who meet criteria (average of 3 per facility) 1,089 Health worker: hospital Duty roster of all physicians, nurses, and midwives responsible for providing maternal or child health services on duty in the selected facility the day of the survey* Simple random sample (1 doctor, 1 nurse, 1 midwife per facility) 363 Medical record review: health center/post and hospital List of women who attended an ANC visit during the last calendar month at facilities selected for the health facility survey Simple random sample of medical charts (10 per health facility or all that meet criteria if less than 10 meet criteria) 4,840 List of women who were discharged or died after delivery during the last calendar month Simple random sample of medical charts (10 per health facility or all that meet criteria) 4,840 List of infants who were born alive during the last calendar month at facilities selected for the health facility survey Simple random sample of medical charts (10 per health facility or all that meet criteria if less than 10) 4,840 208 The Impact of USAID’s Integrated Health Program in the DRC *Excludes pharmacy staff, laboratory staff, and those with clinical credentials who primarily work in an administrative capacity. Surveys will be conducted at baseline (Year 1), midline (Year 4), and endline (Year 7). If possible, data collection will occur during the same months in each wave. Surveys will contain questions designed to assess the level at which governance and management are being carried out, the degree to which the health system is providing quality services, whether and why the population is using these services, gender and age dynamics within the health system, and the contributions of IHP. For each survey administered, a GPS point will be recorded. The survey instruments can be found in Appendix C. All instruments will be professionally translated into French. Data collectors will administer the survey verbally and record responses on tablets using OpenDataKit, with the exception of medical chart review, which the data collectors will conduct independently using pre-developed templates. Surveys will be pilot-tested in Kinshasa prior to deployment. Pilot testing will assess the formulation of questions and responses, skip patterns, and the tablet-based collection method. USAID IHP facility and household survey data: At baseline, USAID IHP will conduct a census of health facilities (from hospitals to health posts) in all nine provinces. This survey is designed to be a rapid assessment that Abt will use to tailor its approach throughout the project areas. Facility-based data collection for this evaluation will be conducted in a sample of facilities and will involve more in-depth questions. Therefore, the two surveys will be conducted independently. However, both will utilize the same sampling frame and facility identification system so that the two data sets can be linked. USAID IHP plans to conduct similar surveys on a subset of facilities at midline and endline. USAID IHP will also conduct a baseline survey in households across all nine provinces (n=9,000 households). This survey will include separate modules for men and women. Data for Impact (D4I) is providing input on the sampling approach and questionnaires. USAID IHP plans to repeat the survey in the same villages at midline and endline. D4I will incorporate data from both USAID IHP’s facility and household surveys in the process evaluation. Table 5 indicates which data are to be collected by IHP and D4I. Table 5. Sources of primary data used in the evaluation, by organization Implementer Data collection method Baseline Midline Endline IHP Health facility census Health facility survey or census Health facility survey or census Household survey (9 provinces) Household survey (9 provinces) Household survey (9 provinces) D4I Health facility survey (more in-depth than IHP, but fewer sites in 6 provinces) Health facility survey (more in-depth than IHP, but fewer sites in 6 provinces) Health facility survey (more in-depth than IHP, but fewer sites in 6 provinces) Health zone and provincial health offices (census in 6 provinces) Health zone and provincial health offices (census in 6 provinces) Health zone and provincial health offices (census in 6 provinces) Baseline Report 209 In-depth and key informant interviews In-depth and key informant interviews In-depth and key informant interviews USAID IHP quarterly report and annual work plans: As mentioned earlier, the specific interventions that will be supported by USAID IHP will vary across the health zones in the nine USAID provinces. In order to describe USAID IHP implementation activities, the D4I team will review USAID IHP quarterly reports, as well as the Annual Work Plans of the IHP USAID-supported health zones. Both sets of documents will be provided by the USAID IHP monitoring and evaluation team. b. Data collectors and training Data collectors will be recruited from the surveyed regions and efforts will be made to recruit comparable numbers of male and female data collectors. All data collectors will be fluent in French and the local regional language. Many will have prior experience in survey administration, and all will receive comprehensive, multi-day training from D4I and KSPH. We will recruit data collectors with clinical backgrounds (physicians, nurses) to conduct the medical record review and administer clinical vignettes. Training will cover research methods, data collection procedures, research ethics and informed consent, and the use of tablets for data collection. c. Sampling and recruitment The sample of provinces contains the province containing the capital city of each of the three regions. We then sampled provinces in a way that created a mix of conditions within the sample. We ensured 4 out of 6 of the selected provinces were newly-created by the 2015 decoupage. At the start of USAID IHP, four provinces (Lualaba, Haut Katanga, Sud Kivu and Kasai Oriental) had already begun capacity-building activities, while the remaining five had not. Again, 4 out of 6 of the selected provinces had ongoing activities, while two did not. Table 6 illustrates the selection strategy for provinces, with selected provinces identified with an *. Table 6. Selection methodology for provinces. Region Province Capital Age Activities Selected Eastern Congo Sud Kivu* Yes Old Ongoing Yes Tanganyika* No New New Yes Kasai Kasai Oriental* Yes Old Ongoing Yes Sankuru* No New New Yes Kasai Central No Old New No Lomami No New New No Katanga Haut Katanga* Yes New Ongoing Yes Lualaba* No New Ongoing Yes Haut Lomami No New New No *Province selected for primary quantitative data collection 210 The Impact of USAID’s Integrated Health Program in the DRC Within each selected province, we will attempt to survey all selected HZ central offices. If a HZ office refuses to participate, it will not be replaced. However, the rest of the planned surveys will be conducted within that HZ as long as permission can be obtained. Within each HZ, we will randomly select three health centers/posts. Once sampled facilities have been selected, data collectors will call via phone or approach the facility and speak with the facility head. If the facility head agrees to participate, we will conduct surveys with that facility and its associated health workers. If the facility does not agree to participate, the next closest health facility in the HZ will be invited to participate. If a health worker refuses, they will be replaced if there is another eligible health worker present. All refusals will be recorded in OpenDataKit. We will attempt to survey the same provinces, health zones, and facilities during all three waves of data collection. If a health center/post refuses to participate in a later wave, it will be replaced by the closest health center/post. d. Data collection Survey data will be collected on mobile devices. Data will be cleaned by thoroughly checking whether indicator values fall within plausible ranges, confirming whether skip patterns have been respected and assessing whether survey responses are consistent with previous responses. In addition, for many indicators that are in common with those in the DRC DHS planned for 2020, the validity of the results will be assessed by comparing data from the baseline survey with the DHS data. 9. Qualitative Data Requirements, Collection, Management, and Use a. Data sources Key informant interviews: Key informant interviews will be conducted with a range of experts from the central to the zonal level working on health systems strengthening, with key informants either working directly on USAID IHP activities or involved in other health service delivery strategies in the DRC. Key informants will be selected purposively based on their expertise and/or role in USAID IHP. In the capital Kinshasa, we aim to carry out key informant interviews with MOH collaborators on USAID IHP (3), Abt senior staff involved in program development, implementation, and monitoring and evaluation (3), and USAID IHP staff and partners leading behavioral change interventions (2-3) and data platform development and technologies designed to enhance data-driven decision making (1) (see Table 7). USAID staff including the head of health and national representative overseeing USAID IHP will also be interviewed. We also aim to interview the Project Director of the USAID IHP predecessor project, as well as a representative of the World Bank overseeing performance-based financing activities. In the province of Lualaba, key informants will include a provincial health office official, a representative from the inspector’s office, a representative from the MNCH program, an USAID IHP representative overseeing interventions in the Katanga region, and the chief medical officer or the chief medical officer assistant in Baseline Report 211 the two HZs selected for data collection. We also anticipate interviewing Abt and USAID headquarters staff involved in the project development and oversight. Key informants will provide initial information on the strengths and weaknesses of health systems in the DRC, perceptions of the functioning and shortcomings of governance in the provision of health care, and perceived needs related to institutional strengthening and support to DRC health systems and service delivery. Topics to be examined will focus on institutional and individual capacities to plan, lead, execute, and monitor health action plans; understandings and implementation of evidence-based decision making; roles in budget development and financial management; personnel recruitment and performance management, supervision structures and other accountability mechanisms; capabilities related to leadership and personnel, administrative, and financial management; utilization of existing support systems such as the integrated human resources information system, results-based financing and community scorecards; disease surveillance, information gathering strategies and data utilization; perceptions of community-based services and capacity strengthening needs; and collaboration, coordination and information sharing with other stakeholders. Additional information will be collected on staff satisfaction, compensation, the potential for advancement, and motivation for job performance improvement, as well as perceptions of transparency, accountability, issues related to gender and gender￾based discrimination, and capacity building needs. At the provincial and zonal level, we will assess inspection roles and capabilities and the evolution of decentralized services. At these levels, we will also collect information on contextual factors potentially affecting health care provision and demand and services offered prior to the USAID IHP intervention. Topics covered with individual key informants will depend on their expertise and involvement in the project. In-depth interviews: In-depth interviews will be administered to health providers at the facility level and CHW’s at the community level in the two HZs to examine essential child health care packages. In each zone, we will conduct interviews with the hospital administrator and one clinician working in the pediatric ward in the reference hospital. In addition, based on DHIS2 data related to utilization of curative care for children under five years of age seeking treatment for malaria, ARI, or diarrhea, we will identify one health center with relatively high volumes of maternal and child cases, and one with relatively low volumes of maternal and child cases where in-depth interviews will be carried out with a head nurse or an assistant head nurse. Interviews with facility-based workers will focus on the availability of essential child health services, quality of care and factors affecting demand. Topics we aim to investigate relate to health facility staffing, training, roles and organization; availability and integration of essential child health services and utilization of treatment algorithms; the condition of facility infrastructures and equipment and availability of commodities and medical supplies needed to carry out key child health services at the facility and community level; referral systems from the community level to health centers and reference hospitals; health provider attitudes towards respectful care, particularly in relation to gender, age and ethnicity; perceptions of innovative financing approaches to reduce barriers to health care access by poor members of the population; monitoring of health service performance and data utilization; medical supply chain inventory management, reporting, requisition and transport; and participation in coordination meetings led by the MOH or donors to ensure sharing of field level learning experiences. We will also assess current outreach strategies and ways to strengthen community engagement and efforts to elicit information from local populations regarding health problems and needs. Topics covered with in-depth 212 The Impact of USAID’s Integrated Health Program in the DRC interview informants will be guided by their role in the provision of health care and involvement in the IHP project. In-depth interviews will also be conducted with CHWs, including the CODESA president or vice￾president and a CHW selected by the CODESA informant, with the selection criteria targeting long￾tenured and high performing CHWs. During interviews, we will assess the existence of community-based activities aimed to encourage SBC-related to healthy behaviors and to mobilize the demand for and uptake of improved health services. Questioning will focus on health promotion, outreach, community mobilization, referrals and reporting; training and roles related to treatment (including the existence and functionality of integrated community case management sites), prevention and monitoring of services; promotional tools and adult learning approaches employed; and the involvement of CBO’s such as the Cellules d’Animation Communautaires, coordination of community activities, and linkages with facility￾based staff and services. Both facility-based providers and CHWs will be asked about local barriers to health care utilization, ways to improve affordability and quality of health care services, perceived biases in health care provision and utilization including those related to gender and youth, supervision of work activities, and previous and ongoing project activities in the area. Additional questions will focus on job satisfaction, compensation, and motivation and capacity building needs. We will also assess informant’s participation in technical and coordination meetings that present an opportunity to share lessons learned with other stakeholders. Focus group discussions: Group discussion will be conducted with caregivers of children five years of age and under, with a target to carry out one focus group in each health center’s catchment area. Caregivers will be identified by CODESA members and CHWs, with the goal to include a mix of caregivers who regularly and do not regularly attend well baby visits in discussion groups. During group discussions, we will examine caregivers’ knowledge of facility and community-based child health services, including preventive and curative care, community mobilization and social support and educational activities, and efforts to improve health care access among vulnerable populations. We will also explore perceptions of treatment and preventive services for childhood illnesses at the facility and community level, health center infrastructures for child health services, health providers capacity to treat the leading causes of childhood mortality and morbidity, availability of essential drugs, vaccinations and other supplies, health provider attitudes and interpersonal communications, health care costs and affordability, and ways to improve services. Discussion topics will also revolve around household decision making and care-seeking behaviors related to treatment for malaria, diarrhea and ARI, distribution and utilization of insecticidal nets, and access to basic vaccinations for children, with a focus on identifying sociocultural barriers to the use of formal health services and practice of key healthy behaviors. We will explore exposure to radio, text and more local awareness-raising messaging (e.g. through CHWs, CODESA, schools, churches, community organization, etc.) related to child health. During group discussions, we will also assess whether efforts such as the use of community scorecards or group meetings have been made to elicit information from community members to assess their health needs. Observations: Using a semi-structured observation guide, observations will be conducted in health centers and general reference hospitals to assess the condition of infrastructures where child health services are provided; availability of drugs, materials and supplies, and equipment to provide essential child health care; official fee schedules; and any evidence of activities to discourage fraud and increase transparency. In addition, we will observe provider-caregiver interactions during treatment of sick Baseline Report 213 children, with a focus on quality of care including whether the health provider greets the caregiver, asks questions related to the illness episode, physically examines the child, explains the diagnosis and treatment regimen, provides medication or a prescription, recommends a follow-up visit, and confirms whether the caregiver understands the diagnosis and recommended treatment or has additional questions. We will also carry out observations of the queuing process and the waiting area, whether triage is carried out, the waiting time, and if patients are turned away without being seen by a provider. We plan to take photographs of the exterior and interior of the health facilities, equipment and materials to record conditions in the health centers. Photographs will not be taken of clientele including caregivers and children observed during the health provider-caregiver interactions. Table 7. Targets for key informant and in-depth interviews, group discussions and observations according to research methods and respondent type. Research methods and types of respondents Total n Key informants Central level/Kinshasa and Headquarters MOH officials collaborating on IHP activities 2-3 USAID IHP senior staff involved in program development, implementation, and M&E 5 USAID IHP staff and partners leading behavioral change interventions 2-3 USAID IHP partners leading data platform development and technologies 1 USAID representatives in Kinshasa overseeing IHP activities 3 Chief of Party of USAID’s predecessor project 1 Representative of the World Bank overseeing performance-based financing 1 Lualaba DPS representative 1 MNCH program representative 1 USAID IHP representative overseeing interventions in the Katanga region 1 Inspector’s office representative 1 Chief medical officer or the chief medical officer assistant in two health zones 1 In-depth interviews Reference hospital administrator and clinician on the pediatric ward 4 Health center head nurse or head nurse assistant 4 CODESA president or vice president 4 Community health care provider 4 Village leader 4 Focus group discussions (4 groups of 8-12 participants) Female caregivers of young children < 5 years old (age 18 or older) 48 Observations (12-19 pairs of providers and clients) Health provider-client interactions during treatment of childhood illness 24-38 b. Data collectors and training A 4-5-day training workshop will be carried out prior to the study. We anticipate training 4-6 potential male and female research assistant candidates with prior experience using qualitative methods. Trainees 214 The Impact of USAID’s Integrated Health Program in the DRC will be introduced to the USAID IHP approach and activities, qualitative study objectives, research methodology, research ethics and informed consent, and study instruments. Initial sessions will focus on the USAID IHP objectives, structure and interventions. Subsequently, trainees will be introduced to the qualitative study design and data collection techniques to be employed during the study, with a focus on open-ended questioning and recommended approaches to be used when interacting with respondents. Extensive time will be devoted to each of the research methods, including the sampling procedures, data collection approaches, and instruments. Training will also include theoretical and practical sessions related to research ethics and obtaining informed consent. Field testing of the instruments will be conducted, with revisions made during the training period. The training will be led by the medical anthropologist and the supervisor overseeing the study. Two trainees who perform well will be selected to take part in the study. After selection of the data collectors, we will carry out additional training if deemed necessary. The study team will be comprised of the medical anthropologist, the study supervisor, and two research assistants. Initial data collection will involve key informant interviews at the central level. Subsequently, key informant interviews will be carried out in the provincial capital of Lualaba. Once completed, the research team will travel to one of the study HZs to interview the chief medical officer and identify a high and low performing health area in the HZ. Key informant interviews will be carried out by the medical anthropologist and the research supervisor. Key informant interviews will be open-ended and last approximately 1-1.5 hours. Interviews will be carried out in informant’s offices or in another location where privacy can be maintained. We will use an interview guide with questioning adjusted to informants’ backgrounds and expertise (see Appendix D). Data collection will be an iterative process, with the information gathered serving to inform subsequent key informant interviewing until data saturation is reached. Information collected from key informants will also be used to inform the content of in-depth interviews, focus group discussions and observations. We anticipate that during each evaluation (baseline, midline, and endline) some key informants will be interviewed on multiple occasions, with the goal to establish a relationship of trust, which will influence the willingness of informants to open up and enhance the data quality. Subsequent to the field visit to Lualaba, questioning of key informants located at the central level will allow us to clarify and interpret information gathered through the other data collection methods at the provincial and HZ level. In-depth interviews with health providers including nurses, CODESA members and CHWs will be carried out in the health area facility or a setting of their preference. Data collectors will follow a semi-structured guide (see Appendix D). Efforts will be made to conduct the in-depth interviews in a private setting, with each interview lasting no longer than 1hour and 15 minutes. If the research assistant is unable to address all of the topics listed in the interview guide, the assistant will schedule a follow-up session to complete the interview at a later time. Group discussions will be comprised of 8-12 child caregivers and held in a space where relative privacy can be maintained, such as a school or church. Discussions will be led by a moderator who will guide the questioning; a second research assistant will record notes to facilitate data transcription. Group discussions will last no longer than 1hour and 30 minutes. A guide based on the research objectives and primary themes and preliminary study results procured through the other data collection methods will be used (see Appendix D). Baseline Report 215 Observations of care provider-patient interactions will be carried out two mornings in each health center, starting early in the morning in the health center location where curative care is offered. Observations will continue until caregivers no longer seek treatment for child illnesses. We will also make direct observations of the condition of the health infrastructure, supplies and equipment available, the general cleanliness and organization of the facility, fee schedules, and any efforts to encourage rights-based education. Research assistants will employ a form to record observational data. Key informant interviews and in-depth interviews with facility-based health providers and CODESA members will be carried out in French or Kiswahili. In-depth interviews with CHWs and group discussion participants will be administered in Kiswahili. Some of the interviews done in Kiswahili may require the assistance of an interpreter, who will be trained by the research assistants. Key informant and in-depth interviews and group discussions will be audio recorded; interviewers will also take handwritten notes of information that will give additional insights into the data. Interviews and group discussions conducted in Kiswahili will first be transcribed in Kiswahili and subsequently translated into French after the data collection is completed, while interviews conducted in French will be transcribed directly. Transcripts will be reviewed by the research assistants and team supervisor, and electronic copies of the transcripts will be kept by D4I. Data will only be collected after the informed consent process has been carried out and if potential study candidates agree to participate in the study. c. Sampling and recruitment Key informants will be selected purposively based on their role in health systems management, expertise in health systems strengthening, and involvement in the USAID IHP project activities. Respondents will include representatives of the MOH at the national (3), provincial (2), and zonal (1 in each of two HZs) level, USAID IHP staff and technical and implementing partners involved in intervention development, implementation, and monitoring and evaluation at headquarters (1-2), central (6-7) and provincial (1) levels, and representatives of donor organizations (3-4). We also plan on interviewing a government official in the inspector’s office at the provincial level. No more than 25 key informants will be interviewed. An appointment to meet with the potential respondents will be made by phone or email, but respondents will be approached to participate in the study in their offices. In-depth interviews will be conducted with facility-based service providers and CHWs including the administrator and a clinician working on the pediatric ward in the reference hospital, the Infirmier Titulaire de l’Aire de Santé or their assistant in the health center, the CODESA president or vice president, a CHW selected by the CODESA respondent with the aim to identify a long-tenured and high performing community worker, and an influential community leader knowledgeable of health services and recommended by the CODESA and CHW respondents. In-depth interviews will be carried out in the two zonal reference hospitals and in each of the four health center areas to assess facility- and community-based health care services and especially child health activities. Health care respondents will be approached in the health facility for recruitment, while the community leader will be contacted in his or her home or place of work. No more than eight facility-based health workers, four CODESA members, for CHWs and four community leaders will be interviewed. 216 The Impact of USAID’s Integrated Health Program in the DRC In each of the four health center areas, we will carry out focus group discussions with child caregivers including mothers and grandmothers. Each group will be comprised of 8-12 participants. Caregivers must have a child under five years of age living in their household and be a permanent resident of the health area. Caregivers will be purposively selected by the relais communautaires working in their village; we will search for experienced caregivers who are willing to share their opinions and experiences related to child health and care seeking in a group setting. Observations will be carried out in each of the two reference hospitals and in the four health centers where the research will be conducted. During observations, a range of information will be collected including the general condition of the facilities, availability of drugs, materials and supplies, and equipment to provide essential child health care, and official fee schedules. We will also carry out observations of the way caregivers and patients are managed upon arrival in the facility and before receiving health care. Direct observations will be made of provider-caregiver interactions during treatment of sick children, with the aim to observe a minimum of four observations in each reference hospital and four interactions in each health center and no more than 38 observations in total. d. Data collection Data from key informant and in-depth interviews and focus group discussions will be audio recorded, translated from local language into French when needed, and transcribed in French. Observational data handwritten on a structured form will be transcribed. Based on reviews of data transcripts, research assistants and the lead researcher will work together to develop a coding system. Coding categories will be derived from the initial research themes and questions, as well as from key concepts that emerged during data collection. Coding of the interview transcripts will be done on ATLAS.ti, a text-organizing software. Content analysis will be used to identify trends of concepts in and across individual codes. Photographs of the health facility environment will be used to complement information concerning facility infrastructures and equipment. The combination of data, environmental and methodological triangulation will allow us to analyze data across different research methods (e.g. key informant and in￾depth interviews) and sites and across and between respondents. Baseline Report 217 10. Ethics and Informed Consent Data collectors will carry out the informed consent process with each potential respondent. The informed consent process includes an explanation of the purpose of the study, the risks and benefits of participating, and the time required by each interview. Also, it explains that the individual has the choice to participate or not participate in the study. The consent process will include assurances that participation is voluntary, that those with power over the participant (work supervisors, physicians, etc.) will not be notified of whether or not the person participated, and that their individual responses will not be shared or linked to them in the research products. All survey respondents will give informed consent before participating. Patients whose medical records are reviewed will not be notified and will not give informed consent. Their identifiers (names, dates of birth) will not be collected. Due to the fact that some participants (for example, the Provincial Health Officers) could be identified by their location alone, all results will be presented in aggregate. Data will be stored on secure servers under password protection, with access only available to research staff. Ethical approval of the evaluation will be obtained from the Institutional Review Board of the Tulane School of Public Health and the Kinshasa School of Public Health prior to data collection (see Appendix E). We are not collecting any biomarkers or performing any medical interventions and we do not anticipate that any questions will trigger negative psychological responses. However, should this happen, the respondent will be referred to the nearest health facility. Respondents will not receive incentives to participate. 218 The Impact of USAID’s Integrated Health Program in the DRC 11. Data Management a. Data storage and security D4I will be responsible for overseeing data management. All data forms and records collected during this research will be held in a secure location at KSPH and/or Tulane University for the duration of the proposed research by Principal Investigator, Dr. David Hotchkiss and co-investigators, Dr. Janna Wisniewski and Dr. Paul-Samson Lusamba-Dikassa. Confidentiality of all respondents will be ensured through the replacement of any personal information with unrelated unique identifiers. Where relevant, names and location information will be separated from the electronic data processed for analysis. The only identifiers used during the analysis will be a unique identification number. All data will be kept under lock and key or password-protected computer, with only key personnel having access. b. Data-sharing procedures Data sharing memorandums of understanding will be developed between D4I and KSPH to facilitate access to data files. De-identified data will be transferred to KSPH via a secure File Transfer Process server and then stored on a secure server managed. In accordance with the USAID Open Data Policy, de￾identified data will be transferred to a data archive such as the Dataverse Network, to facility third-party requests to access the data. Data will only be shared with third parties following written permission from Tulane University, represented by the PI, Dr. Hotchkiss. A copy of the data-sharing agreement between D4I and USAID can be found in Appendix F. c. Knowledge management plan Upon completion of the baseline, midline and endline survey reports, D4I and KSPH will be responsible for hosting a results dissemination workshop to inform all stakeholders of the survey results. Reports will be written in English and in French, summarizing the study results. The workshop will present the findings in a clear and concise manner and invitees will include representatives from USAID, Abt Associates, Implementing Partners, D4I, the MOH and where possible community leaders from the communities selected for the study. In addition, datasets will be publicly released after the conclusion of the study, following data sharing policies of the MOH and USAID. A copy of the Memorandum of Understanding on authorship can found in Appendix G. Baseline Report 219 12. Evaluation Team and Stakeholder Roles This evaluation is being conducted by the D4I project. D4I partner, Tulane University’s School of Public Health and Tropical Medicine, has extensive experience supporting health research and capacity building in the DRC over the past five decades and has established a research office in Kinshasa with a full-time Senior Research Director. The evaluation is led by Dr. David Hotchkiss (the Technical Lead based in New Orleans), Dr. Janna Wisniewski (a co-investigator based in New Orleans), and Dr. Paul-Samson Lusamba-Dikassa (a co-investigator and a faculty member of the Kinshasa School of Public Health based in Kinshasa). They are assisted by a grant management specialist who is responsible for the financial management of the study. Drs. Hotchkiss and Wisniewski will be responsible for overseeing all activities and analysis related to the quantitative component of the study, including submission of the protocol to the Institutional Review Boards of the Tulane and KSPH, the training of supervisors and data collectors responsible for the facility, health zone office, and provincial health office surveys, the compilation and analysis of DHIS2 data, and writing reports based on the study findings. Through sub-contracts with Tulane, KSPH will be responsible for carrying out the D4I facility, health zone office, and provincial health office surveys at baseline, midline, and endline, and participating in the data analysis and report writing. The KSPH team will be led by Drs. Patrick Kayembe and Eric Mufata, faculty members at KSPH. It is also anticipated that a sub-contract will be established with BlueSquare to create a DHIS2 database, assist in carrying out data quality assessment of the DHIS2 data, and automate the indicators needed for the DID-PSM analysis. It should be noted that BlueSquare is a partner on USAID IHP with the responsibility of developing and managing USAID IHP’s Monitoring and Evaluation Platform, which synthesizes all monitoring data, including DHIS2 data, in one place. BlueSquare’s work and deliverables for the D4I evaluation will be in addition to those they are contracted to carry out for USAID IHP. Dr. Lauren Blum, a medical anthropologist and consultant to Tulane, will be responsible for overseeing all activities and analysis related to the qualitative study. Dr. Blum has over 18 years of experience leading qualitative studies. She lived and worked in the DRC for over seven years and speaks French. She will be responsible for the following activities: 1) develop the study protocol including data collection tools and consent forms; 2) submit the protocol to the Institutional Review Boards of Tulane and KSPH; 3) train local qualitative researchers on data collection and management techniques; 4) oversee the pre￾test of the study instruments and revise the instruments based on the pre-test results; 5) ensure the fieldwork and data transcription and translation is carried out according to the protocol; 6) code and analyze the data in conjunction with the study supervisor; 7) write-up the study findings; and 8) participate in the data triangulation and the elaboration of reports based on the study findings. Dr. Paul￾Samson Lusamba-Dikassa will provide technical and managerial support to the study. Drs. Blum and Lusamba-Dikassa will be assisted by Dr. Eric Mafuta, who has extensive experience in qualitative research. As a co-investigator, Dr. Mafuta will supervise data collection and participate in data coding and analysis. The New Orleans-based members of the evaluation team regularly visit Kinshasa to provide oversight of the evaluation activities in coordination with USAID, USAID IHP, the local D4I partner team, and study contractors/subcontractors. D4I reports on the progress of the study to USAID and the Chapel Hill, NC D4I management team. 220 The Impact of USAID’s Integrated Health Program in the DRC 13. Timeline and Deliverables The timeline for the evaluation activities is presented below. Wave three activities are conditional on USAID approving a three-year extension for USAID IHP. Table 8. Timeline of quantitative data collection. Start date End date Activity Wave 1 6/01/19 Study documents finalized and translated 6/14/19 IRB approval (D4I and KSPH) 5/20/19 6/28/19 ODK programming and testing 6/10/19 6/14/19 Supervisor training and pilot testing (Kinshasa) 6/15/19 6/22/19 Supervisor travel from Kinshasa to provinces 6/30/19 7/01/19 Data collector recruitment and preparation (provinces) 7/02/19 7/11/19 Data collector training 7/11/19 8/11/19 Data collection 8/12/19 9/30/19 Data cleaning, analysis, and report writing Wave 2 6/10/22 6/14/22 Supervisor training and pilot testing (Kinshasa) 6/15/22 6/22/22 Supervisor travel from Kinshasa to provinces 6/30/22 7/01/22 Data collector recruitment and preparation (provinces) 7/02/22 7/11/22 Data collector training 7/11/22 8/11/22 Data collection 8/12/22 9/30/22 Data cleaning, analysis, and report writing Wave 3 6/10/25 6/14/25 Supervisor training and pilot testing (Kinshasa) 6/15/25 6/22/25 Supervisor travel from Kinshasa to provinces 6/30/25 7/01/25 Data collector recruitment and preparation (provinces) 7/02/25 7/11/25 Data collector training 7/11/25 8/11/25 Data collection 8/12/25 9/30/25 Data cleaning, analysis, and report writing Baseline Report 221 Table 9. Timeline of qualitative data collection Start date End date Activity Wave 1 3/25/19 Study documents finalized and translated 3/26/19 4/09/19 IRB approval (D4I and KSPH) 4/15/19 5/1/19 Data collection by Lauren Blum (D4I and Lualaba) 5/2/19 5/17/19 Transcription 5/20/19 5/31/19 Data coding, analysis, and report writing 10/07/19 10/11/19 Data collector training 10/14/19 11/01/19 Data collection in provinces (facility and community-level) 11/2/19 12/17/19 Transcription 12/18/19 12/31/19 Data coding, analysis, and report writing Wave 2 10/07/21 10/11/21 Data collector refresher training 10/14/21 11/01/21 Data collection 11/02/21 12/17/21 Transcription 12/18/21 12/31/21 Data coding, analysis, and report writing Wave 3 10/07/24 10/11/24 Data collector refresher training 10/14/24 11/01/24 Data collection 11/02/24 12/17/24 Transcription 12/18/24 12/31/24 Data coding, analysis, and report writing 222 The Impact of USAID’s Integrated Health Program in the DRC Table 10. Schedule of deliverables Date Deliverable Description 8/30/19 Wave 1 topline report Descriptive results of key indicators from facility surveys 9/30/19 Wave 1 preliminary report Descriptive results from facility surveys and qualitative data collection done to date 1/01/20 Wave 1 report Includes qualitative data from October 2019 8/30/22 Wave 2 topline report Changes over time and difference-in-difference for key variables between Waves 1 and 2. 9/30/22 Wave 2 report Changes over time and difference-in-difference between Waves 1 and 2, and qualitative results. 8/30/25 Wave 3 topline report Changes over time and difference-in-difference for key variables between Waves 1, 2 and 3. 9/30/25 Final report Changes over time and difference-in-difference between Waves 1, 2, and 3, and qualitative results. Baseline Report 223 References Adam, T., Hsu, J., de Savigny, D., Lavis, J., Rottingen, J., & Bennett, S. (2012). Evaluating health systems strengthening interventions in low-income and middle-income countries: are we asking the right questions?. Health Policy and Planning, 27(suppl 4), iv9-iv19. doi: 10.1093/heapol/czs086. Bertone, Maria Paola, Grégoire Lurton, Paulin Beya Mutombo. (2016). Investigating the remuneration of health workers in the DR Congo: implications for the health workforce and the health system in a fragile setting, Health Policy and Planning, 31 (9), https://doi.org/10.1093/heapol/czv131/ Meheus, F., & McIntyre, D. (2017). Fiscal space for domestic funding of health and other social services. Health Economics, Policy And Law, 12(02), 159-177. doi: 10.1017/s1744133116000438 Ministère du Plan et Suivi de la Mise en œuvre de la Révolution de la Modernité (MPSMRM), Ministère de la Santé Publique (MSP) et ICF International. (2014). Enquête Démographique et de Santé en République Démocratique du Congo 2013-2014. Rockville, Maryland: MPSMRM, MSP et ICF International. Moszynski, P. (2008). 5.4 million people have died in Democratic Republic of Congo since 1998 because of conflict, report says. BMJ, 336(7638), pp.235.2-235. Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee. (2019). The PCORI Methodology Report. United States Agency for International Development. (2014). Country Development Cooperation Strategy: Democratic Republic of Congo 2014-2019. Washington, DC: United States Agency for International Development. Walker N, Tam Y, Friberg IK. Overview of the lives saved tool (LiST). BMC Public Health. 2013;13 Suppl 3:S1. World Bank. (2017). Congo, Democratic Republic of - Health System Strengthening for Better Maternal and Child Health Results Project: additional financing. Washington, DC: World Bank Group. Retrieved from http://documents.worldbank.org/curated/en/334761491184862554/Congo-Democratic-Republic-of￾Health-System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-additional-financing Baseline Report 225 Appendix 2: Data Collection Instruments IHP Evaluation HEALTH FACILITY SURVEY Part A. Management SECTION 1. ORIENTATION Upon arrival at the facility, do the following: No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1. Record the province 2. Record the health zone 3. Record the name of the health facility 4. Enter your data collector ID number 5. Enter the facility ID number Identify the highest-ranking person. Explain that some questions may have been asked in earlier interviews with USAID IHP representatives and that you appreciate their time and patience. 6. Have you read him/her the consent script? |_| No |_| Yes If no,  7 7. If no, why? 8. Did the respondent agree? |_| No |_| Yes If no,  9 9. If no, what was the reason? |_| No office members present at time of visit |_| Office members absent for a long period of time |_| Deferred |_| Refused |_| Office vacant or not an address |_| Office destroyed |_| Office not found |_| Other (specify) _________ 10. Take a photograph of the front of the facility 226 The Impact of USAID’s Integrated Health Program in the DRC SECTION 2. BASIC FACILITY INFORMATION First, I would like to ask you some general questions about how this facility is organized, and what infrastructure and resources are available. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 11. On what days of the week is this facility typically open and for how many hours on these days? |_|Monday Hours |__|__| |_|Tuesday Hours |__|__| |_|Wednesday Hours |__|__| |_|Thursday Hours |__|__| |_|Friday Hours |__|__| |_|Saturday Hours |__|__| |_|Sunday Hours |__|__| 12. In general, is there a trained health provider assigned to and present at the facility at all times (24 hours a day) for emergencies? Yes, observed Yes, not observed No one on call 24/24 Don't know If no,  13 13. In general, is there a trained health provider available away from the facility but officially on call, at all times, (24 hours a day) for emergencies? Yes, observed Yes, not observed No one on call 24/24 Don't know 14. RECORD OBSERVATION OF THE MAIN MATERIAL OF THE FLOOR NATURAL FLOOR EARTH/SAND DUNG RUDIMENTARY FLOOR WOOD PLANKS PALM/BAMBOO FINISHED FLOOR PARQUET OR POLISHED WOOD VINYL OR ASPHALT STRIPS CERAMIC TILES CEMENT CARPET OTHER _________________________ (SPECIFY) Baseline Report 227 15. RECORD OBSERVATION OF THE MAIN MATERIAL OF THE ROOF NATURAL ROOFLING NO ROOF THATCH/PALM LEAF SOD RUDIMENTARY ROOFING MAT PALM/BAMBOO WOOD PLANKS CARDBOARD FINISHED ROOFING METAL WOOD CALAMINE/CEMENT FIBRE CERAMIC TILES CEMENT OTHER _________________________ (SPECIFY) 16. RECORD OBSERVATION OF THE MAIN MATERIAL OF THE EXTERIOR WALLS NATURAL WALLS NO WALLS BAMBOO/CANE/PALM/TRUNK DIRT RUDIMENTARY WALLS BAMBOO WITH MUD STONE WITH MUD UNCOVERED ADOBE PLYWOOD CARDBOARD REUSED WOOD FINISHED WALLS CEMENT STONE WITH LIME/CEMENT BRICKS CEMENT BLOCKS COVERED ADOBE WOOD PLANKS OTHER _________________________ (SPECIFY) 228 The Impact of USAID’s Integrated Health Program in the DRC SECTION 2. GENERAL STAFFING Now I have some questions about staffing for this facility. Please tell me how many staff members with each qualification are currently assigned to this facility and whether they are male or female staff, and how many positions are unfilled. This includes staff who work here full-time, part-time, or are volunteers. Enter -99 if unknown USE THIS INFORMATION TO COMPLETE COLUMNS a, b and C. IF THE INFORMATION IN UNKNOWN, MARK COLUMN d AND LEAVE COLUMN a, b, and c BLANK. ASK TO SEE THE STAFF ROSTER. FOR EACH QUALIFICATION, RECORD THE NUMBER OF PEOPLE WHO ARE ACTUALLY PRESENT THE DAY OF THE INTERVIEW. USE THIS INFORMATION TO COMPLETE COLUMN e. Qualification Total Staff Number Female Number Male 17. Medical doctor (specialist) 18. Medical doctor (General practitioner) 19. Nurse A0/L2 20. Nurse A1 21. Nurse A2 22. Nurse A3 23. Anesthetist 24. Midwife or trained birth attendant (A1 OR A0/L2) 25. Community health worker (L2 OR G3) 26. Pharmacist 27. Pharmacy dispenser 28. Laboratory technician/technologist (A1) 29. Medical biologist L2 30. Radiology technician 31. Nutritionist 32. Hospital administrator Baseline Report 229 33. Maintenance technologist 34. Other staff (specify) ______________________ 35. Other staff (specify) ______________________ 36. Other staff (specify) ______________________ 37. Other staff (specify) ______________________ 38. What is the highest qualification of the facility head? Physician Nurse Other clinical Other non-clinical Don't know 39. Is the facility head male or female? MALE FEMALE DON'T KNOW 40. For how many years has the facility head held that position in this facility? |__|__| years 230 The Impact of USAID’s Integrated Health Program in the DRC SECTION 3. MANAGEMENT AND SUPERVISION 41. LEVEL FROM WHICH THE SUPERVISOR CAME In 2018, how many times did a supervisor from [LEVEL] visit this facility for the purpose of management or supervision? In which month and year did a supervisor from [LEVEL] last visit? What topics were discussed at the last visit? [SELECT ALL THAT APPLY] After the last visit, did the supervisor send you a report based on his/her findings? 42. Another health center/post in your health zone 0 1 2 3 4 5 6 7 8 9 10 11 12 or more Not applicable (skip to 43) Don't know Month |_|_| Year |_|_|_|_| Enter 9999 if unknown Quality of care Management Record keeping Data reporting Data use Other ___________ No, no feedback Yes, written in the supervision book Yes, written report received Don't know 43. Hospital in your health zone 0 1 2 3 4 5 6 7 8 9 10 11 12 or more Month |_|_| Year |_|_|_|_| Enter 9999 if unknown Quality of care Management Record keeping Data reporting Data use Other ___________ No, no feedback Yes, written in the supervision book Yes, written report received Don't know Baseline Report 231 Not applicable (skip to 44) Don't know 44. Health Zone Officer 0 1 2 3 4 5 6 7 8 9 10 11 12 or more Not applicable (skip to 45) Don't know Month |_|_| Year |_|_|_|_| Enter 9999 if unknown Quality of care Management Record keeping Data reporting Data use Other ___________ No, no feedback Yes, written in the supervision book Yes, written report received Don't know 45. Agent/Officer of the DPS 0 1 2 3 4 5 6 7 8 9 10 11 12 or more Not applicable (skip to 46) Don't know Month |_|_| Year |_|_|_|_| Enter 9999 if unknown Quality of care Management Record keeping Data reporting Data use Other ___________ No, no feedback Yes, written in the supervision book Yes, written report received Don't know 46. Provincial Health Inspector 0 1 Month |_|_| Quality of care Management No, no feedback 232 The Impact of USAID’s Integrated Health Program in the DRC 2 3 4 5 6 7 8 9 10 11 12 or more Not applicable (skip to 47) Don't know Year |_|_|_|_| Enter 9999 if unknown Record keeping Data reporting Data use Other ___________ Yes, written in the supervision book Yes, written report received Don't know 47. Other (specify) 0 1 2 3 4 5 6 7 8 9 10 11 12 or more Not applicable (skip to 48) Don't know Month |_|_| Year |_|_|_|_| Enter 9999 if unknown Quality of care Management Record keeping Data reporting Data use Other ___________ No, no feedback Yes, written in the supervision book Yes, written report received Don't know Baseline Report 233 In this part of the questionnaire I would like to ask you some questions regarding how work is organized and how decisions are made in this health zone office. All answers are confidential. I am now going to read you a series of statements about decision-making and authority in this facility. Please tell me how true each statement is for you: completely true, somewhat true, or not at all true. RESPONSE CODE RECORD RESPONSE COMPLETELY TRUE 1 SOMEWHAT TRUE 2 NOT AT ALL TRUE 3 NOT APPLICABLE 4 DON’T KNOW -98 48. I am able to allocate my facility budget according to how it is needed. There is enough flexibility in my budget. 49. I am able to assign tasks and activities to staff as needed to achieve the outcomes I want in this facility. There is enough flexibility to use staff to address needs. 50. The BCZ/ECZ supports my decisions and actions for doing a better job in my office. 51. I have choice over who I allocate for what tasks. 52. I have choice over what services are provided in this facility. 53. I have enough authority to obtain the resources I need (drugs, supplies, funding) to meet the needs of my facility. 54. The policies and procedures for doing things are clear to me. 55. The policies and procedures for doing things are useful tools for the challenges I face in providing services and reporting on activities. 56. The BCZ/ECZ provides adequate feedback to me about my job and the performance of my facility. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 57. Who is responsible for the management of the health facility? Government Private not-for-profit organization Faith-based organization Private for-profit organization Other (specify) ___________________ Don't know 58. In 2018, how often did meetings to discuss the facility managerial and administrative matters take place? Monthly or more often Quarterly Twice a year Annually Irregularly Never Don't know 234 The Impact of USAID’s Integrated Health Program in the DRC 59. In 2018, how often were routine meetings held with both facility staff and community members? Monthly or more often Quarterly Twice a year Annually Irregularly Never Don't know 60. Is there a facility community advisory committee (CODESA) in this health area? YES NO DON’T KNOW If yes,  61; otherwise,  76 61. In 2018, how often did the head of this facility meet with the CODESA? Monthly or more often Quarterly Twice a year Annually Irregularly Never Don't know CODESAs have many different functions throughout the country. We are interested in what the CODESA does here. In the last 90 days, has a member of the CODESA, other than the IT, done the following at least one 62. Been present when medications arrived at the facility YES NO N/A DK 63. Ensured the cold chain of medicines was preserved YES NO N/A DK 64. Assisted in taking inventory of medications YES NO N/A DK 65. Completed the inventory analysis form and report YES NO N/A DK 66. Completed the medication acceptance report YES NO N/A DK 67. Discussed the fee schedule with the health facility staff YES NO N/A DK 68. Examined the financial books at the health facility to see if they match the financial report YES NO N/A DK 69. Assisted in developing the plan of monthly expenses YES NO N/A DK 70. Examined the facility cash box YES NO N/A DK 71. Assisted in developing the monthly expenses report YES NO N/A DK 72. Took an inventory of equipment and completed the inventory report YES NO N/A DK 73. Assisted in developing health messaging for the community YES NO N/A DK 74. Assisted in calculating health facility indicators YES NO N/A DK 75. Presented/interpreted health facility indicators to the community YES NO N/A DK Baseline Report 235 76. Does this facility have any system for soliciting patients’ opinions about the health facility or its services? IF YES, CIRCLE ALL METHODS THAT ARE USED FOR ELICITING PATIENTS’ OPINIONS. PROBE FOR ALL METHODS USED. YES NO DON’T KNOW If yes,  77; otherwise,  78 77. If yes, what methods? Suggestion box Interview/meetings with patients Patient surveys Information from CODESAs/Relais Meetings with community leadres Community scorecard Community participation in budgeting Community participation in management meetings Other (specify) ________________________ 78. Does this facility routinely carry out quality assurance activities? By this I mean some formal review system or comparison of work or systems to a standard? YES NO DON’T KNOW 79. Are you aware of a program called the community scorecard? YES NO If yes,  80; otherwise,  96 80. Has your facility participated in a community scorecard meeting within the past twelve months? YES NO DON’T KNOW If yes,  81; otherwise,  96 Has participation in the community scorecard program resulted in efforts to improve any of the following areas, even if actual improvements have not been made: 81. Availability of medicines and essential supplies to the health facility YES NO NA DK 82. Materials/equipment available in health facility YES NO NA DK 83. Physical state of buildings YES NO NA DK 84. Cleanliness of buildings and surroundings YES NO NA DK 85. Accessibility of the health facility (roads, bridges, etc.) YES NO NA DK 86. Patients’ opportunity to be treated out of sight of other people (privacy) YES NO NA DK 87. Technical competence of health workers YES NO NA DK 88. Presence and punctuality of health workers YES NO NA DK 89. Staffing levels in health facilities YES NO NA DK 236 The Impact of USAID’s Integrated Health Program in the DRC SECTION 4. HEALTH INFORMATION SYSTEM I would now like to ask you some questions regarding the SNIS and health information or data that the facility collects and reports. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 96. Are you currently using the harmonized reporting tool [SHOW THE RESPONDENT A PHOTO OF THE CURRENT TOOL] YES NO DON’T KNOW 97. In how many of the last six months has this facility submitted SNIS reports to the BCZS? 0 1 2 3 4 5 6 Don’t know If 0 or don’t know,  99 98. How does this facility submit the SNIS reports to the BCZS? [CIRCLE ALL MENTIONED] Hand-deliver to BCZ BCZ picks up Enter directly into the DHIS2 Phone call Text message Email Other (specify) ______________________ Don't know 99. If this facility was unable to submit SNIS reports to the BCZ every month in the last six months, what were the reasons? [CIRCLE ALL MENTIONED] Did not have the correct form No supervision visit Lack of transportation Lack of internet Lack of phones Lack of electricity No time to complete reports No staff to complete reports Reference 97 and ask if between 1 and 5; otherwise,  100. 90. Reception and attitudes of health workers toward patients YES NO NA DK 91. Discretion and confidentiality of health workers YES NO NA DK 92. Respect and confidence of the population in the health facility YES NO NA DK 93. Affordability of user fees YES NO NA DK 94. Timeliness of payment of health facility employees YES NO NA DK 95. Health facility employees’ salary amounts YES NO NA DK Baseline Report 237 Not aware of submission deadlines Other (specify) ________________________ Don't know 100. In how many of the last six months has this facility submitted SNIS reports to the BCZS on time? 0 1 2 3 4 5 6 Don’t know If between 1 and 5,  101; otherwise,  102 101. If this facility was unable to submit SNIS reports to the BCZ on time every month in the last six months, what were the reasons? [CIRCLE ALL MENTIONED] Did not have the correct form No supervision visit Lack of transportation Lack of internet Lack of phones Lack of electricity No time to complete reports No staff to complete reports Not aware of submission deadlines Other (specify) ________________________ Don't know 102. On the monthly SNIS report, are other facilities’ statistics included? For example, health posts, private facilities, etc.? YES NO DON’T KNOW If no or don’t know,  105. 103. How many other facilities’ statistics are included? |___|___| 104. What other types of facilities are included on the monthly SNIS report? [CIRCLE ALL MENTIONED] Hospital (any type) Health center Health post Other (specify) _______________________ Don't know 105. Who do you contact if you need more SNIS forms? Hospital BCZ Other (specify) _______________________ Don't know 106. In the last month, how did this facility submit its weekly Maladie à Potentiel Epidémique (MAPEPI) to the BCZ? [SELECT ALL MENTIONED] Hand-deliver to DPS DPS picks up Phone call Text message Email from office Email-from cyber café or other off site location Electronically via SNIS/DHIS2 Not applicable Other (specify) 238 The Impact of USAID’s Integrated Health Program in the DRC Don't know 107. To the best of your knowledge, the last time you encountered a MAPEPI case, how much time passed between when you were made aware of the MAPEPI event and when you were able to report to the BCZ? Immediately Within 24 hours Next day More than 2 days Not applicable Don't know If N/A,  109 108. What are some of the reasons why an immediate or weekly MAPEPI report may not be submitted on time? [SELECT ALL MENTIONED] Did not have the correct form No supervision visit Lack of transportation Lack of internet Lack of phones Lack of electricity No time to complete reports No staff to complete reports Not aware of submission deadlines Other (specify) ___________________________ Don't know 109. In addition to the SNIS and MAPEPI reports, which types of reports does this facility submit on a regular basis? [SELECT ALL MENTIONED] Other programs (EPI, HIV) IHP-specific reports Other donor reports Financial reports Other (specify) ___________________________ Don't know 110. During the last 90 days, did the facility receive any feedback report from BCZ office on their performance? Yes, verbal Yes, written and viewed Yes, not observed None Don't know If no or don’t know,  116. Was any of the following feedback included in the report? 111. Verify or correct possibly inaccurate data YES NO DK 112. Complete any data that was missing from the original submission YES NO DK 113. Submit the report by the specified deadline YES NO DK 114. Positive feedback YES NO DK 115. Other feedback YES NO DK IF YES, SPECIFY ______________________________ 116. Does this facility have access to a procedure YES NO DON’T KNOW If no or don’t know,  118. Baseline Report 239 manual for data collection? 117. Is the procedure manual for data collection easy or difficult to understand? EASY DIFFICULT DON’T KNOW Does the facility display the following data? Please indicate types of data displayed, whether the data is posted in a public area, and whether the data have been updated for the last reporting period. [RECORD OBSERVATION] Indicator Yes/No Type of Display Area posted Most recent month of data displayed 118. Demographic information Yes No ( 120) Don’t know ( 120) Graph/chart Map Other PUBLIC STAFF ONLY MONTH YEAR 119.Facility utilization/ volume of service delivery Yes No ( 121) Don’t know ( 121) Graph/chart Map Other PUBLIC STAFF ONLY MONTH YEAR 120.Disease surveillance Yes No ( 122) Don’t know ( 122) Graph/chart Map Other PUBLIC STAFF ONLY MONTH YEAR No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 121. Does the facility have a map of the catchment area and if so, what type of map is it? TAKE A PHOTO OF THE MAP. YES, COMPUTER-GENERATED YES, HAND-DRAWN NO MAP DON’T KNOW Now I would like to ask you some questions related to meetings and decisions based on the SNIS data. 122. How frequently does the facility have routine meetings in which the SNIS or facility data is discussed? This could be a separate meeting or in the routine managerial or administrative meetings. Monthly or more often Every 2-3 months Every 4-6 months Less than every 6 months or irregularly Never Don't know If never,  126 123. Is an official record of management meetings maintained? YES NO DON’T KNOW 124. Has the SNIS or facility data been used to make decisions? YES NO DON’T KNOW If no or don’t know,  126 240 The Impact of USAID’s Integrated Health Program in the DRC 125. Has any follow-up action taken place regarding the decisions made during the previous meetings? YES NO DON’T KNOW 126. Do you use a Performance Dashboard tool for management tasks? [SHOW A PHOTO OF THE TOOL] YES NO DON’T KNOW If no or don’t know,  129. 127. What is the month of the last data entry on this dashboard? [ASK THE RESPONDENT TO ACCESS THE SYSTEM AND RECORD OBSERVATION] January February March April May June July August September October November December Not accessible Don't know 128. What is the year of the last data entry on this dashboard? 2019 2018 before 2018 Don't know 129. Record the GPS coordinates to six decimal places Baseline Report 241 IHP Evaluation HEALTH FACILITY SURVEY Part B. Services SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1. Record the province. 2. Record the health zone. 3. Record the name of the health facility or enter the facility ID number. 4. Enter your data collector ID number. 242 The Impact of USAID’s Integrated Health Program in the DRC SECTION 2. SERVICES RESPONSE SKIP/ INSTRUCTIONS Malnutrition 5. Does this facility provide growth monitoring (weighing and measurement) of children under five? Yes No 6. Does this facility provide nutritional rehabilitation for malnourished children? Yes No Antenatal care 7. Does this facility provide antenatal care? Yes No If no,  10. 8. What types of screenings are typically done for pregnant women? (Check all that apply.) Anemia Asymptomatic bacteriuria Gestational diabetes Syphilis HIV Proteinuria High blood pressure Malnutrition Tuberculosis Malaria Intimate partner violence Other (specify) 9. What services are typically administered to pregnant women? (Check all that apply.) Tetanus vaccine Iron supplements Folate supplements IpT for malaria Anthelminthics/Deworming treatment Counseling on family planning Nutrition counseling Other 10. Does this facility provide delivery services? Yes No If no,  12. 11. How long does a patient typically stay at the facility after an uncomplicated normal delivery? Hours: ____________ Days: _____________ 12. Does this facility provide postpartum care? Yes No If no,  14 Baseline Report 243 13. Which services are provided as part of postpartum care? (Check all that apply.) Maternal examination Breastfeeding counseling Newborn examination Growth monitoring and promotion Vaccination counseling Family planning counseling PMTCT counseling Dietary counseling for breastfeeding mother Counseling on mother and child hygiene Other Tuberculosis 14. Does this facility offer tuberculosis screening? Yes No If no,  15 15. What types of tuberculosis screening are offered? (check all that apply) Zielh Neelssen Gene Xpert Skin test Chest X-ray Culture Loweinstein 16. Does this facility offer tuberculosis treatment? Yes No If no,  18 17. What types of treatment are administered? Directly observed Not directly observed Malaria 18. Does this facility offer malaria screening? Yes No If no,  20 19. What types of malaria screening are offered? (check all that apply) Rapid test Laboratory-confirmed 20. Does this facility offer malaria treatment? Yes No 21. Does this facility distribute insecticide-treated nets to patients? Yes No If no,  24 22. Are nets provided free of charge, sold to patients, or both? Free to all patients Free to some patients All patients must pay If “all patients must pay,”  24 23. Who is eligible for a free insecticide-treated net? All patients Pregnant/postpartum women Children under five Patients with malaria Other (specify) ____________ Family planning 24. Does this facility offer family planning counseling? Yes No 25. Does this facility offer any family planning methods? Yes No If no,  42 244 The Impact of USAID’s Integrated Health Program in the DRC 26. Does this facility perform male sterilization? Yes No If no,  28 27. How may providers have been trained to perform male sterilization? |__|__|__| 28. Does this facility perform female sterilization? Yes No If no,  30 29. How may providers have been trained to perform female sterilization? |__|__|__| 30. Does this facility insert IUD’s? Yes No If no,  32 31. How may providers have been trained to insert IUD’s? |__|__|__| 32. Does this facility remove IUD’s? Yes No If no,  34 33. How many providers have been trained to remove IUD’s? |__|__|__| 34. Does this facility insert Norplant, Jadelle, or Sino-Implant II? Yes No If no,  36 35. How many providers have been trained to insert Norplant, Jadelle, or Sino-Implant II? |__|__|__| 36. Does this facility remove Norplant, Jadelle, or Sino-Implant II? Yes No If no,  38 37. How many providers have been trained to remove Norplant, Jadelle, or Sino-Implant II? |__|__|__| 38. Does this facility insert Implanon? Yes No If no,  40 39. How many providers have been trained to insert Implanon? |__|__|__| 40. Does this facility remove Implanon? Yes No If no,  42 41. How many providers have been trained to remove Implanon? |__|__|__| 42. How many providers have been trained in adolescent health / family planning for adolescents? |__|__|__| 43. Does this facility have resources geared to address adolescent concerns and counsel them about sexual and reproductive health including family planning? Yes No Don’t know Vaccination 44. Does this facility offer immunization services? Yes No Don’t know If no, or don’t know,  50 Does this facility provide any of the following immunization services in the facility only, as outreach at fixed post only, or both? Baseline Report 245 45. Birth doses (e.g. hepB0, BCG, OPV0, …) Both in the facility and as outreach In the facility only Outreach only Service not offered Don't know 46. Infant vaccines (under 1 year) Both in the facility and as outreach In the facility only Outreach only Service not offered Don't know 47. Adolescent/adult vaccines (e.g. HPV, tetanus, flu) Both in the facility and as outreach In the facility only Outreach only Service not offered Don't know 48. How often does this facility offer routine full child immunization services at the facility? Daily 2-4 times a week Weekly Monthly Quarterly Annually Other (Specify) _____________ 49. How often does this facility offer routine full child immunization services as outreach? Daily 2-4 times a week Weekly Monthly Quarterly Annually Other (Specify) _____________ 50. Does this facility provide counseling for victims of sexual and gender-based violence? Yes No 51. Does this facility provide care for physical injuries for victims of sexual and gender-based violence? Yes No 246 The Impact of USAID’s Integrated Health Program in the DRC IHP Evaluation HEALTH FACILITY SURVEY Part C. Finance Section 2. USER FEES I would like to ask you a few questions about the payment of health care services by the patients. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 5. Does this facility have a fee schedule? YES NO DON’T KNOW If no or don’t know,  7 6. Is there a fee schedule posted where patients can see it? IF YES, ASK TO SEE THE GUIDELINES AND TAKE A PHOTO. 7. Have you received guidelines on the payment that the patients have to make for the health care services they need/receive? YES NO DON’T KNOW If no or don’t know,  9 8. To the best of your knowledge, where do the user fee guidelines come from? NATIONAL MOH DPS HOSPITAL BCZ IHP OTHER SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1. Record the province 2. Record the health zone 3. Record the name of the health facility or enter the facility ID number 4. Enter your data collector ID number Baseline Report 247 _____________________ (SPECIFY) DON’T KNOW 9. Does this facility have different fees for patients who are considered indigent? YES NO DON’T KNOW 10. Have you received guidelines on the EXEMPTION of payment for indigent patients? YES NO DON’T KNOW If no or don’t know,  12 11. To the best of your knowledge, where do the exemption guidelines come from? NATIONAL MOH. DPS HOSPITAL BCZ IHP OTHER (SPECIFY) _____________________ DON’T KNOW 12. Who are considered indigent? Meaning, what are the criteria for the user fee exemption? CIRCLE ALL THAT ARE MENTIONED ELDERLY ORPHANED Reference 9 and ask if yes. WIDOW WITHOUT A SOURCE OF INCOME PHYSICALLY HANDICAPED WITHOUT A SOURCE OF INCOME REFUGEE/INTERNALLY DISPLACED PERSON OTHER (SPECIFY) _____________________ DON’T KNOW 13. Do you have a list of households or individuals in your community who are considered indigent? YES NO DON’T KNOW Reference 9 and ask if yes. If no or don’t know,  17 14. Who participated in the process of identifying the individuals on this list? CIRCLE ALL THAT ARE MENTIONED THE RESPONDENT OTHER HEALTH CENTRE STAFF CODESA MEMBERS/RELAIS OTHER COMMUNITY LEADERS BCZ OTHER (SPECIFY) _____________________ DON’T KNOW 15. Was this list approved by the CODESA? YES NO DON’T KNOW 248 The Impact of USAID’s Integrated Health Program in the DRC 16. How often is this list updated? EVERY 1-5 MONTHS EVERY 6 MONTHS EVERY 7-11 MONTHS ONCE EVERY YEAR OR LESS NEVER DON’T KNOW What is the standard fee at this facility for a non-indigent patient to receive? ENTER 0 IF THE SERVICE IS FREE and -99 if the service is not offered. ALL AMOUNTS ENTERED IN CF. 17. their first antenatal care visit? ___CF 18. normal (i.e. vaginal) delivery? ____CF 19. Does this facility offer tuberculosis screening or treatment? YES NO If no,  22 20. tuberculosis screening? Sputum test : ____CF Blood test: ____CF Chest x-ray: ____CF 21. tuberculosis full course of treatment? Directly observed: ____CF Not directly observed: ____CF 22. malaria screening? Rapid test: ____CF Laboratory-confirmed: ____CF 23. Does this facility offer family planning services? YES NO If no,  32 24. male sterilization? ____CF 25. female sterilization? ____CF 26. IUD insertion (cost of the method and insertion)? ____CF 27. IUD removal? ____CF 28. Norplant, Jadelle, or Sino-Implant II insertion (cost of the method and insertion)? ____CF 29. Norplant, Jadelle, or Sino-Implant II removal? ____CF 30. Implanon insertion (cost of the method and insertion)? ____CF Baseline Report 249 31. Implanon removal? ____CF 32. measles vaccination? ____CF 33. an insecticide-treated net? ____CF 34. Does this facility always require payment before treatment in emergency cases? YES NO DON’T KNOW 35. Does this facility always require payment before labor and delivery? YES NO DON’T KNOW 36. How does the facility handle cases in which a patient cannot pay for services? CHECK ALL THAT APPLY. No services are given The patient can pay in-kind The patient can give a guarantee The patient is treated for free/reduced cost The patient is not discharged until they can pay The patient is refused services in the future Nothing; no recourse Other (specify) Don’t know 250 The Impact of USAID’s Integrated Health Program in the DRC SECTION 3. SOURCES AND USES OF FUNDING No. QUESTION RESPONSE SKIP/ INSTRUCTIONS What is the total amount of operational funds (i.e. funds not passed on to another organization) received from the following sources in the last calendar year (2018)? 37. Ministry of health/Provincial health office/Health zone office (not including performance-based financing payments) |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 38. Performance-based financing |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 39. USAID/IHP |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 40. Other NGO’s or FBO’s: Cordaid, Memisa, Foundation Damien, Save the Children, Rescue, IRC, Caritas, Sanru, Chemonics, MCSP |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 41. User fees |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 42. Contributions from patients |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 43. Community financing (cooperatives/mutual) |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 44. Health insurance |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD 45. Other (specify)________________ |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD What percentage of operational funds was spent on the following during the last calendar year (2018)? 46. Savings |__|__|__| % 47. Building/grounds improvements |__|__|__| % 48. Equipment (vehicles, computers, etc.) |__|__|__| % 49. Utilities and communication (electricity, water, phone credit, internet, etc.) |__|__|__| % 50. Medical supplies |__|__|__| % 51. Drugs |__|__|__| % Baseline Report 251 52. Transport |__|__|__| % 53. Salaries and primes |__|__|__| % 54. Training |__|__|__| % 55. Other (specify)______________________ |__|__|__| % SECTION 4. PERFORMANCE BASED FINANCING No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 56. Does this facility currently have a performance-based financing contract (i.e. is it eligible to receive financial incentives based on achieving pre-determined targets)? YES NO DON’T KNOW If no or don’t know,  63 57. With what organization do you have a performance-based￾financing contract? CHECK ALL THAT APPLY GOVERNMENT/MOH WORLD BANK USAID/IHP OTHER_______________ DON’T KNOW 58. According to your contract, what is the maximum number of payments that you could receive in a year? 59. In 2018, did you receive a performance-based financing payment for meeting specific targets? 60. How long ago did you receive your last performance-based incentive? 0-3 months ago 4-6 months ago 7-12 months ago More than 1 year ago Never If never,  63 61. Which target(s) did you meet? CHECK ALL THAT APPLY. Maternal health Newborn health Sexual and reproductive health Family planning Adolescent health Child health Quality of care Other (specify)___________ Don't know 252 The Impact of USAID’s Integrated Health Program in the DRC 62. In total, how much did you receive for the most recent payment? |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD SECTION 5. COMMUNITY FUNDING INITIATIVES No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 63. Are there any health mutuals operating in your health area? YES NO DON’T KNOW If no or don’t know, skip to next module. 64. Does this facility have a list of members of this health mutual? YES NO DON’T KNOW 65. Do members of the health mutual make contributions of money or in-kind payments to this facility? YES, MONEY ONLY YES, IN-KIND ONLY YES, MONEY & IN-KIND NO DON’T KNOW If “Yes, money only” or “Yes, money & in-king”,  66; otherwise,  67 66. Are the contributions of the health mutual recorded in a registry? YES NO DON’T KNOW If yes,  67; otherwise, 68 67. Can I see the health mutual contribution registry? YES, OBSERVED NO, NOT OBSERVED 68. RECORD THE SUM OF ALL CONTRIBUTIONS FOR THE LAST 6 COMPLETED MONTHS FROM THE REGISTERY. IF NO REGISTER EXISTS OR THE REGISTRY WAS NOT OBSERVED, ASK THE RESPONDANT TO ESTIMATE THE TOTAL SUM OF ALL CONTRIBUTIONS FOR THE LAST 6 COMPLETED MONTHS. |_|_|_|_|_|_|_|_|_| CF |_|_|_|_|_|_|_|_|_| USD Reference 65 and ask if yes. 69. Do members of the health mutual receive any fee reduction for services performed at this facility? YES NO DON’T KNOW If no or don’t know, skip to next module. Baseline Report 253 70. How is the amount of the fee reduction for health mutual members determined? FIXED AMOUNT PERCENTAGE OF CHARGES VARIES BY SERVICE OTHER DON’T KNOW 71. What services are covered by this fee reduction? CIRCLE ALL THAT ARE MENTIONED General outpatient Immunization for children Antenatal care Normal delivery Caesarean section delivery Postnatal care Family planning Tuberculosis treatment STI treatment Community and outreach services General inpatient medical services General inpatient surgical services All services Other _________________ Don’t Know 254 The Impact of USAID’s Integrated Health Program in the DRC IHP Evaluation HEALTH FACILITY SURVEY Part D. Basic Infrastructure and Laboratory SECTION 2. BASIC INFRASTRUCTURE Now I would like to ask you some questions related to the basic infrastructure of the facility. If the answer to the question is yes, please show me the room/equipment, and where applicable, verify the functionally. No. Questions and filters YES NO SKIP 5. Is there a reception/registration rooms or space? RECORD OBSERVATION 01 00 6. Is there a consultation/examination room for outpatients? RECORD OBSERVATION 01 00 If no,  11 7. Does the examination room have an examination bed? RECORD OBSERVATION 01 00 8. Does the examination room have an instrument trolley? RECORD OBSERVATION 01 00 9. Does the examination room have a hand-washing facility? RECORD OBSERVATION 01 00 10. Does the examination room have a spot light? RECORD OBSERVATION 01 00 11. Is there a separate treatment room? RECORD OBSERVATION 01 00 12. Is there a separate room for observation? RECORD OBSERVATION 01 00 13. Is there an emergency room? RECORD OBSERVATION 01 00 SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1. Record the province 2. Record the health zone 3. Record the name of the health facility or enter the facility ID number 4. Enter your data collector ID number Baseline Report 255 14. Is there an operation theater? RECORD OBSERVATION 01 00 15. Is there a separate medicine dispensing room? RECORD OBSERVATION 01 00 16. Does this facility have an isolation ward? 17. Does this facility have a private delivery suite? 01 00 18. Does this facility have a maternity ward? RECORD OBSERVATION 01 00 If no,  20 19. How many beds are in the maternity ward? RECORD OBSERVATION IF NONE, WRITE “000” IN THE BOXES. |__|__|__| 20. Does the facility have a maternity-waiting house? RECORD OBSERVATION 01 00 21. Does the facility have cell phone reception? RECORD OBSERVATION 01 00 22. Does the facility have a satellite telephone? 01 00 23. Does the facility have a radio? RECORD OBSERVATION 01 00 24. Does the facility have internet? RECORD OBSERVATION 01 00 25. Does the facility have electricity? RECORD OBSERVATION 01 00 If no,  30 26. Is electricity functioning now? (CHECK TO SEE IF ELECTRICITY CAN BE TURNED ON.) RECORD OBSERVATION 01 00 27. Are there power cuts (excluding electricity supplied by a generator backup) during the hours when the facility is open? 01 00 If no,  30 28. What is the average duration of power cuts? TIME IN HOURS |__|__| IF LESS THAN ONE HOUR, WRITE 00. 29. Has there been a break in power for more than two hours during the past seven days? 30. Does this facility have a generator for electricity? This may be a back-up or stand-by generator. RECORD OBSERVATION 01 00 If no,  32 31. Is the generator functional? RECORD OBSERVATION 01 00 32. Is there a functional solar panel? RECORD OBSERVATION 01 00 If no,  34 33. Is there a functional battery for solar panel? RECORD OBSERVATION 01 00 256 The Impact of USAID’s Integrated Health Program in the DRC 34. What is the main source of water at this facility? Piped water from central supply Protected spring Supply truck Rainwater collected and stored in cistern Rainwater collected and not stored in cistern Standpipe Local river/lake Other (specify) Don't know 35. Is there potable water, that is to say safe drinking water, in the structure? YES………………..…………………… NO………………….…………………. DON’T KNOW………………..…. 36. Do you have a water filtering system and is it functioning? YES FUNCTIONING…………… YES, NOT FUNCTIONING…… NO………………….…………………. DON’T KNOW………………..…. If no or don’t know,  38 37. What type (brand) is the filter? RECORD OBSERVATION. LIFESTRAW…..…………………… OTHER………….…………………. DON’T KNOW………………..…. 38. Is there a sanitary toilet or latrine that is available for clients to use, such as those with sewer connections, septic system connections, pour-flush latrines, ventilated improved pit latrines and pit latrines with a slab or covered pit? RECORD OBSERVATION YES………………..…………………… NO………………….…………………. DON’T KNOW………………..…. If no or don’t know,  41 39. Is there a separate sanitary toilet/latrine for the use of women patients? RECORD OBSERVATION YES………………..…………………… NO………………….…………………. DON’T KNOW………………..…. 40. Is there a separate sanitary toilet/latrine for the use of staff? RECORD OBSERVATION YES………………..…………………… NO………………….…………………. DON’T KNOW………………..…. 41. Is there a shower that is available for patients to use? RECORD OBSERVATION YES………………..…………………… NO………………….…………………. DON’T KNOW………………..…. If no or don’t know,  44 42. Is there a separate shower for use by women patients? RECORD OBSERVATION YES………………..…………………… NO………………….…………………. DON’T KNOW………………..…. 43. Is there a separate shower for use by staff? RECORD OBSERVATION YES………………..…………………… NO………………….…………………. DON’T KNOW………………..…. Baseline Report 257 SECTION 3. LABORATORY 44. Does this facility have a laboratory? YES……………………. NO……………………. 01 00 If no,  61 45. Does the facility perform diagnostic tests? YES……………………. NO…………………….. 01 00 If no,  61 I am going to read you a list of laboratory tests. Please indicate whether the facility is able to do this test today, was able to do this test in the past 60 days but not today, or if the facility cannot do this test. Able to do this test today Able to do in past 60 days but not today Cannot do this test 46. Hemoglobin testing 01 02 00 47. White blood cell count 01 02 00 48. Leukocyte formula 01 02 00 49. Sedimentation rate 01 02 00 50. Malaria microscopy 01 02 00 51. Blood glucose 01 02 00 52. HIV testing 01 02 00 53. Ziehl stains 01 02 00 54. Gram stains 01 02 00 55. Blood type and cross match 01 02 00 56. Syphilis testing 01 02 00 57. Urine analysis 01 02 00 58. Stool direct microscopic examination 01 02 00 59. Pregnancy testing 01 02 00 60. Hepatitis testing 01 02 00 61. Does the facility use external diagnostic services or an external laboratory? YES……………………. NO…………………….. DON’T KNOW……. 01 00 98 Now I would like to ask you some questions about the laboratory’s equipment. I will read you a list of equipment and for each piece please tell me how many the facility has and the number functional. For selected items I will ask to see the equipment to verify. IF NONE, WRITE “00” IN THE BOXES. SKIP THIS SECTION IF THERE IS NO LABORATORY AT THIS FACILITY How many? a Number functional? B 62. Microscope |___|___| |___|___| 258 The Impact of USAID’s Integrated Health Program in the DRC 63. Centrifuge |___|___| |___|___| 64. Photometer |___|___| |___|___| SECTION 4. MEDICAL WASTE TREATMENT No. QUESTION RESPONSE 65. What is this facility’s disposal practice for sharps such as needles, glass, surgical instruments, etc.? (CIRCLE THE CODES FOR ALL PRACTICES THAT ARE MENTIONED IN RESPONSE TO THE QUESTION AND INDICATE WHETHER THIS WAS OBSERVED OR NOT) Incineration Burn and bury Bury but do not burn Burn but do not bury Safe box Covered pit (could be latrine) Placenta pit Open pit Store, collect, and move offsite Throw out in open Other (specify) __________________________ Don't know 66. What is this facility’s disposal practice for biomedical wastes such as placenta, human body parts, laboratory waste, etc.? (CIRCLE THE CODES FOR ALL PRACTICES THAT ARE MENTIONED/OBSERVED IN RESPONSE TO THEQUESTION) Incineration Burn and bury Bury but do not burn Burn but do not bury Safe box Covered pit (could be latrine) Placenta pit Open pit Store, collect, and move offsite Throw out in open Other (specify) __________________________ Don't know 67. Does the facility have a picture or written instructions on the wall for waste management/universal precautions? YES…………………………………………………………………………………………. NO…………………………………………………………………………………………. DON’T KNOW…………………………………………………………………………. 01 02 96 Baseline Report 259 IHP Evaluation HEALTH FACILITY SURVEY Part E. Drugs and Supplies SECTION 2. EQUIPMENT Now I would like to ask you some questions about the facilities equipment. I will read you a list of equipment and for each piece please tell me how many the facility has that are functional/usable today. IF NONE, WRITE “00” IN THE BOXES. Large Medical Supplies Number functional a 5. X-ray |___|___| 6. Ultrasound |___|___| 7. Autoclave |___|___| 8. Steam sterilizer / autoclave |___|___| 9. Anesthetic device |___|___| Equipment for Newborns 10. Table to care for newborn |___|___| 11. Scale for baby weighing |___|___| Other Materials 12. Weighing scale for adults |___|___| 13. Height measure for children |___|___| SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1. Record the province 2. Record the health zone 3. Record the name of the health facility or enter the facility ID number 4. Enter your data collector ID number 260 The Impact of USAID’s Integrated Health Program in the DRC 14. Height measure for adults |___|___| 15. Delivery kit |___|___| 16. Episiotomy Kit |___|___| 17. Simple Stretcher |___|___| 18. Instrument cart |___|___| 19. IV Stand |___|___| 20. Delivery table with stirrups |___|___| 21. Examination table |___|___| 22. Gynecological table |___|___| 23. Blood pressure monitor |___|___| 24. Stethoscope |___|___| 25. Thermometer |___|___| 26. Timer (clock with second hand) |___|___| Family Planning Supplies 27. Non-scalpel vasectomy dissecting forceps 28. Non-scalpel vasectomy ringed forceps 29. Uterine elevator 30. Ramathibodi hook 31. Trocar 32. Scalpel |___|___| 33. Forceps |___|___| 34. Implanon applicator Individual Protection Equipment and Infection control 35. Gown |___|___| 36. Mask |___|___| 37. Sterilized gloves |___|___| 38. Bucket with lid 39. Sharps container 40. Sink or basin 41. Eye protection 42. Sterilization test strip 43. Chlorine powder (500 grams or more) Baseline Report 261 Transportation Equipment (in working condition) 44. Motorcycle |___|___| 45. Bicycles |___|___| 46. Canoe/boat |___|___| 47. Ambulance/car/truck |___|___| SECTION 3. PHARMACY FACILITIES Next, I would like to ask you a few questions about the pharmacy facilities. Can you please take me to the pharmacy or the room where you store drugs? No. QUESTION RESPONSE SKIP/ INSTRUC￾TIONS 48. Does this facility have a pharmacy or a separate room for drug storage? RECORD OBSERVATIONS YES…………………………………………………. NO………………………………………………….. 01 00 If no,  51 49. Are there enough shelves for storing drugs/other supplies (nothing on the floor)? RECORD OBSERVATIONS YES…………………………………………………. NO………………………………………………….. 01 00 50. Are the stored items protected from sun? RECORD OBSERVATIONS YES…………………………………………………. NO………………………………………………….. 01 00 SECTION 4. BASIC DRUGS AND SUPPLIES PLEASE ASK THE PERSON RESPONSIBLE FOR THE PHARMACY For this section, I need to speak with the pharmacist, or whoever is responsible for the pharmacy. I would like to ask some questions regarding the facility’s drug supply. I am going to read you a list of drugs. For each one, please tell me if the drug is currently in stock and if the drug has been out of stock at any point in the past 30 days. For selected drugs I will ask to see the drug and drug registry to verify. Part A: Is this drug currently in stock? YES…..01 NO….00 DON’T KNOW….99 Part B: If YES to part A, has this drug been in stock continuously during the past 30 days? YES…..01 NO….00 DON’T KNOW….99 Basic drugs 51. Halothane or Ketamine 01 00 99 01 00 99 52. Atropine 01 00 99 01 00 99 Local anesthetics 53. Lidocaine or Bupvicaine 01 00 99 01 00 99 Analgesics, Antipyretics 54. Acetylsalicylic acid 01 00 99 01 00 99 262 The Impact of USAID’s Integrated Health Program in the DRC 55. Ibuprofene or paracetamol ASK TO SEE THE DRUG AND DRUG’S REGISTRY TO VERIFY. 01 00 99 01 00 99 Anti-Allergy 56. Hydrocortisone or Dexamethasone 01 00 99 01 00 99 Anticonvulsants 57. Diazepam 01 00 99 01 00 99 Intestinal anthelmintics 58. Mebendazole or albendazole ASK TO SEE THE DRUG AND DRUG’S REGISTRY TO VERIFY. 01 00 99 01 00 99 Antibacterial 59. Amoxicilline 01 00 99 01 00 99 60. Ampicilline 01 00 99 01 00 99 61. Ciprofloxacine 01 00 99 01 00 99 62. Cotrimoxazole ASK TO SEE THE DRUG AND DRUG’S REGISTRY TO VERIFY. 01 00 99 01 00 99 Antiviral 63. PEP kit 01 00 99 01 00 99 64. Tuberculosis 65. Isoniazide + rifampicine 01 00 99 01 00 99 66. Éthambutol + isoniazide 01 00 99 01 00 99 67. Streptomycine 01 00 99 01 00 99 68. Rifampicine + Isoniazide + éthambutol 01 00 99 01 00 99 69. Antiamoebic 70. Métronidazole 01 00 99 01 00 99 71. Antimalarial 72. Sulfadoxine-pyrimethamine (SP) ASK TO SEE THE DRUG AND DRUG’S REGISTRY TO VERIFY. 01 00 99 01 00 99 73. Quinine 01 00 99 01 00 99 74. Artéméther + luméfantrine 01 00 99 01 00 99 75. Artésunate + Amodiaquine ASK TO SEE THE DRUG AND DRUG’S REGISTRY TO VERIFY. 01 00 99 01 00 99 76. Insecticide-treated nets 01 00 99 01 00 99 77. Antianemic 78. Acide folique 01 00 99 01 00 99 79. Iron syrup or iron dextran or Iron sulfate + acide folique 01 00 99 01 00 99 Baseline Report 263 80. Antiangoreux 81. Nitroglycérine 01 00 99 01 00 99 82. Hypotensive 83. Aténolol/Nifedipine 01 00 99 01 00 99 84. Heart attack/failure 85. Digoxine 01 00 99 01 00 99 86. Épinéphrine 01 00 99 01 00 99 87. Furosémide ou Hydrochlorothiazide 01 00 99 01 00 99 88. Antidiabetic 89. Insuline 01 00 99 01 00 99 90. Metformine/Glibenclamide 01 00 99 01 00 99 91. Trouble Hydro-électrolutiques 92. ORS 01 00 99 01 00 99 93. Minerals 94. Zinc sulfate 01 00 99 01 00 99 95. Oxytocics 96. Oxytocine 01 00 99 01 00 99 97. Family Planning 98. Male condoms 01 00 99 01 00 99 99. Female condoms 01 00 99 01 00 99 100. Combined oral contraceptive pill (COC) 01 00 99 01 00 99 101. Progesterone only pills (POP) 01 00 99 01 00 99 102. Emergency contraception 103. Intrauterine device (IUD) 01 00 99 01 00 99 104. Norplant, Jadelle, and Sino-Implant II 105. Implanon or implanon nxt 01 00 99 01 00 99 106. Depot Medroxyprogesterine acetate (Depot Provera) 01 00 99 01 00 99 107. Saynana Press 01 00 99 01 00 99 108. Cycle beads 01 00 99 01 00 99 109. Antiseptics and disinfectants 110. Alcool dénaturé 70° 01 00 99 01 00 99 111. Alcool iodé 2 % ou polyvidone iodé 01 00 99 01 00 99 264 The Impact of USAID’s Integrated Health Program in the DRC 112. Other 113. Wound stitching material (Suture synthétique absorbable et Nylon) 01 00 99 01 00 99 114. Does this facility offer vaccination services? YES NO If no, skip to next module Section 5. Cold Chain 115. How many refrigerators are used for vaccines? ASK TO SEE THE REFRIGERATOR TO VERIFY. If 0,  123 Respond to the following separately for each refrigerator. 116. How is the refrigerator powered? FUEL……………………………………………………….01 ELECTRIC………………………………………………..02 SOLAR…………………………………………………….03 GAS………………………………………………………..04 OTHER……………………………………………………05 __________________________________ (SPECIFY) 117. Does it have a working thermometer? YES NO If no,  119 118. What temperature is the thermometer currently displaying? |__|__|__| 119. Does it have a temperature log? TAKE A PHOTO OF THE LOG. IF THE LOG HAS MULTIPLE PAGES, TAKE A PHOTO OF THE MOST RECENT PAGE USED. YES NO If no,  123 120. Was the temperature logged yesterday morning (AM)? YES NO 121. Was the temperature logged yesterday afternoon (PM)? YES NO 122. TAKE A PHOTO OF THE INSIDE OF THE REFRIGERATOR INCLUDING THE DOOR. 123. Are vaccines stored in locations outside of a refrigerator? ASK TO SEE THE LOCATIONS. YES NO If no, skip to next section. Baseline Report 265 TAKE A PHOTO OF THE VACCINE STORAGE AREA. 124. I am going to read you a list of vaccines. For each one, please tell me if the vaccine is currently in stock and if the vaccine has been out of stock at any point in the past 30 days. Part A: Is this drug currently in stock? YES…..01 NO….00 DON’T KNOW….99 Part B: If YES to Part A, has this drug been in stock continuously during the past 30 days? YES…..01 NO….00 DON’T KNOW….99 125. BCG 01 00 99 01 00 99 126. Pentavalent (DTP, Hib, Hepatitis B) 01 00 99 01 00 99 127. IPV (polio) 01 00 99 01 00 99 128. OPV (polio) 01 00 99 01 00 99 129. Measles 01 00 99 01 00 99 130. Yellow fever 01 00 99 01 00 99 131. Pneumococcal vaccine (PCV) 01 00 99 01 00 99 132. Rotavirus 01 00 99 01 00 99 133. Td 01 00 99 01 00 99 134. TAKE A PHOTO OF THE VACCINE STORAGE AREA. 266 The Impact of USAID’s Integrated Health Program in the DRC Select a random sample of three Pentavalent vaccines, or the largest number available up to 10 if less than three. Record the expiration date and whether the temperature monitor is in Phase 3 or 4. Expiration date Phase 3 or Phase 4 Pentavalent - Vial 1 [_] Phase 3 [_] Phase 4 - Vial 2 [_] Phase 3 [_] Phase 4 - Vial 3 [_] Phase 3 [_] Phase 4 - Vial 4 [_] Phase 3 [_] Phase 4 - Vial 5 [_] Phase 3 [_] Phase 4 - Vial 6 [_] Phase 3 [_] Phase 4 - Vial 7 [_] Phase 3 [_] Phase 4 - Vial 8 [_] Phase 3 [_] Phase 4 - Vial 9 [_] Phase 3 [_] Phase 4 - Vial 10 [_] Phase 3 [_] Phase 4 BCG - Vial 1 [_] Phase 3 [_] Phase 4 - Vial 2 [_] Phase 3 [_] Phase 4 - Vial 3 [_] Phase 3 [_] Phase 4 - Vial 4 [_] Phase 3 [_] Phase 4 - Vial 5 [_] Phase 3 [_] Phase 4 - Vial 6 [_] Phase 3 [_] Phase 4 - Vial 7 [_] Phase 3 [_] Phase 4 - Vial 8 [_] Phase 3 [_] Phase 4 - Vial 9 [_] Phase 3 [_] Phase 4 - Vial 10 [_] Phase 3 [_] Phase 4 Measles - Vial 1 [_] Phase 3 [_] Phase 4 - Vial 2 [_] Phase 3 [_] Phase 4 - Vial 3 [_] Phase 3 [_] Phase 4 - Vial 4 [_] Phase 3 [_] Phase 4 - Vial 5 [_] Phase 3 [_] Phase 4 - Vial 6 [_] Phase 3 [_] Phase 4 - Vial 7 [_] Phase 3 [_] Phase 4 - Vial 8 [_] Phase 3 [_] Phase 4 - Vial 9 [_] Phase 3 [_] Phase 4 - Vial 10 [_] Phase 3 [_] Phase 4 Baseline Report 267 IHP Evaluation HEALTH FACILITY SURVEY Part F. Medical Record Review SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1. Record the province. 2. Record the health zone. 3. Record the name of the health facility or enter the facility ID number. 4. Enter your data collector ID number. 268 The Impact of USAID’s Integrated Health Program in the DRC ANTENATAL CARE: Randomly select 10 women who had their first ANC visit at the facility during the last calendar month(or highest number available if less than 10). # Date of visit Age of pregnancy (weeks/months) First trimester? Blood pressure Tested for Syphilis Tested for HIV Received SP dose 1 Received ITN ANC register version date Measured Result (01/2020 if unknown) 1 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 2 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 3 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 4 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 5 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 6 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 7 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 8 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 9 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know 10 |__|__| weeks months yes no yes no S: D:  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know  yes  no  don’t know Baseline Report 269 MATERNAL AND NEONATAL CARE: Randomly select 10 women who delivered at the facility and who died or were discharged during the last calendar month (or highest number available if less than 10). # Date of delivery Ante￾partum hemrrhge Post￾partum hemrrhge Post￾partum infection Active mgmt of the third period of labor Cesarean Child death before 7 days (includes stillbirth) Essential neonatal services Early initiation of breast￾feeding Maternal death Date of discharge Delivery register version date (01/2020 if unknown) 1  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 2  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 3  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 4  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 5  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 6  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 7  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 8  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 9  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 10  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK  yes  no  DK 270 The Impact of USAID’s Integrated Health Program in the DRC According to the registry, how many of the following occurred at this facility in April, May, and June 2019? Enter -99 if data is missing or unavailable. April May June Prenatal consultations (first visit) Prenatal consultation (fourth visit) Cases of simple diarrhea (children under five) Cases of simple malaria, confirmed (children under five) Cases of simple pneumonia (children under five) LLINs quantity consumed during month New acceptors of family planning (total) BCG doses administered during month Baseline Report 271 IHP Evaluation HEALTH FACILITY SURVEY Part G. Health Worker Interview SECTION 2: PROVIDER INFORMATION, POSITION AND EXPERIENCE First, I would like to ask you some general questions about you. 8. Are you the head of this facility? Yes No Not applicable (N/A) 9. Are you the head of a department or unit? Yes No If no,  11 10. What department or unit do you head? General inpatient Surgery Pediatrics Maternity Other (specify) ______________________ SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1. Record the province 2. Record the health zone 3. Record the name of the health facility or enter the facility ID number 4. Enter your data collector ID number Select all of the physicians, nurses, and midwives responsible for service delivery who are at work on the day of the survey. 5. Have you read him/her the consent script? |_| No |_| Yes If yes,  7 6. If no, why? 7. Did the respondent(s) agree? |_| No |_| Yes If no, end interview 272 The Impact of USAID’s Integrated Health Program in the DRC 11. What is your clinical educational qualification (highest degree or certificate earned)? General Physician……………………………...1 Medical Specialist (please specify)…………...2 Nurse A0/L2……………………………………..3 Nurse A1…............................................ ………4 Nurse A2 ….................................................…5 Nurse A3………………………………………..6 Midwife A0/L2 …………...………………….....7 Midwife A1…..............................................…8 Midwife A2 …………………………………. ...........…9 Midwife A3………………………………….….10 Trained birth attendant…………………..…...11 Community health worker L2………………...12 Community health worker G3 ……………….13 None……………………………………………14 Other……………………………………………96 _____________________________________ (Specify) 12. Do you have a Master’s degree? Yes No If no,  15 13. In what subject is your Master’s degree(s)? (check all that apply) Public health Public administration Business administration Social science / Anthropology Other (specify) Baseline Report 273 14. In what province did you receive your highest clinical degree or certificate? Kinshasa Bas-Uele Équateur Haut-Katanga Haut-Lomami Haut-Uele Ituri Kasaï Kasaï-Central Kasaï-Oriental Kongo Central Kwango Kwilu Lomami Lualaba Mai-Ndombe Maniema Mongala Nord Kivu Nord-Ubangi Sankuru Sud Kivu Sud-Ubangi Tanganyika Tshopo Tshuapa Other location (outside of DRC) (Specify country) ___________________________ No response 15. How many years have you worked at this facility? Less than 1 year More than 1 year (specify number of years) __________ Don't know 16. According to your contract, for how many hours are you supposed work in this health facility in a normal week? Hours............................................................... ...................................................... |___|___| 17. In the last month, how many days did you miss a full day of work due to any of the following reasons? [READ LIST and Enter 0 if no days missed for any reason] Days Concern for safety |___| Personal sickness |___| Care for a sick child |___| Care for a sick adult |___| Lack of pay |___| Bereavement/Funeral |___| Other income generating activities |___| Other reasons |___| 274 The Impact of USAID’s Integrated Health Program in the DRC 18. How do you usually get to work during the dry season? SELECT ALL THAT APPLY Walk Bicycle Motorbike Boat Private car Public transportation Other (specify) 19. How do you usually get to work during the rainy season? SELECT ALL THAT APPLY Walk Bicycle Motorbike Boat Private car Public transportation Other (specify) 20. How long does it usually take you to get to work during the dry season? Minutes: Hours: 21. How long does it usually take you to get to work during the rainy season? Minutes: Hours: 22. In general, how safe or unsafe is your trip to work during the dry season? Very safe Safe Unsafe Very unsafe Don’t know 23. In general, how safe or unsafe is your trip to work during the rainy season? Very safe Safe Unsafe Very unsafe Don’t know 24. How do you usually get home from work during the dry season? SELECT ALL THAT APPLY Walk Bicycle Motorbike Boat Private car Public transportation Other (specify) 25. How do you usually get home from work during the rainy season? SELECT ALL THAT APPLY Walk Bicycle Motorbike Boat Private car Public transportation Other (specify) 26. How long does it usually take you to get home from work during the dry season? Minutes: Hours: 27. How long does it usually take you to get home from work during the dry season? Minutes: Hours: 28. In general, how safe or unsafe is your trip home after work during the dry season? Very safe Safe Unsafe Very unsafe Don’t know Baseline Report 275 29. In general, how safe or unsafe is your trip home after work during the rainy season? Very safe Safe Unsafe Very unsafe Don’t know 30. Now, we are going to ask you some personal questions. Remember that you can skip any questions that you do not want to answer. 31. In what province were you born? Kinshasa Bas-Uele Équateur Haut-Katanga Haut-Lomami Haut-Uele Ituri Kasaï Kasaï-Central Kasaï-Oriental Kongo Central Kwango Kwilu Lomami Lualaba Mai-Ndombe Maniema Mongala Nord Kivu Nord-Ubangi Sankuru South Kivu Sud-Ubangi Tanganyika Tshopo Tshuapa Other location (outside of DRC) (Specify country) ______________________________ No response 32. What is your age? (record age in years) |___|___| Years 33. Sex of the healthcare provider Male …..........................................................1 Female …......................................................2 34. What is your marital status? Married/in union……………………………………….1 Single…………………………………………….2 Widowed…………………………………………3 Separated………………………………………..4 Other……………………………………………96 35. How many living children, of any age, do you have? If < 0,  37 36. How many of your living children are younger than five? 37. Do you live away from your family due to work reasons? No………………………………………………..0 Yes……………………………………………….1 276 The Impact of USAID’s Integrated Health Program in the DRC SECTION 3. TRAINING Topic Attended Who administered this training? 38. Prenatal consultations No (skip to 39) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 39. Integrated Management of Childhood Illness No (skip to 40) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 40. Malaria (children under 5) No (skip to 41) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 41. Long-lasting insecticidal net (LLIN) distribution? No (skip to 42) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 42. Family planning? No (skip to 43) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 43. Reach Every District training? No (skip to 44) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 44. Administrative and human resource management? No (skip to 45) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 45. Financial management? No (skip to 46) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 46. Management ethics and patient confidentiality? No (skip to 47) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 47. Training of health care providers? No (skip to 48) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 48. Capacity-building of CODESAs? No (skip to 49) Yes Internal/BCZ office Province/Government The next set of questions is about the training participation and includes internal training provided by members of this health facility as well as training provided by provincial/government officials, USAID/IHP partners or other NGOs with which this facility may be affiliated. If you attended multiple trainings on the same topic, please tell us about the most recent one. In the last calendar year (2018), did you attend training on any of the following subjects as part of your position in the health facility? Baseline Report 277 USAID/IHP Other: ____________________ 49. Health information management (e.g. DHIS2, SNIS, MAPEPI)? No (skip to 50) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 50. Community scorecard processes? No (skip to 51) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 51. Material resource management (medicines, supplies, equipment)? No (skip to 52) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 52. Gender issues and/or the gender transformative approach? No (skip to 53) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 53. Care of victims of SGBV No (skip to 54) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 54. Supportive supervision? No (skip to 55) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 55. Team management? No (skip to 56) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 56. Planning, monitoring and evaluation? No (skip to 57) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 57. Disease prevention, health promotion and re-adaptation care? No (skip to 58) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 58. Management of outbreaks, emergencies and disasters? No (skip to 59) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 59. Health research? No (skip to 60) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 60. Data analysis? No (skip to 61) Yes Internal/BCZ office Province/Government USAID/IHP Other: ____________________ 61. Use of data for evidence-based decision-making? No (skip to 62) Internal/BCZ office 278 The Impact of USAID’s Integrated Health Program in the DRC Yes Province/Government USAID/IHP Other: ____________________ iHUMAN RESOURCES INFORMATION SYSTEM (iHRIS) TRAINING 62. Did you or anyone else in this office participate in a training on Human Resources Management using the iHuman Resources Information System (iHRIS)? No Yes Don’t know If no,  70 63. Did you participate? No Yes 64. How long ago was that training? 0-3 months ago 4-6 months ago 7-12 months ago More than 1 year ago Don't know 65. Sometimes participants cannot attend the entire training workshop. How much of that workshop did you attend? Less than half About half of it All or most of it Not applicable 66. It can be difficult to remember all of the material presented during training. How much do you remember of what you learned in the training on Human Resources Management using the iHRIS? Less than half About half of it All or most of it Not applicable 67. How often do you use the material presented during that workshop in your day to day work? Never Rarely Sometimes Often Always Not applicable 68. What language was the training conducted in? Swahili French Lingala Tshiluba English Other (specify) _____________ Don't know 69. Did anyone attending the training report that the language of the training created challenges with understanding the material? No Yes Don’t know Baseline Report 279 SECTION 4: MANAGEMENT AND SUPERVISION 70. Who is your direct supervisor WITHIN the facility? Head of the facility….…..................................1 Medical specialist (please specify)………………2 Physician (general) …. .......…………………………3 Nurse A0/L2 ……………………………….................4 Nurse A1…………………………………………………….5 Nurse A2…………………………………………………….6 Nurse A3…………………………………………………….7 Midwife A0/L2 …………..…………………..............8 Midwife A1…. .........……………………………………9 Midwife A2 …. ..........………………………………..10 Midwife A3………………………………….……………11 Trained birth attendant…………………………….12 Community health worker L2 …………………..13 Community health worker G3…………………..14 Other………………………………………………………..96 _____________________________________ (Specify) Skip if interviewi ng facility head 71. When was the most recent time that your supervisor WITHIN the facility interacted with you in a supervisory capacity? DO NOT READ CHOICES, BUT CIRCLE THE APPROPRIATE NUMBER FOR THE RESPONSE GIVEN Today....................................1 2-7 days ago .........................2 More than 7 days ago ..........3 Not applicable ....................97 Don’t know.........................98 If N/A,  73 72. Which activities has your supervisor WITHIN the facility performed in the past 30 days? DO NOT READ CHOICES; CIRCLE THE ANSWER(S) GIVEN OR WRITE IN IF “OTHER”. IF MORE THAN ONE ANSWER IS GIVEN, CIRCLE ALL THAT APPLY. Checked records.......................................... 1 Checked finances ........................................ 2 Observed consultation................................ 3 Asked knowledge questions........................ 4 Provided health instruction ........................ 5 Provided administrative instruction............ 6 Provided instruction in filling HMIS............. 7 Provide feedback on job performance………8 Nothing ....................................................... 9 Other……………………………………………………...96 _____________________________________ (Specify) 280 The Impact of USAID’s Integrated Health Program in the DRC 73. What are the three biggest difficulties that you face in doing your job in the facility? How would you rank these in terms of the biggest, second biggest and third biggest difficulties? DO NOT READ CHOICES; CIRCLE THE ANSWER(S) GIVEN, OR WRITE IN IF “OTHER”. CIRCLE ONLY THREE CHOICES. 1st biggest |__|__| 2nd biggest |__|__| 3rd biggest |__|__| Lack of trained personnel/staff…. ...............…1 Lack of knowledge…............……. .................…2 Lack of feedback of performance……………..3 Patients come to clinic too late…................…4 Inadequate transport…. ......……. .................…5 Lack of time…. .....................……. .................…6 Lack of motivation…............……. .................…7 Staff shortages…..................……. .................…8 Poor working environment…. .....................…9 Lack of supplies and drugs…. ....................…10 Lack of equipment…............……. ...............…11 Irregular/no water supply….…...................…12 Irregular/no electricity…. ....……. ...............…13 No laboratory or poor lab quality …..........…14 Lack of space in the facility….....................…15 Lack of supervision…. ..........……. ...............…16 Inadequate furniture...........…................ ……17 No problem................................................. 18 Other…………………………………………………………96 _____________________________________ (Specify) If “No problem”,  76 74. Have you discussed the problems with your supervisor(s) (either within or outside the facility) within the last year? IF RESPONDENT HAS NO INTERNAL OR EXTERNAL SUPERVISORS, CIRCLE NOT APPLICABLE No................................................................ 0 Yes............................................................... 1 Not applicable………………………………………….97 If “Yes” or “N/A”, 76 75. After these discussions did you notice any improvements in the problems? No................................................................ 0 Yes............................................................... 1 Baseline Report 281 SECTION 5: PROVIDER INCOME Now we would like to ask you some questions about your income, both from this job and any other sources of income you may have. These questions help us understand the strategies you use to make a living No. Question 76. What is the total monthly net salary you are supposed to receive from your employment at this facility (after deduction and taxes etc.)? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD If 0,  82 77. What was the total monthly salary you actually received from your employment at this facility (after deduction and taxes etc.) for the last full month that you worked? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 78. Was your last salary payment on time? No, late……………………………………………………..0 Yes, on time .................................................1 Not yet paid……………………………………………….2 79. Have you ever received your full state salary on time? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 If no,  82 80. Do you remember the last time that you were paid your full state salary on time? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 If no or don’t know,  82 81. When was the last time that you were paid your full salary on time? Month: Year: 82. According to your contract are you supposed to receive a monthly hazard pay? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 If no or don’t know,  86 83. What is the amount of hazard pay you are supposed to receive monthly? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 84. What is the amount of monthly hazard pay you actually received for the last full month that you worked? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 85. Was your last hazard payment on time? No, late……………………………………………………..0 Yes, on time .................................................1 Not yet paid……………………………………………….2 86. According to your contract, are you supposed to receive a monthly performance bonus? No ..........................................................0 Yes..........................................................1 Don’t know…..…………………………………….98 If no,  90 87. Are you aware of any performance criteria attached to this bonus? No ..........................................................0 Yes..........................................................1 Don’t know…..…………………………………….98 88. What is the amount of monthly performance bonus you are supposed to receive? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 89. What is the amount of monthly performance bonus that you actually received for the last full month that you worked? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 282 The Impact of USAID’s Integrated Health Program in the DRC 90. Do you receive any monthly income from user fees collected at the facility? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 If no or don’t know,  92 91. What is the amount of monthly user fees you received from the last full month that you worked? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 92. According to your contract, are you supposed to receive a housing allowance? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 If no or don’t know,  95 93. What is the amount of monthly housing allowance that you are supposed to receive? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 94. What is the amount of monthly housing allowance that you actually received for the last full month that you worked? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 95. Do you receive free housing through work? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 96. Do you receive any of the other following allowances through your work? READ EACH POSSIBLE RESPONSE TO THE RESPONDENT. CIRCLE ONLY THOSE FOR WHICH THE RESPONDENT ANSWERED “YES”. MULTIPLE RESPONSES ALLOWED Rural allowance………………………………..….…1 Transport allowance………………………..….….2 Medical allowance…………………………….….…3 Overtime allowance………………………..….....4 Uniform allowance……………………………….…5 Non-practice allowance………………………....6 None………………………………………………………7 Other (please specify)………………………..…96 _____________________________________ (Specify) If none,  98 97. FOR EACH RESPONSE CIRCLED IN QUESTION 521, POSE THE QUESTION: How MUCH do you receive for (STATE APPROPRIATE CATEGORY) per month? IF UNKNOWN WRITE 99999. Rural allowance |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD Transport allowance |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD Medical allowance |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD Overtime allowance |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD Uniform allowance |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD Non-practice allowance |___|___|___|___|___| ___| CF Baseline Report 283 |___|___|___|___|___| ___| USD Other (please specify) ___________________ |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 98. In the past 12 months, did you receive any per diems (e.g. for workshops, training or other travel)? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 If yes,  99 99. If yes, how much do you receive in per diems for training during the past year? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 100. Do you supplement your main income with extra private health care work? No ................................................................0 Yes................................................................1 Don’t know…..………………………………………….98 If yes,  101 101. If yes, which of these options best describes where this private practice is located? (SELECT ALL THAT APPLY) Same building as main job…………………….1 At home………………………………………..........2 At home of a colleague…………………………3 In rented premises……………………………..…4 At patient/client’s home…………………….…5 Private facility…………………………………….…6 NGO facility………………………………………..…7 Other (please specify)…………………………96 _____________________________________ (Specify) 102. How much do you get paid for this private practice during the last month? |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 103. Do you have any additional sources of income? Yes..........................................................1 No ..........................................................2 If no,  106 104. Can you tell me which of the additional sources of income you have? Agriculture .............................................1 Livestock (elevage)………………………………2 Trade/Business ......................................3 House rent .............................................4 Gifts/contributions from family…………..5 Other……………………………………………….…96 _____________________________________ (Specify) 105. If yes to one or more of any of the above, what is the amount for each received in the past 30 days? IF UNKNOWN WRITE 99999. Agriculture |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD Livestock (elevage) |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD Trade/Business |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD House rent |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD 284 The Impact of USAID’s Integrated Health Program in the DRC Other (specify) ________________________ |___|___|___|___|___| ___| CF |___|___|___|___|___| ___| USD SECTION 6: PROVIDER SATISFACTION In this part of the questionnaire we would like to ask you some questions regarding your satisfaction with your current job. All answers are confidential and any identifying information will be removed. I’m now going to read you a series of statements about your level of satisfaction. Please indicate whether you are satisfied or dissatisfied. If you are neither satisfied nor dissatisfied, then say neutral. [READ FROM THE LIST/SCALE AND ASK WHICH CATEGORY APPLIES] How satisfied or dissatisfied are you with the following aspects of your work? No. Satisfied Neutral Dissatisfied Not applicable Relationships 106. Relationships with your supervisors within the facility 107. Relationships with your supervisors outside the facility 108. Relationships with colleagues 109. Relationships with local leaders in the community 110. Your level of respect within the community Management 111. Management of the health facility - by the Ministry of Health or by the relevant management office 112. Transparency of the management of financial resources by the facility 113. Your participation in decision-making to resolve problems in the facility Staffing and Workload 114. The number of staff working in the facility 115. The description of your job role and tasks 116. The stability of your job/contract Work environment 117. Availability of medicines in the health facility 118. Availability of equipment in the health facility 119. Availability of medical supplies in the health facility 120. The physical condition of the health facility building 121. Your workload 122. The division of work between you and your colleagues 123. The division of work between caring for patients and other tasks 124. The variety of your tasks 125. Flexibility with attendance and work hours Training and Performance 126. Your ability to provide a high quality of care to patients 127. Your ability to meet the needs of the community 128. Your level of responsibility 129. Your training opportunities to upgrade your skills and knowledge 130. How you and other colleagues are selected to attend training 131. Your ability to use skills learned from training in your work Baseline Report 285 132. Your opportunities for promotion Salary and Benefits 133. Your salary in relation to your workload 134. Your salary in relation to your competence/skills 135. Employment benefits (travel allowance, bonus, health care etc.) 136. Safety and security to live and practice in the community 137. Living accommodations for your family 138. Quality of your children’s education Overall satisfaction 139. Overall, how satisfied or dissatisfied are you with your job? 140. Which three of the aspects mentioned above, in your opinion, are most important in affecting your level of satisfaction with your job? Relationships with your supervisors within the facility Relationships with your supervisors outside the facility Relationships with colleagues Relationships with local leaders in the community Your level of respect within the community Management of the health facility - by the MoH or by the relevant management office Transparency of the management of financial resources by the facility Your participation in decision-making to resolve problems in the facility The number of staff working in the facility The description of your job role and tasks The stability of your job/contract Availability of medicines in the health facility Availability of equipment in the health facility Availability of medical supplies in the health facility The physical condition of the health facility building Your workload The division of work between you and your colleagues The division of work between caring for patients and other tasks The variety of your tasks Flexibility with attendance and work hours Your ability to provide a high quality of care to patients Your ability to meet the needs of the community Your level of responsibility Your training opportunities to upgrade your skills and knowledge How you and other colleagues are selected to attend training Your ability to use skills learned from training in your work Your opportunities for promotion Your salary in relation to your workload Your salary in relation to your competence/skills Employment benefits (travel allowance, bonus, health care etc Safety and security to live and practice in the community Living accommodations for your family Quality of your children’s education 141. Are you considering voluntarily leaving your current position within the next 12 months? No Yes (go to 142) Undecided 142. If you leave your current position, what will you most likely do? (check all that apply) Retire/Not work Promotion at this facility Promotion at another health facility Similar job at another health facility Private clinical practice Work with a health-related NGO 286 The Impact of USAID’s Integrated Health Program in the DRC Work in a non-health related industry School/other training Other (specify) ______________________ SECTION 7: PROVIDER MOTIVATION Please tell me whether you agree or disagree with the following statements. If you neither agree nor disagree say “neutral.” READ FROM THE LIST/SCALE AND ASK WHICH CATEGORY APPLIES No QUESTION Agree Neutral Disagree N/A PRIDE 143. This health facility has a good reputation in the community 144. It is a source of pride to get a job at this facility 145. I am satisfied that I accomplish something worthwhile in this job 146. Healthcare providers at this health facility pride themselves on providing good services to patients FINANCIAL REWARD 147. The effort that we at this facility put into this job is reflected in our pay 148. My job offers adequate pay compared with similar jobs 149. The income I receive is a fair reflection of my skills, knowledge and training 150. The income that I receive from working at this facility more than covers my basic needs such as food, transport, and accommodation 151. With this job I have worries about how to support myself and my family PERCEIVED SELF-EFFICACY 152. I am confident about my ability to handle my work 153. I effectively cope with any new challenges that occur in my work life 154. I feel that at work things are going the way I would like them to 155. I feel that I have control of things concerning my work 156. I have received sufficient training to be able to perform my job well COMMITMENT 157. I only do this job so that I get paid at the end of the month 158. I intend to leave this facility as soon as I can find another position 159. I would recommend this profession to my children SELF-PERCEIVED CONSCIENTIOUSNESS 160. I am always reliable and dependable at work 161. My work is consistently of a high quality 162. I am a hard worker 163. I am always on time at work 164. I spend my time at work on work-related activities 165. I am rarely absent from work 166. I am careful not to make errors at work 167. When I am not sure how to treat a patient’s condition, I look for information or ask for advice 168. I do things which need to be done without being asked or told Baseline Report 287 SECTION 8. PROVIDER ATTITUDES No QUESTION Agree Neutral Disagree N/A In general, at this facility… 169. Women and men are equally considered for hiring. If disagree: which sex is preferred? |__|men |__| women 170. Women and men have the same opportunities for promotion. If disagree: which sex is preferred? |__|men |__| women 171. Women and men have the same opportunities for training. If disagree: which sex is preferred? |__|men |__| women 172. Women and men in the same positions perform similar tasks. 173. Women and men in the same positions with the same training and experience earn the same salary. If disagree: which sex earns more? |__|men |__| women 174. Women are given time off to deal with family responsibilities. 175. Men are given time off to deal with family responsibilities. 176. Women are given time off after the birth of a child. 177. Men are given time off after the birth of a child. In the past 12 months… 178. I have had to turn down an opportunity for training because of family responsibilities. 179. I have experienced threats or verbal abuse from other staff members. 180. I have experienced physical violence (slapping, hitting, punching) from other staff members. 181. I have experienced other unwanted physical contact from other staff members. 182. I have experienced threats or verbal abuse from a patient or patient’s family. 183. I have experienced physical violence (slapping, hitting, punching) from a patient or patient’s family. 184. I have experienced other unwanted physical contact from a patient or patient’s family. 185. I have experienced threats or verbal abuse from a patient or patient’s family. 186. I have experienced physical violence (slapping, hitting, punching) from a patient or patient’s family. 187. I have experienced other unwanted physical contact from a patient or patient’s family. 288 The Impact of USAID’s Integrated Health Program in the DRC Please tell me whether you strongly agree, agree, disagree or strongly disagree with the following statements. If you neither agree nor disagree say “neutral.” There are no right or wrong answers so feel free to express your thoughts openly READ FROM THE LIST BELOW AND ASK WHICH CATEGORY APPLIES (1-5) No QUESTION Strongly agree Agree Neutral Disagree Strongly disagree PRIDE Patient perceptions 188. Patients I care for are not educated enough make good health decisions for themselves 189. Patients I care for are not grateful for the efforts I make when I care for them 190. I consider my patients to be worthy of respect no matter how poor or low status they are 191. Patients often treat me without respect, so it’s hard to treat them with respect 192. Patients I care for make bad decisions regarding their health no matter what I tell them 193. Engaging Patients in discussions leads to better health outcomes than just telling them what is best for them 194. My Patients will work hard to improve their health when they are given the proper information Provider roles 195. My role is to provide clinical care, not to teach patients about how to take care of themselves 196. I do not spend a lot of thought about what Patients may think about their experience at the clinic as I have other things to worry about 197. An important part of my job is to communicate with Patients to make sure they understand their care 198. I try hard to think about all of the Patients’ health care needs not just solving their immediate problem 199. I was trained to provide clinical care, being respectful to every Patient is not my job 200. When medicine is given, it is important that I explain well what it does for the Patient and how it helps them 201. I think it is important to spend enough time with each Patient, even if I have other job demands 202. My job is to diagnose and treat parents not to be a health educator Gender norms 203. A man should have the final word about decisions in his home 204. It is the man who decides what type of sex to have with his wife 205. A women most important role is to take care of her home and cook for her family 206. If a woman has a good idea, her husband should listen even if he disagrees Baseline Report 289 207. Men and women should decide together about how many children to have 208. A man is expected to discipline his women 209. Men should help take care of the children in the household 210. There is never a good reason for a man to beat his wife 290 The Impact of USAID’s Integrated Health Program in the DRC PROVIDER VIGNETTES 1) Do you regularly provide child health services? a) No (skip vignette #1) b) Yes 2) Do you regularly provide ANC? a) No (skip vignette #2) b) Yes 3) Do you regularly provide family planning services? a) No (skip vignette #3) b) Yes Instructions for the data collector: Read the following scenarios to the health worker and pose questions about the case. The health worker should not refer to any printed materials, the internet, or discuss the case with anyone else. Instructions for the health worker: In this exercise we will lead you through hypothetical patient case descriptions while asking you case management questions along the way. We want you to form an image of the patient presented in the case description and to imagine that this person is sitting in front of you in your consultation room. The answers you provide will be anonymous and will only be used for scientific research. You will not personally be evaluated based on your responses and the information you provide will not be shared with your facility’s administration. - First, I will give a short description of the patient and their symptoms. I want to know what questions you would ask the patient in order to fully understand their situation. These questions could be about their health, their personal characteristics, and their family. Once you have asked your questions, I will give you more information about the patient. - Second, I will ask you to tell me how you would conduct your physical exam. I will tell you the results of the physical exam. - Third, based on the patient’s symptoms and the physical examination, I will ask for your differential diagnosis. That means I would like to know which illnesses or conditions you most strongly suspect that the patient has. - Fourth, I will ask you which tests you would order. I will then give you the results of the tests. - Fifth, I will ask you for your final diagnosis of the patient. - Sixth, you should tell me what medicines and/or treatments you would give. - Lastly, please describe the counseling that you would give to the patient before they leave your office. The answers you provide will be anonymous and will only be used for scientific research. You will not personally be evaluated based on your responses and the information you provide will not be shared with your facility’s administration. Baseline Report 291 Vignette: Child Illness A four-year-old boy presents with 3 days of diarrhea. His mother brought him to the hospital after he had a seizure at home. 1) What questions do you ask the child’s mother? Anything else? a) No questions b) Consistency of stools c) Frequency of stools d) Progression of diarrhea (worsening or getting better) e) Blood in stools f) Mucous in stools g) Temperature h) Vomiting i) Abdominal pain j) Volume and frequency of eating and drinking k) Treatments given thus far l) Has this happened before? m) Anyone else in household is sick? n) Mother's marital status o) Caretakers' occupation p) Housing, water, and sanitation conditions q) Family composition/number of siblings r) Religious affiliation s) Other (specify) ______________________________________________________ At first the diarrhea was watery 5- 10 times a day, but now it’s worse: small volume stools about 20 times a day, mixed with blood and mucous. He’s felt very “hot” and has been weak and listless. He has been consuming small amounts of soft food and water. He has not been given any treatments and this is the first time that he has had this type of illness. His parents are married, and his father is a teacher. He lives in a mud brick house with his parents and two older siblings. The family gets their water from a nearby spring and uses a pit latrine that they share with another family. 2) What does your physical examination of the patient include? Anything else? a) No examination b) Affect/demeanor c) Temperature d) Heart rate e) Respiratory rate f) Capillary refill g) Skin turgor h) Abdominal palpation 292 The Impact of USAID’s Integrated Health Program in the DRC i) Weight j) Other (specify) ___________________________________________________________ The physical exam is notable for general apathy, a temperature of 40 heart rate 130/minute RR 20/minute, not deep capillary refill 2 seconds skin turgor intact and a distended, tender abdomen with visible bowel loops. The mother reports that at his last visit he weighed 17 kg. Today he weighs 15.5 kg. 3) What are the most likely diagnoses (i.e. the differential diagnoses?? a) Dysentery/Shigella b) Rotavirus c) E-Coli d) Cryptosporidium e) Malaria f) Anemia g) Others (specify) _______________________________________________ h) Don’t know 4) What tests, if any, do you order? Assume that the needed tests can be conducted at your facility. a) No tests b) Direct microscopic stool examination c) Stool culture d) Hemoglobin e) Hematocrit f) WBC g) Electrolytes h) Medical imaging i) Other (specify) _______________________________________________ Testing indicates that the child has dysentery caused by Shigella dysenteriae type 1. 5) Based on these results, what treatment(s) do you administer, if any? Assume that everything that is needed is in-stock in the facility. a) No treatment b) Flagyl c) Antibiotic d) Other medicine (specify) _______________________________________________ e) Fluids (oral) f) Fluids (IV) g) Treatment for fever h) Other (specify) _______________________________________________ 6) What are the key points that you tell the child’s mother during counseling? Anything else? a) No counseling Baseline Report 293 b) Diagnosis c) How the infection is transmitted d) How to administer medicine e) Feeding f) Hydration g) Hygiene h) How to monitor/signs of worsening i) When to bring him back for follow-up j) Other (specify) _______________________________________________ 294 The Impact of USAID’s Integrated Health Program in the DRC Vignette: Antenatal care A nineteen-year-old woman comes to you for her first antenatal care visit. She is visibly pregnant and estimates that she is at least 20 weeks along. She has not taken a pregnancy test and did not come earlier because she lives far from the health center. 1) What questions do you ask her? Anything else? a) No questions b) Number of pregnancies c) Number of deliveries (live births) d) Number of miscarriages e) Number of children alive f) Number of children born alive who have died g) Timing of last menstrual period h) History of hypertension i) History of diabetes j) Family health history k) Past illnesses l) Complaints during this pregnancy m) Sexual history (e.g. sexual activity, number of sexual partners) n) History of intimate partner violence and/or sexual violence o) Marital status p) Occupation q) Education level r) Family/support network s) Religious affiliation t) Other (specify) _______________________________________________ She tells you that this is her first pregnancy. She was vomiting at the beginning of her pregnancy but has not vomited for at least 3 weeks. She complains of recent mild headaches and swelling in her feet and ankles. There is a history of twins in her family. Her older sister died in childbirth three years ago. She does not know the reason. The only notable elements from her social history are that she has been married for a year and works on her family’s farm. 2) What does your physical examination of the patient include? Anything else? a) No examination b) Height c) Weight d) Temperature e) Blood pressure f) Pulse g) Respiratory rate Baseline Report 295 h) Abdominal palpation i) Breast exam j) Fetoscope k) Vaginal exam l) Fundal height m) Presence of edema n) Other (specify) _______________________________________________ The woman is 1.7 m tall and weighs 73 kgs. Her blood pressure is 142/93, pulse 85 beats per minute, and respiratory rate is 16 breaths per minute. She does not have a fever or edema. You detect fetal movement. 3) What tests, if any, do you order? Assume that the needed tests can be conducted at your facility. a) No tests b) Pregnancy test c) Proteinuria d) HIV e) Syphilis f) Malaria g) Hemoglobin h) Hematocrit i) Blood grouping j) Echography k) Other (specify) _______________________________________________ The tests indicate that she is positive for pregnancy and proteinuria (protein/creatinine ratio of 60 mg/mmol) and negative for HIV and syphilis. 4) Based on the history, exam and test results, what is your assessment of the patient? a) Preeclampsia b) Healthy pregnancy c) Other (specify) _______________________________________________ d) Don’t know 5) What is your next step? Assume that everything that is needed is in-stock in the facility. Indicate the frequency and dose if applicable. a) No treatment b) Transfer to hospital c) Hypotensive drug d) Anti-convulsive drug e) Induction of labor f) Other (specify) _______________________________________________ 296 The Impact of USAID’s Integrated Health Program in the DRC 6) What are the key points that you would tell Thérèse during counseling? Anything else? a) No counseling b) Causes, symptoms, and risks of having preeclampsia c) Referral to hospital d) How to take medicine e) Reduced physical activity f) Bed rest g) Minimize salt intake h) Increase water intake i) Increase protein intake j) Signs that emergency care is needed k) When to return for follow-up l) Other (specify) _______________________________________________ Baseline Report 297 Vignette: Family planning A 22-year old woman comes to the clinic on her own. She tells you that she is interested in using contraceptives. 1) What questions do you ask her? a) No questions b) Menstrual history (e.g. first day of last menstrual period, length of bleeding (days), menstrual frequency, other patterns of uterine/vaginal bleeding) c) Gynecologic and obstetrical history (e.g. pregnancy/-ies, recent delivery, miscarriage, or termination) d) Drug history including contraceptive use (past and/or current) e) Recent intercourse f) Other health conditions and behaviors (e.g. allergies, breastfeeding, hypertension, smoking). g) Marital status h) Length of marriage i) Education level j) Occupation (self) k) Occupation (husband) l) Number of children m) Age of youngest child n) Pregnancy intentions (including timing and spacing if children are desired) o) Contraceptive preferences p) Sexual history (e.g. sexual activity, sexual partners, past STD history) q) History of intimate partner violence and/or sexual violence r) Religious affiliation s) Other (specify) _______________________________________________ You learn that she has never used any contraceptive method, aside from the occasional use of condoms. She has no children. Her menstrual cycle is regular, and she has no allergies or other health issues. She is currently married and having sex but would like to delay having children for another 3 years at least. 2) Do you counsel her in choosing a contraceptive method? Assume that there are multiple methods in￾stock at your facility today. a) No b) Yes (skip to Q4) 3) Why don’t you counsel her in choosing a contraceptive method? Any other reason? a) She is married. b) She has no children. c) Her husband is not with her at the health center. d) Condoms are sufficient. 298 The Impact of USAID’s Integrated Health Program in the DRC e) Provider’s religious beliefs f) Other (specify) _______________________________________________ 4) What information do you provide when counselling her about family planning methods? Any others? a) No counseling b) Types of contraceptive methods available today (e.g., condoms, oral contraceptives, injectable contraceptives, intrauterine device (IUD), implants, etc.) c) Types of contraceptive methods available consistently (i.e. never/rarely stocked out) d) Duration of protection from pregnancy e) Effectiveness of methods in preventing pregnancy f) Effectiveness of methods in protecting against STDs, such as HIV g) Correct use of methods h) Side effects including lack of periods i) Safety of the method j) Pain/discomfort during administration k) Cost of methods l) Provider’s recommendation of a specific method m) Other (specify) _______________________________________________ 5) How do you determine which family planning method to prescribe her and what factors do you consider when making this decision? a) Effectiveness b) Side effects c) Her medical history d) Her preferences with regard to methods e) Her preferences with regard to timing of pregnancy/childbearing f) Her confidence in being able to use the method correctly and consistently g) Acceptability of method use by her husband h) Acceptability of method use by her peers i) Cost of method j) Whether someone at the facility is trained and/or confident in their ability to administer the method k) Availability of the method on that day l) Other (specify) _______________________________________________ After counselling and discussion, she tells you that she would like to use the implant method. However, this method is not currently being offered in your clinic 6) What do you do? a) Refer her to another clinic that provides this method b) Tell her that she should choose another option c) End the consultation Baseline Report 299 d) Other (specify) _______________________________________________ She tells you that she does not want anyone to know that she is using contraception and asks that you keep it confidential. 7) What do you do? a) Reassure her that you will not tell anyone. b) Encourage her to tell her husband. c) Encourage her to tell other people. d) Refuse to provide a method until she informs her husband. e) Other (specify) _______________________________________________ 300 The Impact of USAID’s Integrated Health Program in the DRC IHP EVALUATION BUREAU CENTRAL DE LA ZONE DE SANTE (BCZS) HMIS SURVEY SECTION 12. HEALTH INFORMATION SYSTEM IDENTIFY THE PERSON PRIMARILY RESPONSIBLE FOR ENTERING DATA INTO THE DHIS2 SYSTEM TO ANSWER THIS SECTION Now I would like to ask you questions related to health information, specifically the DHIS2. NO QUESTION RESPONSE SKIP/ INSTRUCTIONS 4 How many facilities are currently required to submit data to the BCZS? IF UKNOWN, ENTER 999. |__ | __ |__| DON’T KNOW If 0,  8 5 In the last month, how many of the [repeat number given above] facilities submitted data using the harmonized reporting tool or any other form? [SHOW THE RESPONDENT A PHOTO OF THE CURRENT TOOL] ALL SOME NONE DON’T KNOW 6 In the last month, how many facilities submitted data using the harmonized reporting tool? [SHOW THE RESPONDENT A PHOTO OF THE CURRENT TOOL] ALL SOME NONE DON’T KNOW 7 Can I see the facility reports from the last month? YES, ALL OBSERVED YES, SOME OBSERVED NOT SEEN 8 In the last month, how did this office submit its weekly Maladie à Potentiel Epidémique (MAPEPI) to the DPS? Hand deliver to DPS DPS picks up Phone call Text message Radio communication IDENTIFY THE HIGHEST-RANKING PERSON. EXPLAIN THAT SOME QUESTIONS MAY HAVE BEEN ASKED IN EARLIER INTERVIEWS WITH USAID IHP REPRESENTATIVES AND THAT YOU APPRECIATE THEIR TIME AND PATIENCE. SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1 Record the province 2 Record the health zone 3 Enter your data collector ID number Baseline Report 301 Email from office Email from cyber café or other off-site location Electronically via DHIS2 OTHER ____________________ 9 To the best of your knowledge, the last time you encountered a MAPEPI case, how much time passed between when you were made aware of the MAPEPI event and when you were able to report to the DPS? DO NOT READ ANSWERS BUT CHECK THE ONE CLOSEST TO THE ONE GIVEN. IMMEDIATELY WITIHIN 24 HOURS THE NEXT DAY > 2 DAYS DON’T KNOW 10 What are some of the reasons why an immediate or weekly MAPEPI report may not be submitted on time? SEVERAL RESPONSES POSSIBLE. CHECK ALL THAT ARE MENTIONED DID NOT HAVE THE CORRECT FORM NO SUPERVISION VISIT LACK OF TRANSPORTATION LACK OF INTERNET LACK OF TELEPHONES LACK OF ELECTRICITY NO TIME TO COMPLETE REPORTS. NO STAFF TO COMPLETE REPORTS NOT AWARE OF SUBMISSION DEADLINES OTHER _______________________ ____ Read aloud: Every BCZ office may do things differently. I am going to list a few tasks that might be performed using DHIS2 data. Please tell me whether you have done any of the following in the past 6 months: 11 Generated a report with results for one or more facilities? YES NO DON’T KNOW 12 Generated a summary report for the health zone? YES NO DON’T KNOW 13 Compared results with provincial/national targets? YES NO DON’T KNOW 14 Compared results across different service domains? YES NO 302 The Impact of USAID’s Integrated Health Program in the DRC DON’T KNOW 15 Compared results over time? YES NO DON’T KNOW 16 Do you have a copy of the DHIS2 manual? YES, OBSERVED YES, NOT OBSERVED NO DON’T KNOW 17 Have you read the DHIS2 manual? YES NO DON’T KNOW If no,  19 18 How easy or difficult would you say the DHIS2 procedure manual is to understand? VERY EASY SOMEWHAT EASY SOMEWHAT DIFFICULT VERY DIFFICULT 19 THE NEXT QUESTION IS ABOUT TRAINING PARTICIPATION WHICH INCLUDES INTERNAL TRAINING PROVIDED BY MEMBERS OF THIS BCZ AS WELL AS TRAINING PROVIDED BY PROVINCIAL/ GOVERNMENT OFFICIALS, USAID IHP PARTNERS OR OTHER NGOS WITH WHICH THIS OFFICE MAY BE AFFILIATED. In the last calendar year (2018), did you attend a training session on health information management (e.g. DHIS2, SNIS, MAPEPI)? YES NO If no,  22 20 If yes, who administered the training [select all that apply]? Internal/BCZ office Province/Government USAID/ IHP Other (specify) _____________________ 21 What topics did this training cover [select all that apply]? Completeness calculations and reporting Timeliness calculations and reporting Exhaustivity calculations and reporting Validation rules Abhorrent data Baseline Report 303 Correcting data Other (specify) ____________________ 22 Does the BCZS have written guidelines for data entry/compilation and data quality review and control? YES NO DON’T KNOW 23 In the last month, how many feedback reports on data quality and performance has this office sent to health facilities? IF UNKNOWN, ENTER 999. |__|__|__| If 0,  30 24 Can I see the reports? SELECT A REPORT AT RANDOM AND RECORD THE FOLLOWING OBSERVATIONS. YES, OBSERVED NO, NOT OBSERVED If no,  30 Was any of the following feedback included in the report? 25 Accuracy of data YES NO DON’T KNOW 26 Completeness of data YES NO DON’T KNOW 27 Timeliness of data YES NO DON’T KNOW 28 Positive feedback YES NO DON’T KNOW 29 Any other feedback? YES _______________________ __ (SPECIFY) NO DON’T KNOW 30 Since October 2018, how many feedback reports on data quality and performance has this office received from DPS? IF UNKNOWN, ENTER 999. |__|__|__| If 0,  37 31 Can I see the reports? SELECT A REPORT AT RANDOM AND RECORD THE FOLLOWING OBSERVATIONS. YES, OBSERVED NO, NOT OBSERVED If no,  37 Was any of the following feedback included in the report? 32 Accuracy of data YES 304 The Impact of USAID’s Integrated Health Program in the DRC NO DON’T KNOW 33 Completeness of data YES NO DON’T KNOW 34 Timeliness of data YES NO DON’T KNOW 35 Positive feedback YES NO DON’T KNOW 36 Any other feedback? YES _______________________ __ (SPECIFY) NO DON’T KNOW 37 Does this office have a map of the health zone and if so, what type of map is it? YES, COMPUTER GENERATED YES, HAND-DRAWN NO MAP DON’T KNOW 38 Does the office display a summary of demographic information for the health zone (i.e. population numbers for each health area)? YES NO DON’T KNOW 39 Does the office display more specific demographic information (such as population by target groups)? YES NO DON’T KNOW What kinds of actions have been taken based on DHIS2 data? 40 Review strategy by examining service performance target and actual performance from month to month YES NO DON’T KNOW 41 Review facility personnel responsibilities by comparing service targets and actual performance from month to month YES NO DON’T KNOW 42 Mobilization/shifting of resources based on comparison between services YES NO Baseline Report 305 DON’T KNOW 43 Advocacy for more resources by showing gaps in ability to meet targets YES NO DON’T KNOW Now I would like to ask you some questions related to meetings and decisions based on the DHIS2 data. 44 How frequently does this office have routine meetings where the DHIS2 or facility reported data is discussed? This could be a separate meeting or in the routine managerial or administrative meetings. MONTHLY OR MORE OFTEN EVERY 2-3 MONTHS EVERY 4-6 MONTHS LESS THAN EVERY 6 MONTHS OR IRREGULARLY DON’T KNOW 45 Is an official record of management meetings maintained? YES NO DON’T KNOW 46 Have the DHIS2 or facility data been used to make decisions? YES NO DON’T KNOW 47 Has any follow-up action taken place regarding the decisions made during the previous meetings? YES NO DON’T KNOW 306 The Impact of USAID’s Integrated Health Program in the DRC IHP EVALUATION BUREAU CENTRAL DE LA ZONE DE SANTE (BCZS) SURVEY IDENTIFY THE HIGHEST-RANKING PERSON. EXPLAIN THAT SOME QUESTIONS MAY HAVE BEEN ASKED IN EARLIER INTERVIEWS WITH USAID IHP REPRESENTATIVES AND THAT YOU APPRECIATE THEIR TIME AND PATIENCE. SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1 Record the province 2 Record the health zone 3 Enter your data collector ID number 4 Enter the facility ID number 5 Identify the highest-ranking person. Explain that some questions may have been asked in earlier interviews with USAID IHP representatives and that you appreciate their time and patience. 6 Have you read him/her the consent script? |_| No |_| Yes If no,  7 7 If no, why? 8 Did the respondent agree? |_| No |_| Yes If no,  9 9 If no, what was the reason? |_| No office members present at time of visit |_| Office members absent for a long period of time |_| Deferred |_| Refused |_| Office vacant or not an address |_| Office destroyed |_| Office not found |_| Other (specify) _________ Go to 10 & 11 and end survey 10 Take a photograph of the front of the facility 11 Record the GPS coordinates of the health zone office Baseline Report 307 SECTION 2. BASIC INFORMATION First, I would like to ask you some general questions about you. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 12 Record the sex of the participant MALE FEMALE 01 02 13 Are you the head of this health zone office? ☐ Yes ☐ No 14 What is your position in this office? HEALTH ZONE CHIEF MEDICAL OFFICER MEDICAL DIRECTOR OF HGR DIRECTOR OF NURSING HEALTH ZONE ADMINISTRATOR/MANAGER HEALTH ZONE NURSE SUPERVISOR OF PRIMARY HEALTH CARE PHARMACIST SANITATION TECHNICIAN COMMUNITY HEALTH ANIMATOR NUTRITIONIST OTHER__________________ OTHER__________________ OTHER__________________ 01 02 03 04 05 06 07 08 09 10 11 12 13 15 What is the highest level of education you have received? NO SCHOOLING PRIMARY SECONDARY UNIVERSITY 01 02 03 04 If University,  16 16 Do you have a master’s degree? ☐ Yes ☐ No If Yes,  17 17 In what subject is your master’s degree(s)? (check all that apply) PUBLIC HEALTH PUBLIC ADMINISTRATION BUSINESS ADMINISTRATION OTHER__________________ 01 02 03 04 18 How many years have you worked in this office (in any capacity)? Less than 1 year 1 2 3 4 5 6 7 8 9 10+ DON’T KNOW 00 01 02 03 04 05 06 07 08 09 10 -98 If don’t know,  20 19 For how many years have you held the position of {q14} in this office? Less than 1 year 1 2 00 01 02 308 The Impact of USAID’s Integrated Health Program in the DRC 3 4 5 6 7 8 9 10+ DON’T KNOW 03 04 05 06 07 08 09 10 -98 20 For how many years in total have you worked as a ${q11a}, whether at this office or another BCZ? Less than 1 year 1 2 3 4 5 6 7 8 9 10+ DON’T KNOW 00 01 02 03 04 05 06 07 08 09 10 -98 TRAINING The next set of questions is about the training participation and includes internal training provided by members of this BCZ as well as training provided by provincial/government officials, USAID/IHP partners or other NGOs with which this office may be affiliated. In the last calendar year (2018), have you or anyone in this office attended training on any of the following subjects as part of your position in the health zone office? 21 Prenatal consultations? ☐ Yes ☐ No If no,  24 22 Did you attend this training? ☐ Yes ☐ No If no,  24 23 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 24 Integrated Management of Childhood Illness? ☐ Yes ☐ No If no,  27 25 Did you attend this training? ☐ Yes ☐ No If no,  27 26 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 27 Malaria (children under 5)? ☐ Yes ☐ No If no,  30 Baseline Report 309 28 Did you attend this training? ☐ Yes ☐ No If no,  30 29 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 30 Long-lasting insecticidal net (LLIN) distribution? ☐ Yes ☐ No If no,  33 31 Did you attend this training? ☐ Yes ☐ No If no,  33 32 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 33 Family planning? ☐ Yes ☐ No If no,  36 34 Did you attend this training? ☐ Yes ☐ No If no,  36 35 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 36 Reach Every District (zone) training? ☐ Yes ☐ No If no,  39 37 Did you attend this training? ☐ Yes ☐ No If no,  39 38 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 39 Vaccine Safety Training? ☐ Yes ☐ No If no,  42 40 Did you attend this training? ☐ Yes ☐ No If no,  42 41 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 42 Administrative and human resource management? ☐ Yes ☐ No If no,  45 43 Did you attend this training? ☐ Yes ☐ No If no,  45 44 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 45 Financial management? ☐ Yes ☐ No If no,  48 46 Did you attend this training? ☐ Yes ☐ No If no,  48 310 The Impact of USAID’s Integrated Health Program in the DRC 47 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 48 Management ethics and patient confidentiality? ☐ Yes ☐ No If no,  51 49 Did you attend this training? ☐ Yes ☐ No If no,  51 50 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 51 Training of health care providers? ☐ Yes ☐ No If no,  54 52 Did you attend this training? ☐ Yes ☐ No If no,  54 53 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 54 Capacity-building of CODESAs? ☐ Yes ☐ No If no,  57 55 Did you attend this training? ☐ Yes ☐ No If no,  57 56 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 57 Health information management (e.g. DHIS2, SNIS, MAPEPI)? ☐ Yes ☐ No If no,  61 58 Did you attend this training? ☐ Yes ☐ No If no,  61 59 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 60 What topics did this health information management training cover? ☐ Completeness calculations and reporting ☐ Timeliness calculations and reporting ☐ Exhaustivity calculations and reporting ☐ Validations rules ☐ Aberrant data ☐ Correcting data ☐ Other (specify)____________________ 61 Community scorecard processes? ☐ Yes ☐ No If no,  64 62 Did you attend this training? ☐ Yes ☐ No If no,  64 63 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ Baseline Report 311 64 Material resource management (medicines, supplies, equipment)? ☐ Yes ☐ No If no,  67 65 Did you attend this training? ☐ Yes ☐ No If no,  67 66 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 67 Gender issues and/or the gender transformative approach? ☐ Yes ☐ No If no,  70 68 Did you attend this training? ☐ Yes ☐ No If no,  70 69 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 70 Supportive supervision? ☐ Yes ☐ No If no,  73 71 Did you attend this training? ☐ Yes ☐ No If no,  73 72 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 73 Team management? ☐ Yes ☐ No If no,  76 74 Did you attend this training? ☐ Yes ☐ No If no,  76 75 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 76 Planning, monitoring and evaluation? ☐ Yes ☐ No If no,  79 77 Did you attend this training? ☐ Yes ☐ No If no,  79 78 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 79 Disease prevention, health promotion and re￾adaptation care? ☐ Yes ☐ No If no,  82 80 Did you attend this training? ☐ Yes ☐ No If no,  82 81 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 312 The Impact of USAID’s Integrated Health Program in the DRC 82 Management of outbreaks, emergencies and disasters? ☐ Yes ☐ No If no,  85 83 Did you attend this training? ☐ Yes ☐ No If no,  85 84 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 85 Health research? ☐ Yes ☐ No If no,  88 86 Did you attend this training? ☐ Yes ☐ No If no,  88 87 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 88 Data analysis? ☐ Yes ☐ No If no,  91 89 Did you attend this training? ☐ Yes ☐ No If no,  91 90 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ 91 Use of data for evidence-based decision￾making? ☐ Yes ☐ No If no,  94 92 Did you attend this training? ☐ Yes ☐ No If no,  94 93 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Province/Government ☐ USAID/IHP ☐ Other: ___ ☐ Other: __ ☐ Other: __ iHuman Resources Information System (iHRIS) Training 94 Did you or anyone else in this office participate in a training on Human Resources Management using the iHuman Resources Information System (iHRIS)? ☐ Yes ☐ No ☐ Don’t know If no,  101 95 How long ago was that training? ☐ 0-3 months ago ☐ 4-6 months ago ☐ 7-12 months ago ☐ More than 1 year ago 96 Sometimes participants cannot attend the entire training workshop. How much of that workshop did you attend? ☐ Less than half ☐ About half of it ☐ All or most of it If N/A,  101 Baseline Report 313 SECTION 3. BASIC INFRASTRUCTURE Now I would like to ask you some questions related to the basic infrastructure of this office. If the answer to the question is yes, please show me the room/equipment, and where applicable, verify the functionality. NO Questions and filters YES SKIP 101 RECORD OBSERVATION OF THE MAIN MATERIAL OF THE FLOOR NATURAL FLOOR EARTH/SAND DUNG RUDIMENTARY FLOOR WOOD PLANKS PALM/BAMBOO FINISHED FLOOR PARQUET OR POLISHED WOOD VINYL OR ASPHALT STRIPS 11 12 21 22 31 32 ☐ Not applicable 97 It can be difficult to remember all of the material presented during training. How much do you remember of what you learned in the training on Human Resources Management using the iHRIS? ☐ Less than half ☐ About half of it ☐ All or most of it ☐ Not applicable 98 How often do you use the material presented during that workshop in your day to day work? ☐ Never ☐ Rarely ☐ Sometimes ☐ Often ☐ Always ☐ Not applicable 99 What language was the training conducted in? ☐ Swahili ☐ French ☐ Lingala ☐ Tshiluba ☐ English ☐ Other _________________________ 100 Did anyone attending the training report that the language of the training created challenges with understanding the material? ☐ Yes ☐ No 314 The Impact of USAID’s Integrated Health Program in the DRC CERAMIC TILES CEMENT CARPET OTHER ________________________ (SPECIFY) 33 34 35 96 102 RECORD OBSERVATION OF THE MAIN MATERIAL OF THE ROOF NATURAL ROOFLING NO ROOF THATCH/PALM LEAF SOD RUDIMENTARY ROOFING MAT PALM/BAMBOO WOOD PLANKS CARDBOARD FINISHED ROOFING METAL WOOD CALAMINE/CEMENT FIBRE CERAMIC TILES CEMENT OTHER ________________________ (SPECIFY) 11 12 13 21 22 23 24 31 32 33 34 35 96 103 RECORD OBSERVATION OF THE MAIN MATERIAL OF THE EXTERIOR WALLS NATURAL WALLS NO WALLS BAMBOO/CANE/PALM/ TRUNK DIRT RUDIMENTARY WALLS BAMBOO WITH MUD STONE WITH MUD UNCOVERED ADOBE PLYWOOD CARDBOARD REUSED WOOD FINISHED WALLS CEMENT STONE WITH LIME/CEMENT BRICKS CEMENT BLOCKS COVERED ADOBE WOOD PLANKS 11 12 13 21 22 23 24 25 26 31 32 33 34 35 36 Baseline Report 315 OTHER ________________________ (SPECIFY) 96 104 Does this office have cell phone reception? RECORD OBSERVATION ☐ Yes ☐ No If no,  107 105 How would you rate the ease of connectivity to the nearest health centre? VERY EASY SOMEWHAT EASY SOMEWHAT DIFFICULT VERY DIFFICULT 01 02 03 04 106 How would you rate the ease of connectivity to the furthest health centre? VERY EASY SOMEWHAT EASY SOMEWHAT DIFFICULT VERY DIFFICULT 01 02 03 04 107 Does this office have electricity? RECORD OBSERVATION ☐ Yes ☐ No If no,  113 108 What is the main source of electricity? MAIN POWER SOLAR POWER GENERATOR OTHER (SPECIFY) 01 02 03 96 109 On a typical 8-hour work day, how many hours is electricity available at this office? [Provide drop-down list with options ‘0’ through ‘8’] |__|__|__| 110 Is electricity functioning now? (CHECK TO SEE IF ELECTRICITY CAN BE TURNED ON.) RECORD OBSERVATION ☐ Yes ☐ No 111 Are there power cuts (excluding electricity supplied by a generator backup) during the hours when the facility is open? ☐ Yes ☐ No If no,  113 112 What is the average duration of power cuts? |__|__|__| Minutes |__|__|__| Hours |__|__|__| Days 113 Does this office have a generator for electricity? This may be a back-up or stand-by generator. RECORD OBSERVATION ☐ Yes ☐ No If no,  115 114 Is the generator functional and is there fuel today? RECORD OBSERVATION ☐ Yes ☐ No In the last month, how often have power cuts interfered with: 115 Cold chain maintenance? ☐ ALWAYS ☐ OFTEN ☐ SOMETIMES ☐ RARELY ☐ NEVER Reference 111 and ask if yes; otherwise,  120 316 The Impact of USAID’s Integrated Health Program in the DRC 116 DHIS2 reporting? ☐ ALWAYS ☐ OFTEN ☐ SOMETIMES ☐ RARELY ☐ NEVER 117 Other: ______________________ ☐ ALWAYS ☐ OFTEN ☐ SOMETIMES ☐ RARELY ☐ NEVER 118 Other: ______________________ ☐ ALWAYS ☐ OFTEN ☐ SOMETIMES ☐ RARELY ☐ NEVER 119 Other: ______________________ ☐ ALWAYS ☐ OFTEN ☐ SOMETIMES ☐ RARELY ☐ NEVER 120 Is there a functional solar panel? RECORD OBSERVATION ☐ Yes ☐ No If no,  122 121 Is there a functional battery for solar panel? RECORD OBSERVATION ☐ Yes ☐ No 122 Does this office have a radio? RECORD OBSERVATION ☐ Yes ☐ No 123 Does this office have a calculator? RECORD OBSERVATION ☐ Yes ☐ No 124 How many functional computers are there at the office and of which type? RECORD OBSERVATION IF THE RESPONDENT DOESN’T KNOW THE NUMBER FOR A TYPE OF COMPUTER, WRITE “99” IN THE BOX. BCZ OWNED (NOT PERSONAL) DESKTOP |__|__| BCZ OWNED (NOT PERSONAL) LAPTOP |__|__| PERSONAL LAPTOP |__|__| 125 How many functional printers are there at the office? RECORD OBSERVATION |__|__| 126 In general, does this office have access to the internet? ☐ Yes ☐ No 127 Do you have the following sources of internet? Check all that apply 128 Modem using commercial telephone network (VODACOM, AIRTEL, etc.) ☐ Yes ☐ No If yes,  129 Baseline Report 317 129 Is it functioning? RECORD OBSERVATION ☐ Yes ☐ No 130 V-Sat antenna? ☐ Yes ☐ No If yes,  131 131 Is it functioning? RECORD OBSERVATION ☐ Yes ☐ No 132 Personal mobile/wifi hotspot? ☐ Yes ☐ No If yes,  133 133 Is it functioning? RECORD OBSERVATION ☐ Yes ☐ No 134 Other source of internet? SPECIFY__________________ ☐ Yes ☐ No 135 Was internet connectivity provided by USAID/IHP? ☐ Yes ☐ No 136 How common is it for people at this office to rely on their own personal mobile hotspot? ☐ ALWAYS ☐ OFTEN ☐ SOMETIMES ☐ RARELY ☐ NEVER 137 In a typical 8-hour work day, how many hours does the internet at the office work, not including personal mobile hotspots? [Provide drop-down list with options ‘0’ through ‘8’] |__|__| 138 Can you log into the DHIS2 system for me so that I can see whether it is working? Check all that apply. ☐ YES, OBSERVED ☐ NO, NO WORKING COMPUTER ☐ NO, NO ELECTRICITY ☐ NO, NO INTERNET ☐ NO, SLOW INTERNET ☐ NO, FORGOT USERNAME ☐ NO, FORGOT PASSWORD ☐ NO, REFUSED ☐ NO, OTHER REASON: ________________________ (SPECIFY) 318 The Impact of USAID’s Integrated Health Program in the DRC SECTION 4. GENERAL STAFFING Now I have some questions about staffing for this BCZ. Please tell me how many people currently work in the following positions as members of this health zone management team and whether they are male or female. Please include all persons who work in this BCZ, whether in a full-time, part-time or volunteer capacity. USE THIS INFORMATION TO COMPLETE COLUMNS a, b AND c. IF THE INFORMATION IS UNKNOWN, MARK COLUMN d AND LEAVE COLUMNS a, b AND c BLANK. ASK TO SEE THE STAFF ROSTER. FOR EACH QUALIFICATION, RECORD THE NUMBER OF PEOPLE WHO ARE ACTUALLY PRESENT THE DAY OF THE INTERVIEW. USE THIS INFORMATION TO COMPLETE COLUMN e. 139 Does this BCZ currently employ a Health Zone Chief Medical Officer? ☐ Yes ☐ No ☐ Don’t know If yes,  140 140 Is the MCZ male or female? ☐ Male ☐ Female 141 Does this health zone currently employ a Medical Director of the HGR? ☐ Yes ☐ No ☐ Don’t know If yes,  142 142 Is the physician director male or female? ☐ Male ☐ Female 143 Does this health zone currently employ a Director of Nursing? ☐ Yes ☐ No ☐ Don’t know If yes,  144 144 Is the Director of Nursing male or female? ☐ Male ☐ Female 145 Does this health zone currently employ a Health Zone Administrator/Manager? ☐ Yes ☐ No ☐ Don’t know If yes,  146 146 Health Zone Administrator/Manager No. staff |__|__|__| No. female |__|__|__| No. male |__|__|__| 147 Does this health zone currently employ a Health zone Nurse Supervisor of Primary Health Care ☐ Yes ☐ No ☐ Don’t know If yes,  148 148 Health zone Nurse Supervisor of Primary Health Care No. staff |__|__|__| No. female |__|__|__| No. male |__|__|__| 149 Does this health zone currently employ a Pharmacist ☐ Yes ☐ No ☐ Don’t know If yes,  150 150 Pharmacist No. staff |__|__|__| No. female |__|__|__| No. male |__|__|__| 151 Does this health zone currently employ a Sanitation Technician ☐ Yes ☐ No ☐ Don’t know If yes,  152 152 Sanitation Technician No. staff |__|__|__| No. female |__|__|__| No. male |__|__|__| 153 Does this health zone currently employ a Community Health Animator ☐ Yes ☐ No ☐ Don’t know If yes,  154 154 Community Health Animator No. staff |__|__|__| No. female |__|__|__| No. male |__|__|__| 155 Does this health zone currently employ a Nutritionist ☐ Yes ☐ No ☐ Don’t know If yes,  156 156 Nutritionist No. staff |__|__|__| No. female |__|__|__| No. male |__|__|__| 157 Does this office have a designated person to enter/compile DHIS2 data reports from health facilities? ☐ Yes ☐ No ☐ Don’t know 158 Does the district have a designated person to review and control the quality of data entry/compilation? ☐ Yes ☐ No ☐ Don’t know Baseline Report 319 SECTION 6. SOURCES AND USES OF FUNDING Now I would like to ask you some questions about sources and uses of funding for this BCZ office. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS What is the total amount of operational funds (i.e. funds not passed on to another organization) received from the following sources in the last calendar year (2018)?: 159 Ministry of health/Provincial health office/Health zone office |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD 160 USAID/IHP |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD 161 Other NGO’s or FBO’s: Cordaid, Memisa, Foundation Damien, save the Children, Rescue, IRC, Caritas, Sanru, Chemonics, MCSP |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD 162 User fees |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD 163 Community financing (e.g. mutuelles) |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD 164 Other health insurance program? |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD 165 Any other sources of funding? ☐ Yes ☐ No 166 Other 1 (specify) ____________________________ |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD 167 Other 2 (specify) ____________________________ |__ __ __ __ __ __ __ __ __ __| ☐ FC ☐ USD What percentage of operational funds was spent on the following during the last calendar year (2018)? 168 Savings |__ __ __| % 169 Building/grounds improvements |__ __ __| % 170 Equipment (vehicles, computers, etc.) |__ __ __| % 171 Utilities and communication (electricity, water, phone credit, internet, etc.) |__ __ __| % 172 Medical supplies |__ __ __| % 173 Drugs |__ __ __| % 174 Transport |__ __ __| % 175 Salaries and primes |__ __ __| % 176 Training |__ __ __| % 177 Any others? ☐ Yes ☐ No 320 The Impact of USAID’s Integrated Health Program in the DRC SECTION 7. USER FEES Now I would like to ask you a few questions about user fees in this health zone’s health centers and hospitals. NO QUESTION RESPONSE SKIP/INSTRUCTIONS 180 Does this office have guidelines that facilities should follow on the user fees that should be charged to patients? IF YES, ASK TO SEE THE GUIDELINES AND TAKE A PICTURE YES, GUIDELINES OBSERVED YES, NOT OBSERVED NO GUIDELINES RECEIVED 01 02 00 181 To the best of your knowledge, where do the user fee guidelines come from? National MoH DPS BCZ/MCZ alone MCZ with mayor and community Integrated Health project Other (specify) Don't know 01 02 03 04 05 96 99 182 Are health facilities supposed to post the fee schedule where patients can see it? YES NO DON’T KNOW 01 02 98 183 Are all designated health facilities in this health zone supposed to charge the same user fee for the same service? YES NO DON’T KNOW 01 00 99 184 Does the health zone provide a different fee schedule for patients who are considered indigent? YES NO DON’T KNOW 01 00 99 185 Does this office have guidelines on the EXEMPTION OF payment for indigent patients? IF YES, ASK TO SEE THE GUIDELINES YES, GUIDELINES OBSERVED YES, NOT OBSERVED NO GUIDELINES RECEIVED 01 02 00 186 To the best of your knowledge, where do the exemption guidelines come from? National MoH DPS BCZ/MCZ alone MCZ with mayor and community Integrated Health project Other (specify) Don't know 01 02 03 04 05 96 99 187 Who are considered indigent? Meaning, what are the criteria for the user fee exemption? CIRCLE ALL THAT ARE MENTIONED ELDERLY 01 Reference 184 and ask if yes. ORPHANED 02 WIDOW WITHOUT A SOURCE OF INCOME 03 PHYSICALLY HANDICAPED WITHOUT A SOURCE OF INCOME 04 178 Other 1 (specify) ____________________________ |__ __ __| % 179 Other 2 (specify) ____________________________ |__ __ __| % Baseline Report 321 REFUGEE/INTERNALLY DISPLACED PERSON NOT YET INTEGRATED INTO THE COMMUNITY 05 OTHER _____________________ 96 DON’T KNOW 99 188 Are health facilities supposed to have lists of people who are considered indigent? YES NO DON’T KNOW 01 02 99 Reference 184 and ask if yes. If 188 is no,  192 189 Who is supposed to participate in the process of identifying the individuals on this list? CIRCLE ALL THAT ARE MENTIONED The respondent Health centre staff CODESA members Other community leaders Other BCZ staff DPS staff Dirigeants National MoH Other ____________________ Don't know 01 02 03 04 05 06 07 08 -96 -98 190 Are CODESAs supposed to approve these lists? YES NO DON’T KNOW 01 02 99 191 How often are these lists supposed to be updated? DO NOT READ ANSWERS BUT CHECK THE ONE CLOSEST TO THE RESPONSE GIVEN Every 1-5 months Every 6 months Every 7-11 months Once a year or less Never Other _____________________ Don't know 01 02 03 04 05 -96 -98 192 Do health facilities in this health zone always require payment before providing labour and delivery services? YES NO DON’T KNOW 01 02 98 193 Do health facilities in this health zone always require payment before treatment of emergency cases? YES NO DON’T KNOW 01 02 98 194 Generally speaking, how do health facilities handle cases in which a patient cannot pay for services? CHECK ALL THAT APPLY. No services are given The patient can pay in-kind The patient can give a guarantee The patient is treated for free/reduced cost the patient is not discharged until they can pay The patient is refused services in the future Nothing, no recourse Other (specify) Don't know 00 01 02 03 04 05 06 -96 -98 322 The Impact of USAID’s Integrated Health Program in the DRC SECTION 8. COMMUNITY FUNDING INITIATIVES NO QUESTION RESPONSE SKIP/ INSTRUCTIONS 195 Are there any mutuelles operating in this health zone? YES……………………………………. NO…………………………………….. DON’T KNOW……………………. 01 00 99 If no,  201 196 How many mutuelles are operating in this health zone? |____|____|____| 197 Does this office keep a list of all mutuelles? YES……………………………………. NO…………………………………….. DON’T KNOW……………………. 01 00 99 198 Does this office keep a list of the members of ALL mutuelles? YES……………………………………. NO…………………………………….. DON’T KNOW……………………. 01 00 99 199 Do health areas in your health zone participate in these mutuelles? YES NO DON’T KNOW 01 00 99 If no,  201 200 How many health areas in this health zone participate in these mutuelles? A SMALL NUMBER OF HEALTH AREAS PARTICIPATE. HALF OF ALL HEATH AREAS PARTICIPATE ALMOST ALL HEALTH AREAS PARTICIPATE ALL HEALTH AREAS PARTICIPATE 01 02 03 04 201 Do health facilities require permission from the BCZ in order to offer fee reductions to members of the mutuelles? YES NO DON’T KNOW 01 00 99 If no,  204 202 How many of the health facilities in this health zone have been given permission to offer fee reductions to members of mutuelles? ALL HEALTH FACILITIES SOME HEALTH FACILITIES NO HEALTH FACILITIES DON’T KNOW 01 02 03 99 203 What services are covered by this fee reduction? CIRCLE ALL THAT ARE MENTIONED. GENERAL OUTPATIENT 01 IMMUNIZATION FOR CHILDREN 02 ANTENATAL CARE 03 NORMAL DELIVERIES 04 DELIVERIES BY CAESAREAN SECTION 05 POSTNATAL CARE 06 FAMILY PLANNING COUNSELING 07 TUBERCULOSIS TREATMENT 08 Baseline Report 323 STIs TREATMENT 09 COMMUNITY AND OUTREACH SERVICES 10 GENERAL INPATIENT MEDICAL SERVICES 11 GENERAL INPATIENT SURGICAL SERVICES ALL SERVICES OTHER _________________ (SPECIFY) DON’T KNOW 12 13 96 99 IMMUNIZATION FOR CHILDREN 02 ANTENATAL CARE 03 NORMAL DELIVERIES 04 DELIVERIES BY CAESAREAN SECTION 05 POSTNATAL CARE 06 FAMILY PLANNING COUNSELING 07 TUBERCULOSIS TREATMENT 08 STIs TREATMENT 09 COMMUNITY AND OUTREACH SERVICES 10 GENERAL INPATIENT MEDICAL SERVICES 11 GENERAL INPATIENT SURGICAL SERVICES ALL SERVICES OTHER ________________ (SPECIFY) DON’T KNOW 12 13 96 99 204 Do you or someone from the BCZS conduct supervision visits specifically for health areas participating in the mutelles? YES NO DON’T KNOW 01 00 99 Reference 195 and ask if yes. If 204 is no,  207 205 How often are supervision visits supposed to be made per site? MONTHLY OR MORE OFTEN EVERY 2-3 MONTHS EVERY 4-6 MONTHS LESS THAN EVERY 6 MONTHS OR IRREGULARLY DON’T KNOW 01 02 03 04 99 324 The Impact of USAID’s Integrated Health Program in the DRC 206 On average, how many visits were made in the last 6 months for each site? IF UNKNOWN, ENTER 99. |__|__| SECTION 9. SUPPORT FROM USAID/IHP NO QUESTION RESPONSE SKIP/ INSTRUCTIONS Capacity building 207 Has this office ever completed a Participatory Institutional Capacity Assessment and Learning (PICAL) assessment, facilitated by someone from USAID, IHP or Abt Associates? [ ] Yes [ ] No (SKIP TO xxx) [ ] Don’t know If no,  212 208 For the last PICAL assessment, which members of the health zone management team participated? Check all that apply HEALTH ZONE CHIEF MEDICAL OFFICER MEDICAL DIRECTOR OF HGR DIRECTOR OF NURSING HEALTH ZONE ADMINISTRATOR/MANAGER HEALTH ZONE NURSE SUPERVISOR OF PRIMARY HEALTH CARE PHARMACIST SANITATION TECHNICIAN COMMUNITY HEALTH ANIMATOR NUTRITIONIST 209 How long ago was most recent PICAL assessment? [ ] 0-3 months ago [ ] 4-6 months ago [ ] 7-12 months ago [ ] More than 1 year ago [ ] Don’t know If don’t know,  211 210 Did the office receive its “PICAL score” as a result of that assessment? [ ] Yes [ ] No (SKIP TO xxx) [ ] Don’t know If yes,  211 211 What was the score? ____________________________ Baseline Report 325 SECTION 10: MANAGEMENT AND SUPERVISION LEADERSHIP In this part of the questionnaire, I would like to know what you would do in certain situations regarding the health zone office. I will read you a series of scenarios. For each scenario, I will read 4 possible responses that you might have. Please select the response that most closely matches what you would do in this specific situation. You can only select one response for each scenario. There are no correct or incorrect answers - we just want to know how you would approach each situation. INTERVIEWER: EACH RESPONSE IS CODED BETWEEN 1 AND 4. RECORD APPROPRIATE CODE ACCORDING TO RESPONDENT'S RESPONSE. NO SCENARIOS 1 2 3 4 RECORD RESPONSE (RANGE 1-4) 212 Scenario 1: The performance of your staff is improving. You stress their responsibilities and standards. You take no particular additional action. You give positive feedback and make staff feel involved in the achievements. You emphasize the importance of deadlines and tasks. 213 Scenario 2: Members of your staff have been unable to solve a problem over the past month, though they have been trying to address it. You call a meeting and together try to solve the problem. You let your staff address this problem on their own. You give them direction and instructions on how to solve the problem. You encourage the group to solve the problem on their own, and you are available when needed to discuss. 214 Scenario 3: You are considering a major change in how things are done in the office. You collaborate with your staff to develop the needed changes. You announce your vision for the changes and implement a clear plan. You ask your staff to develop and implement their own plan for change. You consult with your staff, but direct the changes yourself. 215 Scenario 4: The performance of your staff has been falling in recent months. You ask your staff to rethink their direction and goals and come up with a plan together. You ask for suggestions from your staff on what to do, and you formulate a specific plan to meet objectives. You redefine goals clearly and supervise whether these are being met closely. You allow your staff freedom to set their own goals and do not push them. 216 Scenario 5: Your staff are no longer working together as an effective team. You discuss ideas as a group and identify how to work better together. You let your staff work out their issues on their own. You act quickly and decisively to get the team back on track. You make yourself available to discuss any issues and support your team to work out their own problems. AUTONOMY In this part of the questionnaire I would like to ask you some questions regarding how work is organized and how decisions are made in this health zone office. All answers are confidential. 326 The Impact of USAID’s Integrated Health Program in the DRC I am now going to read you a series of statements about decision-making and authority in this facility. Please tell me whether you agree, are neutral, or disagree. RESPONSE CODE RECORD RESPONSE AGREE 1 NEUTRAL 2 DISAGREE 3 NOT APPLICABLE (N/A) 4 217 I am able to allocate my HZ budget according to how it is needed. There is enough flexibility in my budget. 218 I am able to assign tasks and activities to staff as needed to achieve the outcomes I want in the HZ office. There is enough flexibility to use staff to address needs. 219 The District Health Management Team supports my decisions and actions for doing a better job in my office. 220 I have choice over who I allocate for what tasks. 221 I have choice over what services are provided in the facilities within this HZ. 222 I have enough authority to obtain the resources I need (drugs, supplies, funding) to meet the needs of my facilities within this HZ. 223 The policies and procedures for doing things are clear to me. 224 The policies and procedures for doing things are useful tools for the challenges I face in providing services and reporting on activities. 225 The District Health Management Team provides adequate feedback to me about my job and the performance of my facility. INTERNAL MANAGEMENT NO QUESTION RESPONSE SKIP/ INSTRUCTIONS 226 Does this office have routine meetings for reviewing managerial or administrative matters? YES NO DON’T KNOW 01 02 99 If no,  241 227 In 2018, how often did meetings to discuss the managerial and administrative matters take place? MONTHLY OR MORE OFTEN QUARTERLY TWICE A YEAR ANNUALLY IRREGULARLY NEVER DON’T KNOW 01 02 03 04 05 06 99 228 Are meeting minutes available? YES NO DON’T KNOW 01 00 99 If no,  240 229 Can I see the meeting minutes from 2018 until now? OBSERVED NOT OBSERVED 01 00 If no,  240 230 RECORD THE NUMBER OF MEETINGS THAT HAVE BEEN HELD SINCE OCTOBER 2018. |__|__|__| Which members of the BCZ attend these meetings and how often? Who attends the routine meetings? Baseline Report 327 231 Health Zone Chief Medical officer 75-100% 50-74% 25-49% 1-24% 01 02 03 04 232 Medical Director of the HGR 75-100% 50-74% 25-49% 1-24% 01 02 03 04 233 Director of Nursing 75-100% 50-74% 25-49% 1-24% 01 02 03 04 234 Health zone Administrator/Manager 75-100% 50-74% 25-49% 1-24% 01 02 03 04 235 Health zone Nurse Supervisor of Primary Health Care 75-100% 50-74% 25-49% 1-24% 01 02 03 04 236 Pharmacist 75-100% 50-74% 25-49% 1-24% 01 02 03 04 237 Sanitation Technician 75-100% 50-74% 25-49% 1-24% 01 02 03 04 238 Community Health Animator 75-100% 50-74% 25-49% 1-24% 01 02 03 04 239 Nutritionist 75-100% 50-74% 25-49% 1-24% 01 02 03 04 240 What types of decisions are made in these meetings? SEVERAL ANSWERS POSSIBLE. DO NOT READ THE ANSWERS BUT CHECK ALL MENTIONED SCHEDULING OF ACTIVITES IN THE HZ. ALLOCATING FINANCIAL RESOURCES. ALLOCATING TECHNICAL RESOURCES DEFINING PROGRAMMATIC PRIORITIES COORDINATING STRATEGIES WITH DONORS HUMAN RESOURCES MANAGEMENT CONFLICT RESOLUTION 01 02 03 04 05 06 07 241 Do you or does someone from this office make supervision visits to health facilities? YES NO DON’T KNOW 01 00 99 If no,  224 242 Are supervision reports available? YES NO DON’T KNOW 01 00 99 If no,  224 243 Can I see the most recent report? OBSERVED NOT OBSERVED 01 00 328 The Impact of USAID’s Integrated Health Program in the DRC EXTERNAL SUPERVISION LEVEL FROM WHICH THE SUPERVISOR CAME In 2018, how many times did the supervisor from [LEVEL] visit this office for the purpose of management or supervision of routine and non-routine activities? Do you know the date (month and year) of the most recent visit by a supervisor from the [LEVEL]? When did a supervisor from the [LEVEL] last visit? What topic was discussed at the last visit? CHECK ALL THAT APPLY After the last visit, did the supervisor send you a report based on his/her findings? a. b. c. d. e. 244 Provincial Office 0 1 2 3 4 5 6 7 8 9 10 11 12 or more Don't know YES NO (skip to d) DON’T KNOW MONTH _______ YEAR _______ DON’T KNOW QUALITY OF CARE …..01 MANAGEMENT ……...02 RECORD KEEPING …..03 DATA REPORTING …..04 DATA USE ……………....05 N/A………………………….96 DON’T KNOW………….98 YES NO DON’T KNOW 245 National-/ Central-level authorities 0 1 2 3 4 5 6 7 8 9 10 11 12 or more Don't know YES NO (skip to d) DON’T KNOW MONTH _______ YEAR _______ DON’T KNOW QUALITY OF CARE …..01 MANAGEMENT ……...02 RECORD KEEPING …..03 DATA REPORTING …..04 DATA USE ……………....05 N/A………………………….96 DON’T KNOW………….98 YES NO DON’T KNOW 246 IHP officials 0 1 2 3 4 5 6 7 8 9 YES NO (skip to d) DON’T KNOW MONTH _______ YEAR _______ DON’T KNOW QUALITY OF CARE …..01 MANAGEMENT ……...02 RECORD KEEPING …..03 DATA REPORTING …..04 DATA USE ……………....05 N/A………………………….96 DON’T KNOW………….98 YES NO DON’T KNOW Baseline Report 329 SECTION 11. STAKEHOLDER COORDINATION & COLLABORATION The next questions are about different kinds of interactions your BCZ may have with other health zone offices. NO QUESTION RESPONSE SKIP/ INSTRUCTIONS Community Engagement 247 Are there any CODESAs functioning in this area? YES NO DON’T KNOW 01 00 99 If no,  250 248 How many CODESAs are functioning in this area? If unknown, enter 999. |__|__|__| 249 How many CODESAs submitted a report to this office within the last 30 days? If unknown, enter 999. |__|__|__| 250 Are you aware of a program called the community scorecard? YES NO 01 00 If no,  254 251 Have facilities/health areas in your health zone participated in a community scorecard meeting? YES NO DON’T KNOW 01 00 99 If no,  253 252 How many health areas in this health zones participate in the community scorecard processes? A SMALL NUMBER OF HEALTH AREAS PARTICIPATE HALF OF ALL HEATH AREAS PARTICIPATE ALMOST ALL HEALTH AREAS PARTICIPATE ALL HEALTH AREAS PARTICIPATE 01 02 03 04 253 Have you received a report about the community scorecard activity from at least one health area in your health zone? YES NO DON’T KNOW 01 02 99 Communication 254 How often do you communicate with different CODESAs? [ ] One a month [ ] Every 3 months [ ] Twice a year [ ] Once a year [ ] Other: _____________________ [ ] Don’t know 255 Since October 2018 when the IHP project was implemented, would you say this BCZ had more, less, or about the same amount of communication with CODESAs? [ ] MORE [ ] LESS [ ] ABOUT THE SAME [ ] DON’T KNOW 10 11 12 or more Don't know 330 The Impact of USAID’s Integrated Health Program in the DRC 256 How many other health zone offices does this BCZ communicate with for professional reasons at least quarterly? [____] [____][____] 257 How often do you communicate with other health zone offices? [ ] One a month [ ] Every 3 months [ ] Twice a year [ ] Once a year [ ] Other: _____________________ [ ] Don’t know 258 Since October 2018 when the IHP project was implemented, would you say this BCZ had more, less, or about the same amount of communication with other health zone offices? [ ] MORE [ ] LESS [ ] ABOUT THE SAME [ ] DON’T KNOW Please tell me how often you or someone from your organization did each of the following things as part of their day-to-day work in 2018? 259 Do you or does someone from your office attend Comités de Gestion (COGE) meetings? [ ] Yes [ ] No If no, continue to next table. 260 How many times were these meetings held in 2018? [__] 01 [__] 02 [__] 03 [__] 04 [__] 05 [__] 06 [__] 07 [__] 08 [__] 09 [__] 10 [__] 11 [__] 12 or more [__] Don’t know 261 How many times have you or someone from your office contributed ideas or lessons learned at COGE meetings? [__] 01 [__] 02 [__] 03 [__] 04 [__] 05 [__] 06 [__] 07 [__] 08 [__] 09 [__] 10 [__] 11 [__] 12 or more [__] Don’t know Baseline Report 331 262 How many times have you or someone from your office served as a COGE meeting facilitator or guest speaker? [__] 01 [__] 02 [__] 03 [__] 04 [__] 05 [__] 06 [__] 07 [__] 08 [__] 09 [__] 10 [__] 11 [__] 12 or more [__] Don’t know 263 How many times have you or someone from your office worked with other HZ officials at COGE meetings to influence decision-making or development of strategies? [__] 01 [__] 02 [__] 03 [__] 04 [__] 05 [__] 06 [__] 07 [__] 08 [__] 09 [__] 10 [__] 11 [__] 12 or more [__] Don’t know 332 The Impact of USAID’s Integrated Health Program in the DRC This next section includes questions about specific local-level entities and organizations your BCZ has worked with to strengthen institutional capacity and improve health of Congolese citizens. By “work with” we mean both informally, for example through exchange of information, and formally, for example by attending the same training or workshop. The term “local-level entities” includes other health zone offices in addition to government enterprises other than the Ministry of Health (e.g. Ministry of Finance, Ministry of Education) and private sector partners. “Organizations” refers to all non-governmental institutions that have provided technical, financial or other support. (RECORD THE NUMBER corresponding to the chosen response) Over the last 6 months how many organizations and other local-level entities have you worked with to help to achieve goals related to strengthened capacity and improved health outcomes? Enter number|__|__| If none,  264 Over the last 6 months how frequently have you worked with this organization? In the last 6 months, how often did your office share or exchange information with THIS ORGANIZATION? In the last 6 months, how often did your organization jointly plan, implement or monitor an activity, event, or service with THIS ORGANIZATION? In the last 6 months, how often did your office receive technical assistance from THIS ORGANIZATION to strengthen skills in data analysis, use and reporting? In the last 6 months, how often did your office receive financial or other support from THIS ORGANIZATION to build institutional capacity? Does your office have a formal relationship with THIS ORGANIZATION, through a contract, memorandum of understanding or some other written document describing your partnership? Write the name of each organization below. 1= Never 2= Once, twice, or just a few times 3= Monthly 4= Weekly 5= Daily 1= Never 2= Once, twice, or just a few times 3= Monthly 4= Weekly 5= Daily 1= Never 2= Once, twice, or just a few times 3= Monthly 4= Weekly 5= Daily 1= Never 2= Once, twice, or just a few times 3= Monthly 4= Weekly 5= Daily 1= Never 2= Once, twice, or just a few times 3= Monthly 4= Weekly 5= Daily 1= YES 0= NO 2= DON’T KNOW Baseline Report 333 SECTION 12. HEALTH INFORMATION SYSTEM IDENTIFY THE PERSON PRIMARILY RESPONSIBLE FOR ENTERING DATA INTO THE DHIS2 SYSTEM TO ANSWER THIS SECTION Now I would like to ask you questions related to health information, specifically the DHIS2. NO QUESTION RESPONSE SKIP/ INSTRUCTIONS 264 How many facilities are currently required to submit data to the BCZS? IF UKNOWN, ENTER 999. |__ | __ |__| DON’T KNOW 265 In the last month, how many of the [repeat number given above] facilities submitted data using the harmonized reporting tool or any other form? [SHOW THE RESPONDENT A PHOTO OF THE CURRENT TOOL] ALL SOME NONE DON’T KNOW 01 02 00 99 266 In the last month, how many facilities submitted data using the harmonized reporting tool? [SHOW THE RESPONDENT A PHOTO OF THE CURRENT TOOL] ALL SOME NONE DON’T KNOW 01 02 00 99 267 Can I see the facility reports from the last month? YES, ALL OBSERVED YES, SOME OBSERVED NOT SEEN 01 02 00 268 In the last month, how did this office submit its weekly Maladie à Potentiel Epidémique (MAPEPI) to the DPS? Hand deliver to DPS DPS picks up Phone call Text message Radio communication Email from office Email from cyber café or other off-site location Electronically via DHIS2 OTHER ____________________ 01 02 03 04 05 06 07 08 96 269 To the best of your knowledge, the last time you encountered a MAPEPI case, how much time passed between when you were made aware of the MAPEPI event and when you were able to report to the DPS? DO NOT READ ANSWERS BUT CHECK THE ONE CLOSEST TO THE ONE GIVEN. IMMEDIATELY WITIHIN 24 HOURS THE NEXT DAY > 2 DAYS DON’T KNOW 01 02 03 04 98 270 What are some of the reasons why an immediate or weekly MAPEPI report may not be submitted on time? SEVERAL RESPONSES POSSIBLE. CHECK ALL THAT ARE MENTIONED DID NOT HAVE THE CORRECT FORM NO SUPERVISION VISIT LACK OF TRANSPORTATION LACK OF INTERNET LACK OF TELEPHONES LACK OF ELECTRICITY NO TIME TO COMPLETE REPORTS. NO STAFF TO COMPLETE REPORTS NOT AWARE OF SUBMISSION DEADLINES OTHER ___________________________ 01 02 03 04 05 06 07 08 09 96 334 The Impact of USAID’s Integrated Health Program in the DRC Read aloud: Every BCZ office may do things differently. I am going to list a few tasks that might be performed using DHIS2 data. Please tell me whether you have done any of the following in the past 6 months: 271 Generated a report with results for one or more facilities? YES NO DON’T KNOW 01 00 99 272 Generated a summary report for the health zone? YES NO DON’T KNOW 01 00 99 273 Compared results with provincial/national targets? YES NO DON’T KNOW 01 00 99 274 Compared results across different service domains? YES NO DON’T KNOW 01 00 99 275 Compared results over time? YES NO DON’T KNOW 01 00 99 276 Do you have a copy of the DHIS2 manual? YES, OBSERVED YES, NOT OBSERVED NO DON’T KNOW 01 02 00 99 277 Have you read the DHIS2 manual? YES NO DON’T KNOW 01 00 99 278 How easy or difficult would you say the DHIS2 procedure manual is to understand? VERY EASY SOMEWHAT EASY SOMEWHAT DIFFICULT VERY DIFFICULT 01 02 03 04 279 THE NEXT QUESTION IS ABOUT TRAINING PARTICIPATION WHICH INCLUDES INTERNAL TRAINING PROVIDED BY MEMBERS OF THIS BCZ AS WELL AS TRAINING PROVIDED BY PROVINCIAL/ GOVERNMENT OFFICIALS, USAID IHP PARTNERS OR OTHER NGOS WITH WHICH THIS OFFICE MAY BE AFFILIATED. In the last calendar year (2018), did you attend a training session on health information management (e.g. DHIS2, SNIS, MAPEPI)? YES NO 01 00 280 If yes, who administered the training [select all that apply]? Internal/BCZ office Province/Government USAID/ IHP Other (specify) _____________________ 01 02 03 96 281 What topics did this training cover [select all that apply]? Completeness calculations and reporting Timeliness calculations and reporting Exhaustivity calculations and reporting Validation rules Abhorrent data Correcting data Other (specify) ____________________ 01 02 03 04 05 06 96 Baseline Report 335 282 Does the BCZS have written guidelines for data entry/compilation and data quality review and control? YES NO DON’T KNOW 01 00 99 283 In the last month, how many feedback reports on data quality and performance has this office sent to health facilities? IF UNKNOWN, ENTER 999. |__|__|__| GO TO 151 284 Can I see the reports? SELECT A REPORT AT RANDOM AND RECORD THE FOLLOWING OBSERVATIONS. YES, OBSERVED NO, NOT OBSERVED 01 00 Was any of the following feedback included in the report? 285 Accuracy of data YES NO DON’T KNOW 01 00 99 286 Completeness of data YES NO DON’T KNOW 01 00 99 287 Timeliness of data YES NO DON’T KNOW 01 00 99 288 Positive feedback YES NO DON’T KNOW 01 00 99 289 Any other feedback? YES _________________________ (SPECIFY) NO DON’T KNOW 01 00 99 290 Since October 2018, how many feedback reports on data quality and performance has this office received from DPS? IF UNKNOWN, ENTER 999. |__|__|__| GO TO 158 291 Can I see the reports? SELECT A REPORT AT RANDOM AND RECORD THE FOLLOWING OBSERVATIONS. YES, OBSERVED NO, NOT OBSERVED 01 00 Was any of the following feedback included in the report? 292 Accuracy of data YES NO DON’T KNOW 01 00 99 293 Completeness of data YES NO DON’T KNOW 01 00 99 294 Timeliness of data YES NO DON’T KNOW 01 00 99 295 Positive feedback YES NO 01 00 336 The Impact of USAID’s Integrated Health Program in the DRC DON’T KNOW 99 296 Any other feedback? YES _________________________ (SPECIFY) NO DON’T KNOW 01 00 99 297 Does this office have a map of the health zone and if so, what type of map is it? YES, COMPUTER GENERATED YES, HAND-DRAWN NO MAP DON’T KNOW 01 02 03 99 298 Does the office display a summary of demographic information for the health zone (i.e. population numbers for each health area)? YES NO DON’T KNOW 01 00 99 299 Does the office display more specific demographic information (such as population by target groups)? YES NO DON’T KNOW 01 00 99 What kinds of actions have been taken based on DHIS2 data? 300 Review strategy by examining service performance target and actual performance from month to month YES NO DON’T KNOW 01 00 99 301 Review facility personnel responsibilities by comparing service targets and actual performance from month to month YES NO DON’T KNOW 01 00 99 302 Mobilization/shifting of resources based on comparison between services YES NO DON’T KNOW 01 00 99 303 Advocacy for more resources by showing gaps in ability to meet targets YES NO DON’T KNOW 01 00 99 Now I would like to ask you some questions related to meetings and decisions based on the DHIS2 data. 304 How frequently does this office have routine meetings where the DHIS2 or facility reported data is discussed? This could be a separate meeting or in the routine managerial or administrative meetings. MONTHLY OR MORE OFTEN EVERY 2-3 MONTHS EVERY 4-6 MONTHS LESS THAN EVERY 6 MONTHS OR IRREGULARLY DON’T KNOW 01 02 03 04 99 GO TO 169 305 Is an official record of management meetings maintained? YES NO DON’T KNOW 01 02 99 306 Have the DHIS2 or facility data been used to make decisions? YES NO DON’T KNOW 01 00 99 307 Has any follow-up action taken place regarding the decisions made during the previous meetings? YES NO DON’T KNOW 01 00 99 Baseline Report 337 Thank you for the time. Interview End Time: Hour |___|___| Minutes |___|___| 338 The Impact of USAID’s Integrated Health Program in the DRC IHP EVALUATION DIVISION PROVINCIAL DE LA SANTÉ (DPS) SURVEY IDENTIFY THE HIGHEST-RANKING PERSON ON THE DAY OF THE INTERVIEW. EXPLAIN THAT SOME QUESTIONS MAY HAVE BEEN ASKED IN EARLIER INTERVIEWS WITH USAID/IHP REPRESENTATIVES AND THAT YOU APPRECIATE THEIR TIME AND PATIENCE. SECTION 1. ORIENTATION No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 1 Record the province 2 Enter your data collector ID number 3 Enter the facility ID number 4 Identify the highest-ranking person. Explain that some questions may have been asked in earlier interviews with USAID IHP representatives and that you appreciate their time and patience. 5 Have you read him/her the consent script? |_| No |_| Yes If no,  6 6 If no, why? 7 Did the respondent agree? |_| No |_| Yes If no,  8 8 If no, what was the reason? |_| No office members present at time of visit |_| Office members absent for a long period of time |_| Deferred |_| Refused |_| Office vacant or not an address |_| Office destroyed |_| Office not found |_| Other (specify) _________ Go to 9 & 10 and end survey 9 Take a photograph of the front of the office 10 Record the GPS coordinates to six decimal places Baseline Report 339 SECTION 2. BASIC INFORMATION First, I would like to ask you some general questions about you. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS 11 Record the sex of participant ☐ Male ☐ Female 12 Are you the head of this DPS? ☐ Yes ☐ No 13 What is your position in this office? CHEF DE DIVISION SECRETARY OFFICE CHIEF OF RESOURCE MANAGEMENT OFFICE CHIEF OF TECHNICAL SUPPORT OFFICE CHIEF OF INSPECTION AND CONTROL OFFICE CHIEF OF HEALTH INFORMATION OFFICE CHIEF OF HYGIENE AND PUBLIC SANITATION OFFICE CHIEF OF HEALTH SCIENCES TRAINING OTHER _____________________ 01 02 03 04 05 06 07 08 96 14 For how many years have you held this position in this office? Less than 1 year 1 2 3 4 5 6 7 8 9 10+ Don't know 00 01 02 03 04 05 06 07 08 09 10 98 15 What is the highest level of education you have received? No schooling Primary Secondary University 01 02 03 04 If University,  16; otherwise,  18 16 Do you have a master’s degree? ☐ Yes ☐ No If no,  18 17 In what subject is your master’s degree(s)? (check all that apply) Public health Public administration Business administration Other (specify) _________________ 01 02 03 96 340 The Impact of USAID’s Integrated Health Program in the DRC 18 How many years have you worked in this office (in any capacity)? Less than 1 year 1 2 3 4 5 6 7 8 9 10+ Don't know 00 01 02 03 04 05 06 07 08 09 10 98 19 For how many years have you held the position of [refer to response in Q13] in this office? Less than 1 year 1 2 3 4 5 6 7 8 9 10+ Don't know 00 01 02 03 04 05 06 07 08 09 10 98 20 For how many years in total have you worked as a [refer to response in Q13] , whether at this office or another DPS? Less than 1 year 1 2 3 4 5 6 7 8 9 10+ Don't know 00 01 02 03 04 05 06 07 08 09 10 98 TRAINING The next set of questions is about training participation and includes internal training provided by members of this DPS as well as training provided by central/government officials, USAID/IHP partners or other NGOs with which this office may be affiliated. In the last calendar year (2018), have you or anyone in this provincial office attended training on any of the following subjects? 21 Prenatal consultations? ☐ Yes ☐ No If no,  24 22 Did you attend this training? ☐ Yes ☐ No If no,  24 23 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ Baseline Report 341 24 Integrated Management of Childhood Illness? ☐ Yes ☐ No If no,  27 25 Did you attend this training? ☐ Yes ☐ No If no,  27 26 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 27 Malaria (children under 5)? ☐ Yes ☐ No If no,  30 28 Did you attend this training? ☐ Yes ☐ No If no,  30 29 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 30 Long-lasting insecticidal net (LLIN) distribution? ☐ Yes ☐ No If no,  33 31 Did you attend this training? ☐ Yes ☐ No If no,  33 32 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 33 Family planning? ☐ Yes ☐ No If no,  36 34 Did you attend this training? ☐ Yes ☐ No If no,  36 35 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 36 Reach Every District (zone) training? ☐ Yes ☐ No If no,  39 37 Did you attend this training? ☐ Yes ☐ No If no,  39 38 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 39 Vaccine Safety Training? ☐ Yes ☐ No If no,  42 40 Did you attend this training? ☐ Yes ☐ No If no,  42 41 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 42 Administrative and human resource management? ☐ Yes ☐ No If no,  45 43 Did you attend this training? ☐ Yes ☐ No If no,  45 44 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 45 Financial management? ☐ Yes ☐ No If no,  48 342 The Impact of USAID’s Integrated Health Program in the DRC 46 Did you attend this training? ☐ Yes ☐ No If no,  48 47 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 48 Management ethics and patient confidentiality? ☐ Yes ☐ No If no,  51 49 Did you attend this training? ☐ Yes ☐ No If no,  51 50 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 51 Training of health care providers? ☐ Yes ☐ No If no,  54 52 Did you attend this training? ☐ Yes ☐ No If no,  54 53 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 54 Capacity-building of CODESAs? ☐ Yes ☐ No If no,  57 55 Did you attend this training? ☐ Yes ☐ No If no,  57 56 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 57 Health information systems (e.g. DHIS2, SNIS, MAPEPI)? ☐ Yes ☐ No If no,  60 58 Did you attend this training? ☐ Yes ☐ No If no,  60 59 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 60 Community scorecard processes? ☐ Yes ☐ No If no,  63 61 Did you attend this training? ☐ Yes ☐ No If no,  63 62 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 63 Material resource management (medicines, supplies, equipment)? ☐ Yes ☐ No If no,  66 64 Did you attend this training? ☐ Yes ☐ No If no,  66 65 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 66 Gender issues and/or the gender transformative approach? ☐ Yes ☐ No If no,  69 Baseline Report 343 67 Did you attend this training? ☐ Yes ☐ No If no,  69 68 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 69 Supportive supervision? ☐ Yes ☐ No If no,  72 70 Did you attend this training? ☐ Yes ☐ No If no,  72 71 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 72 Team management? ☐ Yes ☐ No If no,  75 73 Did you attend this training? ☐ Yes ☐ No If no,  75 74 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 75 Planning, monitoring and evaluation? ☐ Yes ☐ No If no,  78 76 Did you attend this training? ☐ Yes ☐ No If no,  78 77 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 78 Disease prevention, health promotion and re-adaptation care? ☐ Yes ☐ No If no,  81 79 Did you attend this training? ☐ Yes ☐ No If no,  81 80 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 81 Management of outbreaks, emergencies and disasters? ☐ Yes ☐ No If no,  84 82 Did you attend this training? ☐ Yes ☐ No If no,  84 83 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 84 Health research? ☐ Yes ☐ No If no,  87 85 Did you attend this training? ☐ Yes ☐ No If no,  87 86 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 87 Data analysis? ☐ Yes ☐ No If no,  90 344 The Impact of USAID’s Integrated Health Program in the DRC SECTION 3. BASIC INFRASTRUCTURE NO Questions and filters SKIP 93 RECORD OBSERVATION OF THE MAIN MATERIAL OF THE FLOOR NATURAL FLOOR EARTH/SAND DUNG RUDIMENTARY FLOOR WOOD PLANKS PALM/BAMBOO FINISHED FLOOR PARQUET OR POLISHED WOOD VINYL OR ASPHALT STRIPS CERAMIC TILES CEMENT CARPET OTHER ________________________ (SPECIFY) 11 12 21 22 31 32 33 34 35 96 94 RECORD OBSERVATION OF THE MAIN MATERIAL OF THE ROOF NATURAL ROOFLING NO ROOF THATCH/PALM LEAF SOD RUDIMENTARY ROOFING MAT PALM/BAMBOO WOOD PLANKS CARDBOARD FINISHED ROOFING METAL WOOD CALAMINE/CEMENT FIBRE 11 12 13 21 22 23 24 31 32 33 88 Did you attend this training? ☐ Yes ☐ No If no,  90 89 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ 90 Use of data for evidence-based decision-making? ☐ Yes ☐ No If no,  93 91 Did you attend this training? ☐ Yes ☐ No If no,  93 92 If yes, who administered the training: [select all that apply] ☐ Internal/HZ office ☐ Central/Government ☐ USAID/IHP ☐ Other: __________________________ Baseline Report 345 CERAMIC TILES CEMENT OTHER ________________________ (SPECIFY) 34 35 96 95 RECORD OBSERVATION OF THE MAIN MATERIAL OF THE EXTERIOR WALLS NATURAL WALLS NO WALLS BAMBOO/CANE/PALM/ TRUNK DIRT RUDIMENTARY WALLS BAMBOO WITH MUD STONE WITH MUD UNCOVERED ADOBE PLYWOOD CARDBOARD REUSED WOOD FINISHED WALLS CEMENT SOTEON WITH LIME/CEMENT BRICKS CEMENT BLOCKS COVERED ADOBE WOOD PLANKS OTHER ________________________ (SPECIFY) 11 12 13 21 22 23 24 25 26 31 32 33 34 35 36 96 96 Does this office have cell phone reception? RECORD OBSERVATION ☐ Yes ☐ No If no,  99 97 How would you rate the ease of connectivity to the nearest health center? VERY EASY SOMEWHAT EASY SOMEWHAT DIFFICULT VERY DIFFICULT 01 02 03 04 98 How would you rate the ease of connectivity to the furthest health center? VERY EASY SOMEWHAT EASY SOMEWHAT DIFFICULT VERY DIFFICULT 01 02 03 04 99 Does this office have electricity? RECORD OBSERVATION ☐ Yes ☐ No If no,  105 100 What is the main source of electricity? Main power Solar power Generator Other (specify) ___________ 01 02 03 96 101 On a typical 8-hour work day, how many hours is electricity available at this office? |__| hours 346 The Impact of USAID’s Integrated Health Program in the DRC 102 Is electricity functioning now? (CHECK TO SEE IF ELECTRICITY CAN BE TURNED ON.) RECORD OBSERVATION ☐ Yes ☐ No 103 Are there power cuts (excluding electricity supplied by a generator backup) during the hours when the office is open? ☐ Yes ☐ No If no,  105 104 What is the average duration of power cuts? |__|__| hours |__|__| days 105 Does this office have a generator for electricity? This may be a back-up or stand-by generator. RECORD OBSERVATION ☐ Yes ☐ No Refer to 100 and answer if Generator, then  106; otherwise,  107 106 Is the generator functional and is there fuel today? RECORD OBSERVATION ☐ Yes ☐ No 107 In the last month, how often have power cuts interfered with: 108 Cold chain maintenance? Never Rarely Sometimes Often Always Don't know Not applicable (N/A) 01 02 03 04 05 98 97 Refer to 103 and answer if yes; otherwise,  110 109 DHIS2 reporting? Never Rarely Sometimes Often Always Don't know Not applicable (N/A) 01 02 03 04 05 98 97 110 Is there a functional solar panel? RECORD OBSERVATION ☐ Yes ☐ No If yes,  111; otherwise,  112 111 Is there a functional battery for solar panel? RECORD OBSERVATION ☐ Yes ☐ No 112 Does this office have a radio? RECORD OBSERVATION ☐ Yes ☐ No 113 Does this office have a calculator? RECORD OBSERVATION ☐ Yes ☐ No 114 How many functional computers are there at the office and of which type? RECORD OBSERVATION IF THE RESPONDENT DOESN’T KNOW THE NUMBER FOR A TYPE OF COMPUTER, WRITE “99” IN THE BOX. DPS OWNED (NOT PERSONAL) DESKTOP |__|__| DPS OWNED (NOT PERSONAL) LAPTOP |__|__| PERSONAL LAPTOP |__|__| 115 How many functional printers are there at the office? RECORD OBSERVATION |__|__| 116 In general, does this office have access to the internet? ☐ Yes ☐ No 117 Do you have the following sources of internet? Check all that apply Baseline Report 347 118 Modem using commercial telephone network (VODACOM, AIRTEL, etc.) ☐ Yes ☐ No If yes,  119 119 Is it functioning? RECORD OBSERVATION ☐ Yes ☐ No 120 V-Sat antenna? ☐ Yes ☐ No If yes,  121 121 Is it functioning? RECORD OBSERVATION ☐ Yes ☐ No 122 Personal mobile/wifi hotspot? ☐ Yes ☐ No If yes,  123 123 Is it functioning? RECORD OBSERVATION ☐ Yes ☐ No 124 Other source of internet? SPECIFY__________________ ☐ Yes ☐ No 125 Was internet connectivity provided by USAID/IHP? ☐ Yes ☐ No 126 How common is it for people at this office to rely on their own personal mobile hotspot? Never Rarely Sometimes Often Always Don't know Not applicable (N/A) 01 02 03 04 05 98 97 127 In a typical 8-hour workday, how many hours does the internet at the office work, not including personal mobile hotspots? |__| hours 128 Can you log into the DHIS2 system for me so that I can see whether it is working? Check all that apply. Yes, observed No, no working computer No, no electricity No, no internet No, slow internet No, forgot username No, forgot password No, refused No, other reason (specify) ______________________ 01 02 03 04 05 06 07 99 96 SECTION 4. GENERAL STAFFING Now I have some questions about staffing for this DPS. Please tell me how many people currently work in the following positions/units as members of this provincial management team and whether they are male or female. Please include all persons who work in this DPS, whether in a full-time, part-time or volunteer capacity. Does this DPS currently employ a or have employees in the… No. Staff No. Male No. Female 129 Chef de Division ☐ Yes ☐ No (go to 130) ☐ Don’t know (go to 130) |__|__|__| |__|__|__| |__|__|__| 130 Secretary ☐ Yes ☐ No (go to 131) ☐ Don’t know (go to 131) |__|__|__| |__|__|__| |__|__|__| 131 Office of Resource Management ☐ Yes ☐ No (go to 132) |__|__|__| |__|__|__| |__|__|__| 348 The Impact of USAID’s Integrated Health Program in the DRC SECTION 5. HEALTH INFORMATION SYSTEM IDENTIFY THE PERSON PRIMARILY RESPONSIBLE FOR ENTERING DATA INTO THE DHIS2 SYSTEM TO ANSWER THIS SECTION Now I would like to ask you questions related to health information, specifically the DHIS2. NO QUESTION RESPONSE SKIP/ INSTRUCTIONS 139 How many health zones are currently required to submit DHIS2 reports to the DPS? IF UKNOWN, ENTER 999. |__ | __ |__| If 0,  145 140 In the last month, how many of the [repeat number given above] health zones submitted their DHIS2 reports on time? ALL SOME NONE DON’T KNOW 01 02 00 98 If none,  142 141 Can I see the health zone reports from the last month? Not observed Observed 00 01 142 In the last month, how did this office submit its weekly Maladie à Potentiel Epidémique (MAPEPI) to the Central Level (Disease Control Directorate)? Hand deliver to DPS DPS picks up Phone call Text message Radio communication Email from office Email from cybercafé or other off-site location Electronically via DHIS2 Other (specify)_____________________ 01 02 03 04 05 06 07 08 96 ☐ Don’t know (go to 132) 132 Office of Technical Support ☐ Yes ☐ No (go to 133) ☐ Don’t know (go to 133) |__|__|__| |__|__|__| |__|__|__| 133 Office of Inspection Control ☐ Yes ☐ No (go to 134) ☐ Don’t know (go to 134) |__|__|__| |__|__|__| |__|__|__| 134 Office of Health Information ☐ Yes ☐ No (go to 135) ☐ Don’t know (go to 135) |__|__|__| |__|__|__| |__|__|__| 135 Office of Hygiene and Public Sanitation ☐ Yes ☐ No (go to 136) ☐ Don’t know (go to 136) |__|__|__| |__|__|__| |__|__|__| 136 Office of Health Sciences Training ☐ Yes ☐ No (go to 137) ☐ Don’t know (go to 137) |__|__|__| |__|__|__| |__|__|__| 137 Does this office have a designated person to enter/compile DHIS2 data reports from health facilities? YES NO DON’T KNOW 01 00 99 138 Does the district have a designated person to review and control the quality of data entry/compilation? YES NO DON’T KNOW 01 00 99 Baseline Report 349 143 To the best of your knowledge, the last time you encountered a MAPEPI case, how much time passed between when you were made aware of the MAPEPI event and when you were able to report to the Central Level (Disease Control Directorate)? DO NOT READ ANSWERS BUT CHECK THE ONE CLOSEST TO THE ONE GIVEN. Immediately Within 24 hours The next day > 2 days Don't know 01 02 03 04 98 144 What are some of the reasons why an immediate or weekly MAPEPI report may not be submitted on time? SEVERAL RESPONSES POSSIBLE. CHECK ALL THAT ARE MENTIONED Did not have the correct form No supervision visits Lack of transportation Lack of internet Lack of telephones Lack of electricity No time to complete reports No staff to complete reports Not aware of submission deadlines Other (specify)_______________________ 01 02 03 04 05 06 07 08 09 96 Read aloud: Every DPS office may do things differently. I am going to list a few tasks that might be performed using DHIS2 data. Please tell me whether you have done any of the following in the past 6 months: 145 Generated a report with results for one or more health zones? No Yes Not applicable (N/A) Don’t know 00 01 97 98 146 Generated a summary report for the province? No Yes Not applicable (N/A) Don’t know 00 01 97 98 147 Compared results with provincial/national targets? No Yes Not applicable (N/A) Don’t know 00 01 97 98 148 Compared results across different service domains? No Yes Not applicable (N/A) Don’t know 00 01 97 98 149 Compared results over time? No Yes Not applicable (N/A) Don’t know 00 01 97 98 150 Do you have a copy of the DHIS2 manual? YES, OBSERVED YES, NOT OBSERVED NO DON’T KNOW 01 02 00 98 151 Have you read the DHIS2 manual? No Yes Not applicable (N/A) Don’t know 00 01 97 98 If no,  152 152 How easy or difficult would you say the DHIS2 procedure manual is to understand? VERY EASY SOMEWHAT EASY SOMEWHAT DIFFICULT 01 02 03 350 The Impact of USAID’s Integrated Health Program in the DRC DON’T KNOW 98 153 Does the district have written guidelines for data entry/compilation and data quality review and control? No Yes Not applicable (N/A) Don't know 00 01 97 98 154 In the last month, how many feedback reports on data quality and performance has this DPS sent to health zone offices? IF UNKNOWN, ENTER 999. |__|__|__| If 0,  161 155 Can I see the reports? SELECT A REPORT AT RANDOM AND RECORD THE FOLLOWING OBSERVATIONS. YES, OBSERVED NO, NOT OBSERVED 01 00 If no,  161 Was any of the following feedback included in the report? 156 Accuracy of data No Yes Not applicable (N/A) Don't know 00 01 97 98 157 Completeness of data No Yes Not applicable (N/A) Don't know 00 01 97 98 158 Timeliness of data No Yes Not applicable (N/A) Don't know 00 01 97 98 159 Positive feedback No Yes Not applicable (N/A) Don't know 00 01 97 98 160 Other feedback No Yes (specify) __________________________ Not applicable (N/A) Don't know 00 01 97 98 161 Since October 2018, how many feedback reports on data quality and performance has this office received from Central Level (Disease Control Directorate)? IF UNKNOWN, ENTER 999. |__|__|__| If 0,  168 162 Can I see the reports? SELECT A REPORT AT RANDOM AND RECORD THE FOLLOWING OBSERVATIONS. YES, OBSERVED NO, NOT OBSERVED 01 00 If no,  168 Was any of the following feedback included in the report? 163 Accuracy of data No Yes Not applicable (N/A) Don't know 00 01 97 98 164 Completeness of data No Yes Not applicable (N/A) Don't know 00 01 97 98 Baseline Report 351 165 Timeliness of data No Yes Not applicable (N/A) Don't know 00 01 97 98 166 Positive feedback No Yes Not applicable (N/A) Don't know 00 01 97 98 167 Other feedback No Yes (specify) __________________________ Not applicable (N/A) Don't know 00 01 97 98 168 Does this office have a map of the province and if so, what type of map is it? YES, COMPUTER GENERATED YES, HAND-DRAWN NO MAP DON’T KNOW 01 02 03 98 169 Does the office display a summary of demographic information for the province (i.e. number of health zones; population numbers for each health area)? No Yes Not applicable (N/A) Don't know 00 01 97 98 170 Does the office display more specific demographic information (such as population by target groups)? No Yes Not applicable (N/A) Don't know 00 01 97 98 What kinds of actions have been taken based on DHIS2 data? 171 Review strategy by examining service performance target and actual performance from month to month No Yes Not applicable (N/A) Don't know 00 01 97 98 172 Review facility personnel responsibilities by comparing service targets and actual performance from month to month No Yes Not applicable (N/A) Don't know 00 01 97 98 173 Mobilization/shifting of resources based on comparison between services No Yes Not applicable (N/A) Don't know 00 01 97 98 174 Advocacy for more resources by showing gaps in ability to meet targets No Yes Not applicable (N/A) Don't know 00 01 97 98 Now I would like to ask you some questions related to meetings and decisions based on the DHIS2 data. 175 How frequently does this office have routine meetings where the DHIS2 or facility reported data is discussed? This could be a separate meeting or in the routine managerial or administrative meetings. Monthly or more often Every 2-3 months Every 4-6 months Less than every 6 months or irregularly Don't know 01 02 03 04 98 GO TO 166 176 Is an official record of management meetings maintained? No Yes Not applicable (N/A) 00 01 97 352 The Impact of USAID’s Integrated Health Program in the DRC Don't know 98 177 Have the DHIS2 data been used to make decisions? No Yes Not applicable (N/A) Don't know 00 01 97 98 178 Has any follow-up action taken place regarding the decisions made during the previous meetings? No Yes Not applicable (N/A) Don't know 00 01 97 98 SECTION 6. SOURCES AND USES OF FUNDING Now I would like to ask you some questions about sources and uses of funding for this DPS office. No. QUESTION RESPONSE SKIP/ INSTRUCTIONS What is the total amount of operational funds (i.e. funds not passed on to another organization) received from the following sources in the last calendar year (2018)? 179 Ministry of health/Provincial health office |__|__|__|__|__|__|__|FC |__|__|__|__|__|__|__|USD 180 USAID/IHP |__|__|__|__|__|__|__|FC |__|__|__|__|__|__|__|USD 181 Other NGO’s or FBO’s: Cordaid, Memisa, Foundation Damien, save the Children, Rescue, IRC, Caritas, Sanru, Chemonics, MCSP |__|__|__|__|__|__|__|FC |__|__|__|__|__|__|__|USD 182 User fees |__|__|__|__|__|__|__|FC |__|__|__|__|__|__|__|USD 183 Other health insurance program? |__|__|__|__|__|__|__|FC |__|__|__|__|__|__|__|USD 184 Other 1 (specify) ________________________________ |__|__|__|__|__|__|__|FC |__|__|__|__|__|__|__|USD 185 Other 2 (specify) ________________________________ |__|__|__|__|__|__|__|FC |__|__|__|__|__|__|__|USD What percentage of operational funds was spent on the following during the last calendar year (2018)? 186 Savings |__|__|__| % 187 Building/grounds improvements |__|__|__| % 188 Equipment (vehicles, computers, etc.) |__|__|__| % 189 Utilities and communication (electricity, water, phone credit, internet, etc.) |__|__|__| % 190 Medical supplies |__|__|__| % 191 Drugs |__|__|__| % Baseline Report 353 192 Transport |__|__|__| % 193 Salaries and primes |__|__|__| % 194 Training |__|__|__| % 195 Other 1 (specify) ______________________________ |__|__|__| % 196 Other 2 (specify) ______________________________ |__|__|__| % SECTION 7. SUPPORT FROM USAID/IHP NO QUESTION RESPONSE SKIP/ INSTRUCTIONS Capacity building 197 Has this office ever completed a Participatory Institutional Capacity Assessment and Learning (PICAL) assessment, facilitated by someone from USAID, IHP or Abt Associates? [ ] Yes [ ] No [ ] Don’t know/ not sure If no,  202 198 For the last PICAL assessment, which members of the DPS participated? Check all that apply CHEF DE DIVISION SECRETARY OFFICE CHIEF OF RESOURCE MANAGEMENT OFFICE CHIEF OF TECHNICAL SUPPORT OFFICE CHIEF OF INSPECTION AND CONTROL OFFICE CHIEF OF HEALTH INFORMATION OFFICE CHIEF OF HYGIENE AND PUBLIC SANITATION OFFICE CHIEF OF HEALTH SCIENCES TRAINING OTHER _______________________ 199 How long ago was most recent PICAL assessment? [ ] 0-3 months ago [ ] 4-6 months ago [ ] 7-12 months ago [ ] More than 1 year ago [ ] Don’t know 200 Did the office receive its “PICAL score” as a result of that assessment? [ ] Yes [ ] No [ ] Don’t know If no,  202 201 What was the score? ____________________________ 354 The Impact of USAID’s Integrated Health Program in the DRC SECTION 8: MANAGEMENT AND SUPERVISION LEADERSHIP In this part of the questionnaire, I would like to know what you would do in certain situations regarding this DPS. I will read you a series of scenarios. For each scenario, I will read 4 possible responses that you might have. Please select the response that most closely matches what you would do in this specific situation. You can only select one response foreach scenario. There are no correct or incorrect answers -we just want to know how you would approach each situation. INTERVIEWER: EACH RESPONSE IS CODED BETWEEN 1 AND 4. RECORD APPROPRIATE CODE ACCORDING TO RESPONDENT'S RESPONSE. NO SCENARIOS 1 2 3 4 RECORD RESPONSE (RANGE 1-4) 202 Scenario 1: The performance of your staff is improving. You stress their responsibilities and standards. You take no particular additional action. You give positive feedback and make staff feel involved in the achievements. You emphasize the importance of deadlines and tasks. 203 Scenario 2: Members of your staff have been unable to solve a problem over the past month, though they have been trying to address it. You call a meeting and together try to solve the problem. You let your staff address this problem on their own. You give them direction and instructions on how to solve the problem. You encourage the group to solve the problem on their own, and you are available when needed to discuss. 204 Scenario 3: You are considering a major change in how things are done in the office. You collaborate with your staff to develop the needed changes. You announce your vision for the changes and implement a clear plan. You ask your staff to develop and implement their own plan for change. You consult with your staff, but direct the changes yourself. 205 Scenario 4: The performance of your staff has been falling in recent months. You ask your staff to rethink their direction and goals and come up with a plan together. You ask for suggestions from your staff on what to do, and you formulate a specific plan to meet objectives. You redefine goals clearly and supervise whether these are being met closely. You allow your staff freedom to set their own goals and do not push them. 206 Scenario 5: Your staff are no longer working together as an effective team. You discuss ideas as a group and identify how to work better together. You let your staff work out their issues on their own. You act quickly and decisively to get the team back on track. You make yourself available to discuss any issues and support your team to work out their own problems. AUTONOMY In this part of the questionnaire I would like to ask you some questions regarding how work is organized and how decisions are made in this DPS. All answers are confidential. Baseline Report 355 I am now going to read you a series of statements about decision-making and authority in this facility. Please tell me whether you feel these are true always, often, sometimes, rarely or never. RESPONSE CODE RECORD RESPONSE ALWAYS 1 OFTEN 2 SOMETIMES 3 RARELY 4 NEVER 5 207 I am able to allocate my DPS budget according to how it is needed. There is enough flexibility in my budget. 208 I am able to assign tasks and activities to staff as needed to achieve the outcomes I want in the DPS office. There is enough flexibility to use staff to address needs. 209 The Central Level Office/Minister of Health supports my decisions and actions for doing a better job in my office. 210 I have choice over who I allocate for what tasks. 211 I have choice over what services are provided in the facilities within the health zones. 212 I have enough authority to obtain the resources I need (drugs, supplies, funding) to meet the needs of the facilities within my health zones. 213 The policies and procedures for doing things are clear to me. 214 The policies and procedures for doing things are useful tools for the challenges I face in providing services and reporting on activities. 215 The Central Level Office/Minister of Health provides adequate feedback to me about my job and the performance of my facilities/health zones. INTERNAL MANAGEMENT NO QUESTION RESPONSE SKIP/ INSTRUCTIONS 216 Does this office have routine meetings for reviewing managerial or administrative matters? YES NO DON’T KNOW 01 02 99 If no,  230 217 In 2018, how often did meetings to discuss the managerial and administrative matters take place? MONTHLY OR MORE OFTEN QUARTERLY TWICE A YEAR ANNUALLY IRREGULARLY NEVER DON’T KNOW 01 02 03 04 05 06 98 If never or don’t know,  230 218 Are meeting minutes available? YES NO DON’T KNOW 01 00 98 If no,  230 219 Can I see the meeting minutes from 2018 until now? OBSERVED NOT OBSERVED 01 00 If no,  230 220 RECORD THE NUMBER OF MEETINGS THAT HAVE BEEN HELD SINCE OCTOBER 2018. |__|__|__| 356 The Impact of USAID’s Integrated Health Program in the DRC Which members of the DPS attend these meetings and how often? Who attends the routine meetings? 221 CHEF DE DIVISION 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 222 SECRETARY 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 223 OFFICE CHIEF OF RESOURCE MANAGEMENT 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 224 OFFICE CHIEF OF TECHNICAL SUPPORT 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 225 OFFICE CHIEF OF INSPECTION AND CONTROL 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 226 OFFICE CHIEF OF HEALTH INFORMATION 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 227 OFFICE CHIEF OF HYGIENE AND PUBLIC SANITATION 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 228 OFFICE CHIEF OF HEALTH SCIENCES TRAINING 75-100% 50-74% 25-49% 1-24% Don’t know Not applicable (N/A) 01 02 03 04 98 97 229 What types of decisions are made in these meetings? Scheduling of activities in the health zone Allocating financial resources 01 02 Baseline Report 357 SEVERAL ANSWERS POSSIBLE. DO NOT READ THE ANSWERS BUT CHECK ALL MENTIONED Allocating technical resources Defining programmatic priorities Coordinating strategies with donors Human resources management Conflict resolution 03 04 05 06 07 230 Do you or does someone from this office make supervision visits to health zone offices? YES NO DON’T KNOW 01 00 99 If no,  233 231 Are supervision reports available? YES NO DON’T KNOW 01 00 99 If no,  233 232 Can I see the most recent report? OBSERVED NOT OBSERVED. 01 00 EXTERNAL SUPERVISION LEVEL FROM WHICH THE SUPERVISOR CAME In 2018, how many times did the supervisor from the [LEVEL] visit this office for the purpose of management or supervision of routine and non-routine activities? Do you know the date (month and year) of the most recent visit by a supervisor from the [LEVEL]? When did a supervisor from the [LEVEL] last visit? What topic was discussed at the last visit? CHECK ALL THAT APPLY After the last visit, did the supervisor send you a report based on his/her findings? 233 National-/ Central-level authorities ☐ 0 ☐ 1 ☐ 2 ☐ 3 ☐ 4 ☐ 5 ☐ 6 ☐ 7 ☐ 8 ☐ 9 ☐ 10 ☐ 11 ☐ 12 + ☐ Don’t know ☐ No ☐ Yes ☐ Don’t know Month: ______ Year: ______ ☐ Quality of care ☐ Management ☐ Record keeping ☐ Data reporting ☐ Data use ☐ Don’t know ☐ No ☐ Yes ☐ Don’t know 234 IHP officials ☐ 0 ☐ 1 ☐ 2 ☐ 3 ☐ 4 ☐ 5 ☐ 6 ☐ 7 ☐ 8 ☐ No ☐ Yes ☐ Don’t know Month: ______ Year: ______ ☐ Quality of care ☐ Management ☐ Record keeping ☐ Data reporting ☐ Data use ☐ Don’t know ☐ No ☐ Yes ☐ Don’t know 358 The Impact of USAID’s Integrated Health Program in the DRC SECTION 9. STAKEHOLDER COORDINATION & COLLABORATION The next questions are about different kinds of interactions your DPS may have with other provincial offices. NO QUESTION RESPONSE SKIP/ INSTRUCTIONS Communication 235 How often do you communicate with different health zone offices? [ ] Monthly or more often [ ] Quarterly [ ] Twice a year [ ] Annually [ ] Irregularly [ ] Don't know 236 Since October 2018 when the IHP project was implemented, would you say this DPS had more, less, or about the same amount of communication with health zone offices? [ ] MORE [ ] LESS [ ] ABOUT THE SAME [ ] DON’T KNOW 237 How many other provincial offices does this DPS communicate with for professional reasons at least quarterly? |__|__|__| If 0,  239 238 How often do you communicate with other provincial offices? [ ] Monthly or more often [ ] Quarterly [ ] Twice a year [ ] Annually [ ] Irregularly [ ] Don't know 239 Since October 2018 when the IHP project was implemented, would you say this DPS had more, less, or about the same amount of communication with other provincial offices? [ ] MORE [ ] LESS [ ] ABOUT THE SAME [ ] DON’T KNOW Please tell me how often you or someone from your organization does each of the following things as part of their day-to￾day work. 240 Do you or does someone from your office attend technical or partner review meetings? [ ] Yes [ ] No If no, go to Section 10 241 How many times have these meetings been held? [ ] 0 [ ] 1 [ ] 2 [ ] 3 [ ] 4 [ ] 5 [ ] 6 [ ] 7 [ ] 8 [ ] 9 ☐ 9 ☐ 10 ☐ 11 ☐ 12 + ☐ Don’t know Baseline Report 359 [ ] 10 [ ] 11 [ ] 12 or more [ ] Don't know 242 How many times have you or someone from your office contributed ideas or lessons learned at these technical or partner review meetings? [ ] 0 [ ] 1 [ ] 2 [ ] 3 [ ] 4 [ ] 5 [ ] 6 [ ] 7 [ ] 8 [ ] 9 [ ] 10 [ ] 11 [ ] 12 or more [ ] Don't know 243 How many times have you or someone from your office served as a meeting facilitator or guest speaker? [ ] 0 [ ] 1 [ ] 2 [ ] 3 [ ] 4 [ ] 5 [ ] 6 [ ] 7 [ ] 8 [ ] 9 [ ] 10 [ ] 11 [ ] 12 or more [ ] Don't know 244 How many times have you or someone from your office worked with other DPS officials at these meetings to influence decision-making or development of strategies? [ ] 0 [ ] 1 [ ] 2 [ ] 3 [ ] 4 [ ] 5 [ ] 6 [ ] 7 [ ] 8 [ ] 9 [ ] 10 [ ] 11 [ ] 12 or more [ ] Don't know 360 The Impact of USAID’s Integrated Health Program in the DRC SECTION 10. CAPACITY BUILDING This next section includes questions about specific local-level entities and organizations your DPS has worked with to strengthen institutional capacity and improve health of Congolese citizens. By “work with” we mean both informally, for example through exchange of information, and formally, for example by attending the same training or workshop. The term “local-level entities” includes other health zone offices in addition to government enterprises other than the Ministry of Health (e.g. Ministry of Finance, Ministry of Education) and private sector partners. “Organizations” refers to all non-governmental institutions that have provided technical, financial or other support. List the organizations and other local-level entities with whom you have worked over the last 6 months to help achieve goals related to strengthened capacity and improved health outcomes. Over the last 6 months how frequently have you worked with this organization? In the last 6 months, how often did your office share or exchange information with THIS ORGANIZATION? In the last 6 months, how often did your organization jointly plan, implement or monitor an activity, event, or service with THIS ORGANIZATION? In the last 6 months, how often did your office receive technical assistance from THIS ORGANIZATION to strengthen skills in data analysis, use and reporting? In the last 6 months, how often did your office receive financial or other support from THIS ORGANIZATION to build institutional capacity? Does your office have a formal relationship with THIS ORGANIZATION, through a contract, memorandum of understanding or some other written document describing your partnership? Responses 1=Never 2=Once, twice, or just a few times 3=Monthly 4=Weekly 5=Daily 1=Never 2=Once, twice, or just a few times 3=Monthly 4=Weekly 5=Daily 1=Never 2=Once, twice, or just a few times 3=Monthly 4=Weekly 5=Daily 1=Never 2=Once, twice, or just a few times 3=Monthly 4=Weekly 5=Daily 1=Never 2=Once, twice, or just a few times 3=Monthly 4=Weekly 5=Daily 0 = NO 1 = YES 98 = DON’T KNOW Thank you for the time. Interview End Time: Hour |___|___| Minutes |___|___| Baseline Report 361 In-depth Interview Guide General What in your view are the major obstacles to health service utilization? How do these factors vary across health zones and regions? How would you describe the present quality of health services offered in this facility? What are the deficits and how could quality be improved? Availability of quality, integrated facility-based health services What comprises the basic package of health services? What services does your facility provide? What services are devoted to child health? Has an assessment of the availability of quality, integrated health services been carried out in this facility? If so, when and what were the results and follow up actions? Have any of the facility staff received training or attended a workshop in the past year? If so, what did this entail? How would you assess the training received? To what extent are treatment protocols available for different childhood diseases? How are these protocols used? Are there any other tools such as service flowcharts and technical guidelines which health providers in this center follow? When were these tools introduced? Do you feel that these tools are useful? Why or why not? To what extent are child health services integrated in respect to either treatment or preventive care? Can you please give me some examples of integration of these services? How could integration be improved? Availability of quality, integrated community-based health services How would you describe the community-based health services in your area? Has there been an assessment of the CODESAs in this area? Do the CODESAs receive any training, technical assistance or mentoring? What is the current role of the CODESAs in this area? How many CODESAs are female? How many CHWs are presently active? How many of the active CHWs are female and how recently did they become CHWs? As CHWs, what does their work entail? Have the CHWs received any recent training or attended competency workshops? If so, what did the training entail and how would you assess the training received? What are the community-based organizations and structures in place to promote health care and provide services? What types of activities do they engage in? Do they have adequate materials to carry out these activities? What are some of the strengths and weaknesses of these activities? To what extent are these activities coordinated? What are some needed changes to improve their work? Do you have functioning iCCMs in this area? If so, what services related to child health care are available and how often are services offered? What medications, supplies, equipment and tools are available to provide services? Are CHWs permitted to prescribe medications for child illnesses, such as antibiotics? Are the services sufficient to meet the population needs? What are some of the challenges in providing community-based services? 362 The Impact of USAID’s Integrated Health Program in the DRC Are facility-based workers involved in any community outreach activities or training of community workers? Please explain. Is community monitoring such as community scorecard approaches carried out in this area? If so, how does this work and who is involved? What is the overall purpose of community monitoring? Referral systems from community-based platform to health centers to reference hospitals What are the current referral processes from the community and health center level to higher facilities? What types of health conditions require referral to a higher facility? Which workers have the authority to make a referral? How is tracking of referrals or counter referrals conducted? To what extent are mobile phones used to monitor referrals? Are specific guidelines in place for referrals and counter referrals? If so, what are some of the recommendations regarding referrals? At present, when do referrals generally occur? What are some of the barriers that hinder acceptance to referrals? How are patients transported to referral facilities? Are there any mechanisms set up to assist community members with transport? What training have CHWs received regarding referrals? What about the facility-based providers? Health provider attitudes and interpersonal skills at the facility and community levels How would you describe facility-based health provider relations with patients? Are there aspects of the health provider-patient interaction that needs improvement? Do you feel that facility-based providers are generally respectful towards and sensitive to the needs of caregivers of young children? Are there any instances where you or your colleagues exhibit behavior with caregivers that you feel is inappropriate or could be improved? Please explain. Why do you feel that these occurrences occur? How would you describe CHW relations with community members? Are there aspects of the CHW￾patient interaction that needs improvement? Do you feel that CHWs are generally respectful towards and sensitive to the needs of caregivers of young children? Are there any instances that you can recall when CHWs exhibited behavior with caregivers that you felt was inappropriate? Please explain. Why do you feel that these occurrences occur? We all know that certain behaviors or health conditions may evoke negative responses in the community. Are there types of patients (e.g. pregnant adolescent girls, HIV-infected patients, poor patients, children with disabilities) or medical conditions (SGBV, HIV, fistula, etc) you see in the health center or community that may cause health care providers to show less sensitivity or compassion? Please explain. How does the sex of the provider and patient impact on health provider-patient interactions? How can this be explained, is it related to cultural issues, power, economics? How does the age of the provider and patient impact on health provider-patient interactions? Why do you think that some health providers have negative attitudes towards patients? How does this affect community members health care seeking behavior? How do you think that health provider attitudes and behaviors could be improved? Have you ever participated in a group session when health Baseline Report 363 provider attitudes and behaviors have been discussed? Have you ever received training focused on health provider behaviors and ways to improve provider-client interactions? Are there ways to report negative attitudes or practices carried out by health providers? In other parts of the country, hotlines have been set up to allow for reporting of negative health provider behaviors and practices. Does anything like that exist in this area? If so, do people use the hotline? If not, do you think that people would be willing to use the hotline? Why or why not? Might there be any concerns about retaliation for reporting bad behavior or practices? Availability of innovative financing approaches What are some of the biggest barriers to health care seeking for child health services? To what extent do user fees dictate careseeking for treatment of childhood illness? What about for preventive care such as well baby visits and vaccinations? Have any assessments been carried out in this health facility or area to explore possible ways to finance health care so that costs for poorer members of the population would be reduced? Are any strategies presently used to reduce costs for vulnerable members of the population who cannot afford health care? If so, how does this work? Is it effective in increasing demand for services? Do you know of health financing initiatives (e.g. vouchers, mutual, emergency funds, etc.) introduced to make health care costs more affordable and to increase demand for services? What about cost recovery schemes? How do you feel that these initiatives would work in this context? Basic facility infrastructure and equipment to ensure quality services What year was this facility constructed? Has your health facility ever been renovated? If so, why was this facility chosen to be renovated? Please explain what exactly was done. Do you feel that any renovations are presently needed in this facility? If so, what exactly would you like to see renovated? Have you taken any initiatives to make sure that these renovations are done? If so, what did you do? Who, if anybody, did you contact? Do the personnel in the central health zone know that you feel that these renovations are needed? Is basic equipment available in this health facility? When you are in need of equipment, what is done? Who do you contact/how is it obtained? When was the last time you received health care equipment? Is there any equipment that you do not have that you feel is needed to carry out essential health care? Is there any equipment that you do not have that you feel is needed to carry out essential child health care? What do you do when health equipment needs repair? Who do you contact and how are repairs financed? How is the health facility equipment maintained? Collaboration between central and decentralized levels through sharing of best practices and contributions to policy dialogue Do you participate in any forums or working groups such as meetings with the DPS or zonal offices that facilitate exchanges of lessons learned from the field? If so, what forums do you participate in, how regularly do you participate and what transpires during these meetings? Are you encouraged to share 364 The Impact of USAID’s Integrated Health Program in the DRC field experiences of best practices or challenges during these meetings? If so, how is this information used? Are there other ways that you disseminate or share field experiences with other government workers or implementing partners such as NGOs or UN agencies? Do you feel that these exchanges are important? Why or why not? How could they be improved? Have you participated in meetings that focus on the development of community activities? To what extent have these interactions affected community activities? Are you able to receive information that helps you to be more effective in your work? Are you satisfied with the information that you receive or would you like to receive more information? Do you have any suggestions regarding ways to receive information that would allow you to be better informed? Has any research been carried out in this area related to health services? If so, how was the research used or disseminated? Do you feel that research related to the health services offered would be beneficial? Why or why not? Practice of priority healthy behaviors at the individual, household and community levels How is information about curative or preventive health care for young children related to malaria, diarrhea or pneumonia or child vaccinations disseminated? When and what type of information is shared in the facility? What about at the community level? Have you ever invited community members to the health center to learn about the facilities being offered? If so, please explain. Do CHWs, CODESAs or other health center staff ever visit people’s homes to provide information on child health? If so, how often are home visits conducted? What type of information is shared and who in the household do they share it with? Are there other ways that information on child health care is disseminated? For instance, are community meetings ever led by health workers or community leaders? If so, how often and what information is provided? Are any campaigns related to child health care offered? If so, how often and what services are provided? Who in the community is eligible to receive these services? Are there any local community organizations or groups providing information on child health care? Is information ever shared in schools or churches? If so, please describe who is involved and the type of information shared. Are there other ways that information on child health care is disseminated such as through the television, radio or mobile phone? If so, what type of information is provided? What do you think about this venue and the type of information provided? To what extent is research used to follow and reach out to specific cases in the community? Do you ever use more personalized messaging approaches to reach out to patients or caregivers? What type of treatment is offered to children suffering from malaria, diarrhea or pneumonia? Are there other places where community members can receive treatment for these illnesses? Is treatment for Baseline Report 365 these illnesses offered at the community level? What are some of the challenges you face in offering treatment for these childhood illnesses? What about preventive care for these illnesses? For instance, are you involved in distribution of insecticide treated nets? How and how often are nets for young children distributed? Who is eligible to receive a net? What are some of the challenges you face in providing nets for young children? How are child vaccinations offered? What vaccinations are offered and how often? Who is eligible to receive vaccinations? What are some of the challenges you face in providing essential vaccinations for young children? What can you tell us about Champion Communities? Do you have any Champion Communities in this area? Have you or any of the health staff ever been invited to participate in community discussions about barriers community members face in accessing the health center? How about discussions around the practice of healthy behaviors such as rapid treatment for diarrhea, vaccinations for young children, or placing children under a net at night? If so, what exactly was discussed? Do you know how the messages related to healthy behaviors disseminated during well baby visits, community meetings or home visits were developed? During your time working in the health center or at the community level, have these messages changed or been modified in any way? Please explain. What do you think about the content of these messages? How would you suggest that the messages be improved? Staff motivation and needs What motivates health facility employees to perform their work well? What sort of potential is there for work advancement? How are employees compensated? How do you feel about the way that employees are compensated? What is needed to improve job performance? What are the capacity building needs in your health facility? How do you think that training opportunities would strengthen work performance? To what extent do you think that health center staff are satisfied with their work? Why do you feel this way? What motivates community workers to carry out their work? What is needed to improve their performance? What are the capacity building needs of community health workers including CODESA and CHWs? How do you think that training opportunities would strengthen their work performance? To what extent do you think that community health workers are satisfied with their work? Why do you feel this way? What mechanisms are presently set up to ensure transparency and accountability in the health center? How effective are these mechanisms? How might they be improved? Are funds or resources ever misused to compensate for poor pay? 366 The Impact of USAID’s Integrated Health Program in the DRC Key Informant Guide General questions How would you describe the health systems in the DR Congo? What are the strengths of the health systems? What are the weaknesses? What should the role of the government be in the provision of health care? How would you describe the way the government is fulfilling that role? What needs to be done to improve the government’s role in strengthening health systems so that they are better meet the needs of the population? Overall, what are the primary needs related to institutional strengthening and support to DRC health systems and service delivery? How do you see these needs being fulfilled? Where does the decentralization of health care stand in the DRC? What is needed to ensure that health systems decentralization is carried out as planned? Institutional and individual capacities to plan, lead, execute, and monitor health action plans National level How would you describe the capacity of health personnel to plan, implement and monitor health services at the different levels? What are the necessary steps to build capacity to plan, implement and monitor health services at the provincial, zonal and facility level? What organizations need to be involved to ensure that these steps be followed? What is your role or your organization’s role to ensure that these steps be carried out? What challenges do you anticipate facing related to building institutional and individual capacities? How can these challenges be addressed? Provincial and zonal levels How would you describe a well-functioning provincial or zonal office? How does your office meet that description? What are some of the strengths and weaknesses in regard to the functioning of your office? What about your role in the office, how do you see your strengths and weaknesses in regard to contributing to a well-functioning office setting? What do you see as the primary capacity building needs in this office? Have you or the office ever undergone an assessment such as PICAL to identify capacity building needs? How would you describe the office organization? Who is in charge and what is the hierarchy of command? How would you describe a good leader? What are the skills needed to provide effective leadership? Do you feel that you or others in this office possess those skills? Why or why not? In this office, how do you plan health activities? Is there a set schedule for the development of health action plans? Who is involved in developing health plans and what are their roles? What needs to be considered when developing a health action plan? Baseline Report 367 How does your office ensure implementation of health plans? What steps are involved in overseeing a health plan is carried out as envisioned and who is involved? What are some of the challenges in implementing a health plan and how are they addressed? How does your office monitor health services? What are some of the challenges in monitoring health services? What do you do with the monitoring information? How does the information collected inform decision making? What are management needs in your office setting? Who takes on management roles? What are some of the challenges your office faces in managing the office? What are your needs in regarding to improving capacity of management skills? Have you or others in your office ever participated in peer-to-peer learning during provincial or health zone meetings? If so, what did this entail? What do you know about performance-based financing? Have you ever participated in RBF workshops or been involved in RBF efforts? If so, please explain. Transparency and oversight in health service financing and administration National level What are the key actions needed to improve transparency and oversight in health service financing and administration at the zonal, facility and community level? What in your view is needed to improve capacity in administrative and financial management at the different levels? How do you define general inspection? What is the role of general inspection in regard to strengthening transparency and accountability? How is general inspection carried out now? What activities will be implemented over the course of the project aimed at strengthening general inspection? What do you anticipate in regard to challenges in implementing these activities? What actions are needed to increase transparency and report abuse? What do you know about the fraud and complaints hotline? How does this work and to what extent it could be effective in addressing corruption? For the IGS, what is your role in terms of coaching, regulation and supervision of the IPS in the provinces in financial management? How often do you make visits? How many visits have you made over the past year? Do you feel that these visits are effective in supporting capacity building? How could these visits be improved? Provincial and zonal level How does financial management in your office work? What sort of training and assistance have staff members received in regard to financial management and budget planning? What are some of the challenges in regard to financial management and budget planning? How does this affect the functioning of your offices and health care delivery? What are your needs in regard to improving financial management? 368 The Impact of USAID’s Integrated Health Program in the DRC How does administrative management in your office work? What sort of training has been received in regard to administrative management? What are some of the challenges in regard to administrative management? How does this affect the functioning of your offices and health care delivery? What are your needs in regard to improving administrative management? What is the role of CACs and the CODESAs in regard to administrative and financial management? How could these actors be involved to improve transparency and accountability of the healthcare system? Why do you think that corruption is such a big problem in the health care systems? What is presently being done to address fraud and corruption? How could oversight of fraud and corruption be improved? What do you know about the fraud and complaints hotline? How does this work and to what extent it could be effective in addressing corruption? How do you define general inspection? How is general inspection being conducted? What is the role of general inspection in improving accountability? CSO and community structures to provide health systems oversight What in your view is the role of community members in the oversight of health services? How could their involvement be improved? To what extent do community members and organizations participate in the identification of health problems and the needs of local populations? Do you view their participation as important? Why or why not? What mechanisms could be used to ensure greater dialogue between community members and health providers and community participation in health care planning, implementation, and monitoring? What are some of the challenges in ensuring greater community involvement? What community monitoring tools presently exist and how are they used? To what extent are women involved in decision making in communities? Do you view women’s involvement in decision making related to health care important? Why or why not? What approaches could be used to facilitate greater female involvement? At the provincial and zonal level, when was the last time an assessment of CODESAs was conducted? What is the present role of CAC and CODESA in the oversight of health services? When was the last time the CACs and CODESA were trained or received technical assistance and what did this entail? Have CODESAs in this area been trained to carry out community assessments? If so, how are these assessments being used? Have any of the health zones in this area been awarded community grants for facility improvements or emergency transportation plans? Have community organization and CODESA ever received training on gender issues? Stakeholder coordination at the provincial and zonal levels How would you describe coordination of health activities by different stakeholders at the provincial and zonal levels? Why is this important to the functioning of the health system? What are some Baseline Report 369 recommendations regarding ways to improve coordination? What resources are needed to make those improvements? At the provincial and zonal levels, what coordination meetings are presently held at the provincial, zonal and community levels? How often are these meetings held? How would you describe the meeting management in regard to organization, documentation and follow up? Have any changes in the management recently occurred? What is discussed during these meetings? Is the financial and technical support adequate to carry out these meetings regularly? Disease surveillance and strategic information gathering How is disease surveillance presently carried out? What are some of the strengths and weaknesses related to disease surveillance? Are there challenges related to capacity or the availability of technical resources and equipment? How would you describe the data quality (comprehensiveness, promptness, and accuracy) and what needs to be done to improve the quality of data collected during surveillance? How is routine data collection in health facilities presently carried out? What are some of the strengths and weaknesses related to routine data collection? Are there challenges related to capacity or the availability of technical resources? How would you describe the data quality (comprehensiveness, promptness, and accuracy) and what needs to be done to improve the quality of routine data collection? How would you describe the capacity of personnel at the provincial and zonal levels to collect, transmit, analyze, interpret data collected through surveillance (e.g. Maladie a Potential Epidemique) or routine data collection (e.g. DHIS2) to inform decision making? What sort of technical training or other support have they received in the past? What is expected of health center staff in regard to reporting of data during monthly zonal meetings or quarterly provincial data reviews? What in your view is needed to improve use of the data collected to inform health service planning? To what extent has mobile phone-based reporting been introduced? How is reporting of community health worker activities done? What are the needs in regard to community reporting? At present, how is data validation carried out? Are meetings or technical sessions carried out at the provincial level to validate the data? If so, how often and who attends? Management and motivation of human resources in health What triggers the recruitment of health personnel? How are health personnel presently recruited? How is gender considered when recruiting or deploying staff to another area? Do you think that gender should be a primary consideration related to staff recruitment? Have you ever been advised to recruit more women in the health services? If so, were the recommendations followed? Why or why not? To what extent are resource management tools such as iHRIS and the DHIS2 data used for staff recruitment, recruitment based on real needs, and the available budget for recruitment? What other systems are available to assess needs and manage staff recruitment? What do you know about the data quality audit processes to identify and remove ghost workers and ensure current staff are on payroll? 370 The Impact of USAID’s Integrated Health Program in the DRC What if any incentives (both positive and negative) are presently offered to motivate personnel or discourage bad practices? Do you consider these motivators effective? What other motivators would you like to see considered? At the provincial and zonal level, how are job descriptions and roles developed? What if any coaching or supervision takes place to ensure that staff are fulfilling their roles? What approaches or tools are used to support supervisions? Availability of essential commodities How would you describe the supply chain for drugs and essential products down to the health facilities and communities? What are some of the weaknesses in the system that need to be addressed? How is the coordination of the supply chain stakeholders and how does this effect the efficiency of distribution? How would you describe the national and provincial supply chain data management, quantification, procurement and distribution? What are the major challenges and how can they be addressed? What training has been received regarding supply chain management including forecasting needs, distribution, tracking and inventory management of supplies and drugs? At the zonal and health facility level, have any assessments been conducted of the supply chain? What is the capacity at the zonal level to report, forecast and quantify supply chain needs? What are the major bottlenecks in getting essential supplies to health zones? Where are the supplies stored and what are the storage conditions? What is the capacity at the health facility level to manage inventory and report needs for supplies on a regular basis? What training have zonal and facility level staff received on quantification of supply chain needs and reporting and inventory management? What stock and data management tools are available? What supervisory visits are undertaken at the zonal and health facility level to ensure that supply chain management, quantification and requisition is carried out? How is transport of essential medicine from health zones to health facilities presently conducted? Is private or public transport used? How are transport costs supported? Have there been any recent changes in transport of supplies? Collaboration between central and decentralized levels How is information sharing related to program activities and lessons learned currently carried out? What are the weaknesses and strengths in the current mechanism to share information? What meetings or workshops do you currently participate in that allow you to coordinate activities and share information that could potentially impact on national policies and strategies? Are these forums sufficient or could they be improved? Staff motivation and needs What motivates government health employees to perform their work well? What sort of potential is there for work advancement? How would you assess the way that government officials are compensated? What is needed to improve job performance? Baseline Report 371 What are the capacity building needs in your offices? How do you think that training opportunities would strengthen work performance? To what extent do you think that people in your office are satisfied with their work? 372 The Impact of USAID’s Integrated Health Program in the DRC Focus Group Discussion Guide General When was the last time you went to the health center to access services? What was the reason for your visit? When did you last go to the health center to obtain health services for your young child? What services were obtained? Why did you select the health center for care? What are some of the challenges in obtaining care at the health center? What health services are offered at the health center for young children? How would you describe the child health care services offered at the health center? How could the quality be improved? Practice of priority healthy behaviors at the individual, household and community levels/use of facility and community-based health services Where do you get information about health care for young children such as treatment or prevention related to malaria, diarrhea or pneumonia or child vaccinations? If the health center is mentioned, when is information shared? What type of information is shared? How often do you receive this type of information? To what extent do you follow the advice provided? Why or why not? Have you ever been invited to the health center to learn about the facilities being offered? If so, please explain. Do people ever come to your home providing information on child health? If so, who are these people and how often do they visit? What type of information do they share and who in the household do they share it with? To what extent do you follow the advice provided? Why or why not? Are there other ways that you receive information on child health care? For instance, do you ever participate in community meetings where information is disseminated by health workers or community leaders? If so, how often and what information is provided? Has this information changed any of your behaviors or practices related to child health care? If so, please give us an example. Are any campaigns related to child health care offered in your area? If so, how often and what services are provided? Who in the community is eligible to receive these services? Do you know of any local community organizations or groups providing information on child health care? Is information ever shared in schools or churches? If so, please describe who shares the information and the type of information presented. Are there other ways that you have received information on child health care? How many of you have a television? Radio? Mobile phone? Have you even received information related to child health through any of the television, radio or phone? If so, what type of information have you received? What do you think about the way this information was provided? What do you think about the content of the information? Has this information changed any of your behaviors or practices related to child health care? If so, please give us an example. Baseline Report 373 Where do you generally get treatment for your children when they have malaria? Diarrhea? Pneumonia? Are there other places where you can receive treatment for these illnesses? Are there treatment services offered at the community level through the health center? What guides decision making related to careseeking and who in your household is involved? What do you know about child vaccinations? How often do you have your children vaccinated? Where does your child get vaccinated? Are there any challenges to getting your young children vaccinated? Please explain. Do you have a net in the household? If so, when and where did you get it? How and by whom is it used? Sociocultural barriers to the use of health services and the practice of key healthy behaviors Have any of you ever been invited to participate in community discussions about the use of the health center and barriers in accessing the health center? How about discussions around the practice of healthy behaviors such as rapid treatment for diarrhea, vaccinations for young children, or placing your child under a net at night? If so, what exactly was discussed? What in your view are the biggest obstacles to using the health facilities for treatment of childhood illnesses? How many of you attend well baby visits? Why or why not? Why do certain members of your community choose not to attend these sessions? Are most young children in your community fully vaccinated? What are some of the challenges caregivers face in getting their young children vaccinated? How often and by whom are nets distributed? Who is eligible to receive nets when they are distributed? Do most people in your community place young children under nets at night? Why or why not? 374 The Impact of USAID’s Integrated Health Program in the DRC Observation of the health center infrastructure and supplies Yes No Comments Has the health center been renovated within the last five years? Is electricity available in the health center? Is there a separate area of the building where child health services are provided? Is an infant treatment table available? Is an infant scale and salter weighing scale with trousers available? Is a weighing scale and height measure available? Are growth monitoring kits available? Is sterilizing equipment available? Does the health provider have key instruments such as a stethoscope, thermometer, and timer? Are essential medications available such as zinc, ORS, amoxicillin, ACTs and other malaria drugs? Are treated mosquito nets available? Are rapid diagnostic kits for malaria available? Are all essential vaccinations available? Is there a refrigerator which is functioning? Are fee schedules posted? Baseline Report 375 Are there educational materials such as posters on the walls? Are there BCC materials/aides for the health workers to use? Is there evidence of activities to discourage fraud and increase transparency such as a hotline or a complaint box? We plan to take photographs of the exterior and interior of the health facilities, equipment and materials to record conditions in the health centers. Photographs will not be taken of clientele including caregivers and children observed during the health provider-caregiver interactions. 376 The Impact of USAID’s Integrated Health Program in the DRC Interviewer and interview location I. Identify of research assistant First name Last name Date Research assistant ……………………………… ……………… ….../……/…… II. Place of interview Reference hospital/Health center or post ……………………………………………………………………………………. Name of structure ……………………………………………………………………………………. Background information on the health provider observed Age ______ years Maximum schooling attained Primaire Secondaire Supérieur Training of the health worker observed Title Physician State nurse Midwife Other ……………………………. Years working in this position ______ Years Participation in training focusing on treatment for childhood illnesses? Yes No If yes, how recently? Illnesses focused on during the training? ARI Diarrhea Malaria Others ………………………….….. Length of time of last training attended? 1 day or less 2 – 4 days More than 4 days Observations of health provider/caregiver interaction Baseline Report 377 Waiting area and triage Yes No Comments At the hour that treatment services became available, were caregivers waiting to see the health worker? Was there a designated waiting area for caregivers and sick children? Was this area separate from the area where well baby services are carried out? Were there seats available and were there enough seats for all of the caregivers and patients? Was triage carried out to ensure that more serious cases were examined first? Did caregivers have to wait more than 15 minutes before the child was seen for treatment? Was the waiting area clean and orderly? Were there educational materials such as posters on the walls of the waiting area? Were sick children (other than those who were seen earlier due to the seriousness of their condition) seen in the order that they arrived at the health center? Were any caregivers/children sent away without being treated? 378 The Impact of USAID’s Integrated Health Program in the DRC Health provider interactions with the caregiver and child Yes No Comments Did the health worker greet the caregiver? Did the health worker ask for the name of the child? Did the health worker ask questions regarding the presence of fever and the duration? Did the health worker ask questions on the presence of cough and the duration? Did the health worker ask questions on the presence of diarrhea, frequency and duration and whether there is mucus or blood in the stool? Did the health worker ask questions about the treatment given prior to bringing the child to the health facility? Did the health worker determine the age and weight of the child? Did the health worker physically examine the child? Did the health worker follow a treatment protocol when diagnosing the illness and providing treatment recommendations? Did the health worker explain the diagnosis and treatment regimen? Did the health worker provide medication or a prescription? In the case of diarrhea, did the health worker provide ORS sachets and zinc tablets? In the case of diarrhea, did the health worker provide Baseline Report 379 counseling on the administration of ORS and zinc? Did the health worker provide counseling on feeding the young child ? Did the health worker request that the caregiver return to the health facility immediately if illness signs worsen? Did the health worker ask questions to confirm that the caregiver understood the diagnosis and home treatment and ask the caregiver whether s/he had any additional questions ? Did the caregiver ask the health provider any questions? Did the caregiver appear to understand the information provided by the health worker? Did the health provider request that the caregiver bring the child back for a follow up visit? Was the session interactive? Was the health worker respectful towards the caregiver and the child? Was the consultation carried out in a setting where privacy was maintained? Other comments/observations 380 The Impact of USAID’s Integrated Health Program in the DRC Appendix 3. Evaluation Team Members Evaluation team David Hotchkiss, PhD is a Professor in the Department of Global Community Health and Behavioral Sciences at the Tulane School of Public Health and Tropical Medicine. His current research focuses on the monitoring and evaluation of health systems strengthening strategies in low- and middle-income countries including: health care financing mechanisms; health workforce strategies; and routine health information systems. Dr. Hotchkiss has been conducting health systems research in the DRC since 2013. As principle investigator, he is responsible for overseeing all phases of the evaluation, including research design, data collection, data analysis, and report writing. Janna Wisniewski, PhD is a Research Assistant Professor in the Department of Health Policy and Management at the Tulane School of Public Health and Tropical Medicine. Dr. Wisniewski specializes in operations research and evaluation of health system strengthening projects and has been conducting research in the DRC since 2015. As a co-investigator, she is responsible for overseeing activities and analysis related to the quantitative component of the study, including the training of supervisors and data collectors responsible for the facility, health zone office, and provincial health office surveys, the compilation and analysis of DHIS2 data, and writing reports based on the study findings. Lauren S. Blum, PhD, MPH is an Adjunct Professor at the Tulane School of Public Health and Tropical Medicine and a consultant to D4I. Dr. Blum has over 20 years of experience designing, implementing, and analyzing qualitative research focusing on sociocultural factors and behaviors related to child and maternal health and nutrition, infectious disease transmission, and WASH in low-income communities in Africa and Asia. Since 2013, she has collaborated with Tulane University on evaluations of health systems strengthening projects in the DRC. Dr. Blum has worked in the DRC over a period spanning several decades and speaks French. Dr. Blum is responsible for overseeing all activities and analysis related to the qualitative research. Paul Lusamba Dikassa, MD, PhD is a Professor in the Department of Epidemiology and Biostatistics at the School of Public Health, University of Kinshasa. He has been conducting research for several years in DRC. As co-investigator, he participated in the elaboration of the evaluation protocol. On the ground, he is additionally responsible for organizing and coordinating the local research team and supervising the field activities related to this project, including the pretesting of the quantitative data collection instruments and the training of field supervisors and data collectors. Paul Lusamba also participates in the drafting of study reports. Eva Silvestre, PhD is a Clinical Assistant Professor in the department of Global Community Health and Behavioral Sciences and has worked in the areas of HIV prevention, treatment, and communication, reproductive health, health information systems strengthening, and technology use in healthcare settings. She has experience using both quantitative and qualitative methods for program evaluation. She has worked both domestically and internationally, including living in Ethiopia working with a Regional Health Bureau to improve data quality and use and working with community organizations in New Orleans. Her work currently focuses on health information system strengthening in low- and middle￾income countries. She is currently the principal investigator for MEASURE Evaluation and Data 4 Impact. Baseline Report 381 Charles Stoecker, PhD is an Associate Professor in the Department of Health Policy and Management at the Tulane School of Public Health and Tropical Medicine. Dr. Stoecker holds a doctorate in economics and functions as an applied econometrician on the research team. He brings experience with practical applications of differences-in-differences models, propensity score matching models, doubly robust estimators, and issues of multiple imputation to the project. Matt Worges, MSc is a doctoral candidate in the department of Tropical Medicine at the Tulane School of Public Health and Tropical Medicine. Mr. Worges' current and previous work has involved critical evaluation of survey methodologies used in low-resource settings, monitoring and evaluation of projects designed to improve malaria diagnostics, and conducting broad assessments of the state of malaria case management practices at health facilities in low and middle income countries. As a team member on the D4I project, he is responsible for coding baseline survey questionnaires in Open Data Kit, monitoring the data collection process and reporting out issues to the field-based supervisors in near real-time, managing incoming data streams to construct databases, and contributing to the analysis and interpretation of those data. Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Title Organization Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date Appendix 4. Signed Disclosures of Conflicts of Interest Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Title Organization Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date Janna Wisniewski Research Assistant Professor Data for Impact 7200AA18LA00008 USAID’s Integrated Health Program in the Democratic Republic of the Congo Janna Wisniewski 03/31/2020 Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Lauren S. Blum Title Medical Anthropologist Organization Data for Impact Evaluation Position? D Team Leader Ii] Team member Evaluation Award Number (contract 7200AA 18LA00008 or other instrument) USAID Project(s) Evaluated {Include USAID's Integrated Health Program in the Democratic project name(s), implementer Republic of the Congo name(s) and award number(s), if applicable) I have real or potential conflicts of D Yes Ii] No interest to disclose. If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USA/0 operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the praject design or previous iterations of the project. 4. Current or previaus wark experience ar seeking employment with the USAID aperating unit managing the evaluatian or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the informa io other than that for which it was furnished. Signature Date April 2, 2020 Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Paul Samson Lusamba Dikassa Title Professor Organization Data for Impact Evaluation Position? □ Team Leader � Team member Evaluation Award Number (contract 7200AA 18LA00008 or other instrument) USAID Project(s) Evaluated {Include USAID's Integrated Health Program in the Democratic project name(s}, implementer name(s) and award number(s), if Republic of the Congo applicable} I have real or potential conflicts of LJ Yes ■ No interest to disclose. If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USA/0 operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular µrujecls u11u uryu11i,uliu11s /Jei11y evuluuleci llrul could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Eva Silvestre Title Tulane team lead Organization Data for Impact Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) 7200AA18LA00008 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID’s Integrated Health Program in the Democratic Republic of the Congo I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest mayinclude, but are not limited to: 1. Close family member who is an employee ofthe USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or issignificant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as anindustry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluatedthat could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Eva A. Silvestre Digitally signed by Eva A. Silvestre DN: cn=Eva A. Silvestre, o=Tulane University, ou=Global Community Health & Behavioral Sciences, email=esilvest@tulane.edu, c=US Date: 2020.04.03 11:18:34 -05'00' Date 4/3/2020 Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Charles Stoecker Title Associate Professor Organization Data for Impact Evaluation Position? □ Team Leader � Team member Evaluation Award Number (contract 7200AA 18LA00008 or other instrument) USAID Project{s) Evaluated {Include USAID's Integrated Health Program in the Democratic project name(s), implementer name(s) and award number(s), if Republic of the Congo applicable) I have real or potential conflicts of D Yes ■ No interest to disclose. If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unir managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the proiect(s) beina evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date Charles Stoecker 3-31-2020 Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Matt Worges Title Tulane University Organization Data for Impact Evaluation Position? □ Team Leader � Team member Evaluation Award Number (contract 7200AA 18LA00008 or other instrument) USAID Project(s) Evaluated {Include USAID's Integrated Health Program in the Democratic project name(s), implementer name(s) and award number(s), if Republic of the Congo applicable) I have real or potential conflicts of D Yes ■ No interest to disclose. If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Matthew Scott Worges Date 04/03/2020 This publication was produced with the support of the United States Agency for International Development (USAID) under the terms of the Data for Impact (D4I) associate award 7200AA18LA00008, which is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, in partnership with Palladium International, LLC; ICF Macro, Inc.; John Snow, Inc.; and Tulane University. The views expressed in this publication do not necessarily reflect the views of USAID or the United States government. TRE-20-28 D4I ISBN 978-1-64232-246-0 Data for Impact University of North Carolina at Chapel Hill 123 West Franklin Street, Suite 330 Chapel Hill, NC 27516 USA Phone: 919-445-9350 | Fax: 919-445-9353 D4I@unc.edu http://www.data4impactproject.org