This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago and authored by Luis A. Camacho, Senior Research Scientist at NORC, Kim Y. Dionne, Assistant Professor of Political Science at the University of California, Riverside, and Ingrid Rojas-Arellano, Principal Research Analyst at NORC. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. IMPACT EVALUATION OF USAID/DRC INTEGRATED GOVERNANCE ACTIVITY MIDLINE REPORT DRG Learning, Evaluation, and Research (DRG-LER) Activity Prepared under Contract No.: GS-10F-0033M / AID-0AA-M-13-00013, Tasking N035 DRG LEARNING, EVALUATION, AND RESEARCH ACTIVITY IMPACT EVALUATION OF USAID/DRC INTEGRATED GOVERNANCE ACTIVITY MIDLINE REPORT DRG Learning, Evaluation, and Research (DRG-LER) Activity TASKING N035 JUNE 2020 Prepared under Contract No.: GS-10F-0033M/AID-0AA-M-13-00013 SUBMITTED TO: Brandy Witthoft, COR SUBMITTED BY: Luis A. Camacho, Principal Investigator (NORC at the University of Chicago) Kim Yi Dionne, Principal Investigator (University of California, Riverside) Ingrid Rojas-Arellano, Senior Research Analyst (NORC at the University of Chicago) CONTRACTOR: NORC at the University of Chicago Attention: Renee Hendley, Program Manager Bethesda, MD 20814 Tel: 301-634-9489 E-mail: Hendley-Renee@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | i TABLE OF CONTENTS ACRONYMS III EXECUTIVE SUMMARY 1 1. INTRODUCTION 3 2. BACKGROUND 4 2.1 HEALTH SERVICE PROVISION IN THE DRC 4 2.2 THEORY OF CHANGE AND RESEARCH QUESTIONS 6 2.3 CAPACITY BUILDING TRAINING AND PUBLIC SERVICE DELIVERY 6 2.4 INCREASING AWARENESS AND MOBILIZATION THROUGH COMMUNITY SCORECARDS 8 3. METHODOLOGY 10 3.1 INTERVENTIONS 10 3.2 SCOPE 12 3.3 RANDOMIZATION 13 3.4 DATA 13 3.4.1 BASELINE DATA COLLECTION 14 3.4.2 MIDLINE DATA COLLECTION 15 3.5 OUTCOMES 15 3.5.1 HEALTH SERVICE DELIVERY 15 3.5.2 PERCEPTIONS ABOUT HEALTH CARE PROVISION AND GOVERNANCE 17 3.5.3 HEALTH SEEKING AND PROMOTING BEHAVIORS 17 3.5.4 HEALTH OUTCOMES 18 3.6 ESTIMATION 19 4. FINDINGS 20 4.1 TREATMENT UPTAKE AND IMPLEMENTATION 20 4.2 HEALTH SERVICE DELIVERY 22 4.3 PERCEPTIONS ABOUT HEALTH CARE PROVISION AND GOVERNANCE 23 4.4 HEALTH SEEKING AND PROMOTING BEHAVIORS 24 4.5 HEALTH OUTCOMES 24 4.6 HETEROGENEOUS EFFECTS 24 5. CONCLUSIONS 26 REFERENCES 28 APPENDIX A: POWER CALCULATIONS 30 APPENDIX B: RANDOMIZATION AND BALANCE CHECKS 32 APPENDIX C: DETAILED ESTIMATION RESULTS 35 C.1 TREATMENT UPTAKE AND IMPLEMENTATION 36 C.2 HEALTH SERVICE DELIVERY 42 C.3 PERCEPTIONS ABOUT HEALTH CARE PROVISION AND GOVERNANCE 45 C.4 HEALTH SEEKING AND PROMOTING BEHAVIORS 46 C.5 HEALTH OUTCOMES 48 CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | ii LIST OF FIGURES Figure 1. CSC Intervention Summary .........................................................................................................12 Figure 2. Impact Evaluation Design..............................................................................................................13 Figure C.1.1. Percentage of Audit Informants that Attended Training ......................................................37 Figure C.1.2. Number of Topics Mentioned by Audit Informants ..............................................................37 Figure C.1.3. Length of Training (in Days) According to Audit Informants..............................................38 Figure C.1.4. Percentage of CODESA Survey Respondents that Attended Training .............................39 Figure C.1.5. Number of Topics Mentioned by CODESA Survey Respondents.....................................39 Figure C.1.6. Length of Training (in Days) According to CODESA Survey Respondents.....................40 LIST OF TABLES Table A.1. Estimated Minimum Detectable Effect Sizes............................................................................31 Table B.1. Variables Used for Blocking .........................................................................................................33 Table B.2. Balance across Groups..................................................................................................................33 Table C.1.1. CB Intervention Uptake Estimation Results............................................................................36 Table C.1.2. CSC Intervention Uptake Estimation Results..........................................................................41 Table C.1.3. Manipulation Checks......................................................................................................................41 Table C.2.1. Health Service Delivery ................................................................................................................42 Table C.2.2. Health Service Governance .........................................................................................................44 Table C.2.3. Households’ Experiences Visiting Local Health Center........................................................44 Table C.3.1. Households’ Assessments about Health Care Services........................................................45 Table C.3.2. Opinions about Health Center, Health Governance, and CODESA Performance........46 Table C.4.1. Under-five Immunization ..............................................................................................................46 Table C.4.2. Health Seeking Behavior...............................................................................................................46 Table C.4.3. Antenatal and Postnatal Care......................................................................................................47 Table C.5.1. Health Outcomes...........................................................................................................................48 CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | iii ACRONYMS ANC Antenatal care ASSP Access to Primary Health Care program (Accès aux Soins de Santé Primaires) CAPI Computer-assisted personal interviewing CB Capacity building CB+CSC Capacity building training plus community scorecard intervention CDCS Country Development Cooperation Strategy CODESA Health Development Committee (Comité de Développement de l’Aire de Santé) CSC Community score card CSO Civil Society Organization DHS Demographic and Health Survey DPS Provincial Health Division (Division Provinciale de la Santé) DRC Democratic Republic of Congo HA Health area (aire de santé) HDI Human Development Index HIV Human Immunodeficiency Virus HZ Health zone (zone de santé) ICC Intra-cluster correlation coefficient IE Impact evaluation IGA Integrated Governance Activity IHP Integrated Health Program MDES Minimum detectable effect size MSP Ministry of Public Health (Ministère de la Santé Publique) ODK Open Data Kit PMA Minimum service package (Paquet minimum d’activités) RCT Randomized controlled trial SNIS National System of Health Information (Système National d’Information Sanitaire) STI Sexually-transmitted infection UNDP United Nations Development Program USAID United States Agency for International Development USAID/DRC United States Agency for International Development’s Mission in the Democratic Republic of Congo WHO World Health Organization CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 1 EXECUTIVE SUMMARY Public service provision is generally poor in fragile states and the Democratic Republic of Congo (DRC) is no exception. The United States Agency for International Development’s Mission in the DRC (USAID/DRC) is funding the Integrated Governance Activity (IGA), a five-year (2017-2022) initiative implemented by DAI seeking to improve public services and establish trust between citizens and government. More specifically, IGA supports leaders at the national, provincial, and local levels to improve government services and trains civil society and private-sector groups to better advocate their needs to elected officials in the DRC provinces of Haut Katanga, Kasai Central, Lualaba, and Sud Kivu. IGA complements sectoral programming by improving the incentive structures of service providers, enhancing demand for better services, improving citizens’ ability to hold service providers accountable, and strengthening the capacity of government entities, service providers, and civil society organizations. In partnership with USAID’s Center of Excellence on Democracy, Human Rights, and Governance (DRG), USAID/DRC, and DAI, NORC at the University of Chicago (NORC) designed an impact evaluation (IE) to measure whether governance interventions integrated with health programming can help improve health outcomes. Our research design considers both supply- and demand-side factors in promoting good governance. Thus, this IE measures the impacts of two interventions: (1) capacity building (CB) trainings designed to strengthen the resource management skills of local health service providers; and (2) a community scorecard (CSC) intervention aimed at increasing citizens’ awareness of health service provision and their ability to mobilize and hold service providers accountable. The CB intervention trains providers and community health development committee (comité de développement de l’aire de santé, CODESA) members; the CSC intervention engages these groups as well as citizens. These interventions seek to improve health service delivery, perceptions of health services and health governance, health seeking and promoting behaviors, and, ultimately, household and community health outcomes. Health areas (HAs, aires de santé) are the lowest units of the DRC public health care system and they include a lead health facility and in some cases additional facilities. Each HA also has a CODESA serving as a link between citizens and medical providers. This IE is designed as a randomized controlled trial (RCT) of 182 eligible rural HAs in Haut Katanga and Sud Kivu that were already receiving USAID-supported health programming. We randomly assigned these HAs to one of three arms: 1. CB training (CB only) 2. CB training plus community scorecard intervention (CB+CSC) 3. No governance programming (control) This design allows identification of the impact of the CB training alone, as well as the added effect of CSC meetings in places receiving the CB intervention. To gauge the impacts we collect data through a panel household survey, health facility audits, and a survey of CODESA members at baseline (April￾August 2018), midline (February-August 2019), and endline (originally planned for May-July 2020 and now tentatively postponed to the second half of 2020 due to the COVID-19 pandemic). This report uses baseline and midline data to estimate the interventions’ short-term impacts and to examine treatment uptake and implementation. The midline data was collected up to one year after the interventions: 120 to 360 days after the CB training sessions, and 60 to 180 days after the CSC meetings. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 2 We find mixed uptake results. The CB training was implemented roughly in line with the design, but it did not produce expected changes in knowledge questions among health center staff. In turn, the CSC meetings do not appear to have been adequately publicized and widely attended. Moreover, the CB+CSC treatment did not produce improvements in knowledge questions among CODESA and household survey respondents. Our analysis indicates that, in the short term, neither the CB training alone nor the CB training combined with the CSC intervention substantively improved health service delivery, citizens’ perceptions about health care provision and governance, citizens’ health seeking and promoting behaviors, or health outcomes. Below we summarize the specific outcomes we analyze within each grouping: • Health service delivery: Service offering and utilization; financing and resource management and availability; staffing, equipment, and facilities, other health center-related outcomes; health service governance; and household’s experiences visiting the local health center. • Perceptions of health services and health governance: Households’ assessments of health care services; household survey respondents’ opinions about health center and health governance; and households’ and CODESA members’ opinions on CODESA performance. • Health seeking and promoting behaviors: Immunization; health seeking behavior in response to diarrhea in under-five children; health seeking behavior in response to illness in five-and-over household members; and antenatal and postnatal care. • Health outcomes: Neonatal mortality rate; child mortality rate; diarrhea incidence; stillbirth rate; maternal mortality rate; and illness incidence in five-and-over population. Our analysis also indicates that short-term treatment effects did not vary across different groupings of study participants. Contrary to our expectations, we find no evidence that treatments produced short-term improvements in HAs where health care services and outcomes were particularly defective at baseline, or in HAs with lower levels of social fragmentation and wealth inequality, as these are impediments to social coordination and mobilization. As we prepare for endline data collection, we remain hopeful that our analysis of endline data will show that the CB training alone and the CB training combined with the CSC intervention did have effects in the medium term. If that were the case, we would conclude that treatment effects needed additional time to materialize. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 3 1. INTRODUCTION USAID/DRC’s IGA is a five-year (2017-2022) initiative implemented by DAI that aims to improve public services and establish trust between citizens and government. More specifically, IGA supports leaders at the national, provincial, and local levels to improve government services and trains civil society and private-sector groups to better advocate their needs to elected officials. IGA seeks to improve service provision in key sectors like health and education through governance programming that overlaps with other USAID sectoral programs. In collaboration with USAID and IGA, the NORC team designed an IE to measure whether governance interventions integrated with health programming can help improve health outcomes. Our research design considers both supply- and demand-side factors in promoting good governance. We combine a CB intervention with an information and mobilization intervention recognizing both the need for increased state capacity to deliver services and the significant power asymmetries between service providers and users in the context we study. This IE includes two interventions: (1) CB trainings designed to strengthen the resource management skills of local health service providers; and (2) a CSC intervention aimed at increasing citizens’ awareness of health service provision and their ability to mobilize and hold service providers accountable. The CB intervention trains providers along with the members of the area’s CODESA; the CSC intervention engages these groups as well as citizens. These interventions seek to improve health service delivery, citizens’ perceptions of health care provision and governance, health seeking behaviors, and, ultimately, household and community health outcomes. Health service provision in the DRC is deconcentrated. HAs are the lowest units of the DRC health care system and the units that are randomly assigned to interventions in this IE. These special purpose territorial units contain between 5,000 and 10,000 people in a radius of up to 15 km in rural areas, and between 10,000 and 15,000 people in urban areas. As the lowest units of the health care system, HAs are responsible for providing primary care as well as a small selection of specialized services. Each HA has a lead health facility and in some cases other additional facilities. This IE is designed as an RCT of 182 eligible rural HAs in Haut Katanga and Sud Kivu that were already receiving USAID-supported health programming. We randomly assigned these HAs to one of three arms: 1. CB training (CB only) 2. CB training plus community scorecard intervention (CB+CSC) 3. No governance programming (control) This design allows identification of the impact of the CB training alone, as well as the added effect of CSC meetings in places receiving the CB intervention. To gauge the impacts we collect data through a panel household survey, health facility audits, and a survey of CODESA members at baseline (April￾August 2018), midline (February-August 2019), and endline (originally planned for May-July 2020 and now tentatively postponed to the second half of 2020 due to the COVID-19 pandemic). This report uses baseline and midline data to estimate the interventions’ short-term impacts and to examine treatment uptake and implementation. The midline data was collected up to 12 months after the treatments: 4 to 12 months after the CB training, and 2 to 6 months after the CSC events. The remainder of this report is organized as follows. Section 2 provides background information about health service provision in the DRC and introduces IGA and its theory of change along with CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 4 our research questions. This section also reviews the literature that motivates our study. Section 3 describes the methodology including the interventions, scope, randomization, data collection, and outcomes. Section 4 presents our impact estimates at midline after examining treatment uptake. Section 5 concludes. 2. BACKGROUND 2.1 HEALTH SERVICE PROVISION IN THE DRC Public service delivery is usually poor in developing countries, with high rates of absenteeism by frontline providers of health and education,1 misuse of public funds, 2 and essential services lacking in basic infrastructure.3 The situation in post-conflict states is even more dire. Periods of conflict and instability lead to additional neglect of infrastructure and severe disruption of service delivery, and states struggle to provide basic services and (re)gain legitimacy in the aftermath. The inability or unwillingness of post-conflict states to carry out essential functions, like administering justice, guaranteeing public security, and providing health care and education, undermines their legitimacy. Namely, the failure to provide basic services can erode public faith in government and lead to further instability.4 Citizens suffer the consequences of these states’ failure to provide basic services in very concrete ways. Gates et al. 5 estimated that 1.4 billion people—roughly a quarter of the world’s population— live in conflict and post-conflict countries. Yet, these countries account for 29 percent of the world’s population living in poverty, 34 percent of the world’s undernourished population, 35 percent of births given in the absence of health personnel, and 56 percent of the world’s population without primary education. These statistics make clear how citizens in these countries are disproportionately under-served. The situation in the DRC is no different. For example, in the most recent Demographic and Health Survey (DHS) in DRC, just over three-quarters of women (76 percent) reported at least one serious problem in accessing health care. In rural areas, this proportion is even larger (82 percent), with significant variation across provinces: 87 percent of women in Kasai Occidental report obstacles accessing health care compared to just 54 percent of women in Kinshasa. 6 These and related challenges manifest in the DRC's low ranking in the world on the United Nations Development 1 Chaudhury, N., Hammer, J., Kremer, M., Muralidharan, K., and Rogers, F.H. 2006. “Missing in action: teacher and health worker absence in developing countries.” Journal of Economic perspectives, 20(1), 91-116. 2 Reinikka, R. and Svensson, J., 2004. “Local capture: evidence from a central government transfer program in Uganda.” The Quarterly Journal of Economics, 119(2), 679-705. 3 Bold, T., Svensson, J., Gauthier, B., Maestad, O., and Wane, W. 2011. “Service Delivery Indicators: Pilot in Education and Health Care in Africa.” CMI Report 2011/08. 4 Brinkerhoff, D.W. 2005. “Rebuilding governance in failed states and post‐conflict societies: core concepts and cross‐ cutting themes.” Public Administration and Development: The International Journal of Management Research and Practice, 25(1), 3-14. 5 Gates, S., Hegre, H., Nygård, H.M., and Strand, H., 2012. “Development consequences of armed conflict.” World Development, 40(9), 1713-1722. 6 Ministère du Plan et Suivi de la Mise en œuvre de la Révolution de la Modernité (MPSMRM), Ministère de la Santé Publique (MSP), and ICF International, 2014. “Democratic Republic of Congo Demographic and Health Survey 2013-14: Key Findings.” Rockville, Maryland, USA: MPSMRM, MSP, and ICF International. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 5 Program’s (UNDP) Human Development Index (HDI): 179th out of 189 countries measured in 2019. 7 A 2012 Tetra Tech ARD study attributed the state's failure to deliver public services to a breakdown in the social contract between Congolese society and the state. 8 In a more recent background paper, Englebert pointed to increased public transparency and accelerated decentralization as necessary vehicles of reform in DRC. 9 USAID/DRC has therefore placed governance at the center of its Country Development Cooperation Strategy (CDCS), whose goal is to support a long-term transition to more effective and empowering development in the DRC. USAID/DRC’s current approach to governance programming integrates governance into its sectoral programming. IGA is a five-year program (January 2017-January 2022) that was designed in line with USAID/DRC’s integrated approach and is implemented by DAI. The activity works with government and civil society at national, provincial, and local levels to enhance service delivery by improving governance. IGA is being implemented in the provinces of Haut Katanga, Kasai Central, Lualaba, and Sud Kivu, and focuses on services related to health, education, and economic growth. IGA complements sector-focused activities by reorienting misaligned incentives, enhancing demand for better services and capacity to hold service providers accountable among citizens, and strengthening the capacity of government entities, service providers, and civil society organizations (CSOs). In the health sector at the local level, IGA works with health zones (HZs, zones de santé) and CSOs. HZs are the decentralized units responsible for providing health services and comprise several HAs with one or more health facilities. IGA also works with health ministries at the national and provincial level, as well as with Provincial Health Divisions (DPSs, divisions provinciales de la santé). HAs are the lowest units of the DRC health care system and the units that are randomly assigned to interventions in this IE. In rural areas, the lead health facility in a HA is generally a health center (centre de santé) administered by a head nurse (infirmière titulaire). The head nurse is also responsible for overseeing service provision in the HA as well as for collecting public health and service utilization information from all health facilities in the HA for reporting to the HZ at the end of each month. According to Ministry of Public Health (Ministère de la Santé Publique, MSP) regulations, each HA should have at least one facility at the health center level. Health centers can be public, faith-based, or private. Health centers ought to provide a minimum service package (paquet minimum d’activités, PMA) in the areas of preventive, curative, and rehabilitative care, and health promotion. Regulations also mandate standards for health service provision, including health center staffing, availability of medicines and equipment, infrastructure, and hours of operation. According to the same regulations, each HA has a community health development committee (comité de développement de l’aire de santé, CODESA). CODESAs comprise a board of five members and four working groups. CODESA members are elected by the community to serve three-year 7 The HDI is a summary measure that assesses long-term progress on three dimensions of human development: a long and healthy life, access to knowledge, and a decent standard of living. See scores and rankings online at: http://hdr.undp.org/en/data. 8 Tetra Tech ARD. 2012. “Democracy, Human Rights, and Governance Assessment of the Democratic Republic of the Congo: Final Report.” Report commissioned by USAID. Burlington, Vermont. 9 Englebert, P. 2014. “Democratic Republic of Congo: Growth for All? Challenges and Opportunities for a New Economic Future.” The Brenthurst Foundation Discussion Paper 6/2014. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 6 terms and regulations. At least 30 percent of CODESA members are required to be women. CODESAs have three distinct sets of roles: • Vis-à-vis the community: Represent the interests of the community before the health center; promote community ownership over the health center and public health; and promote public health through information and sensitization campaigns, and community mobilization. • Vis-à-vis the health center: Oversee health center performance; co-manage medicine stocks, finances, materials, equipment, and infrastructure; and promote effective collaboration between the community and health care providers. • Vis-à-vis the HZ: Provide community inputs for the HZ’s annual action plan; and participate in meetings related to HZ administration and management. According to MSP regulations, CODESAs are supposed to meet at least once per month with health care providers to co-manage health centers. They are also mandated to meet at least once per year with providers and the community at large to assess the performance of the health center and discuss public health concerns, as well as to develop a community action plan to address shortcomings. The extent to which CODESAs operate according to regulations is limited, however. During the design phase, DAI reported that in several HAs, CODESAs are inactive or CODESA members had been co-opted by health care providers, offering preferential access to health services and medicines in exchange for less or more favorable oversight over the health center’s activities. We also received reports that CODESA members are not always regularly elected and that annual CODESA-providers-community meetings typically do not take place. 2.2 THEORY OF CHANGE AND RESEARCH QUESTIONS The broad aim of IGA is to increase the ability and willingness of decision-makers to improve the quality and quantity of public service provision through improvements in management that incorporate citizen engagement. The following theory of change underpins IGA programming: If the government of the DRC is incentivized to improve the underlying governance structures that support service delivery and if citizens and civil society are empowered to define and monitor these services, then the government of the DRC can better meet the challenge of delivering quality services to its citizens in health, education, and economic development in a more equitable, transparent, and sustainable manner thereby strengthening the democratic social contract between citizens and the state. Accordingly, the IE seeks to answer two main research questions related to this theory of change and in the context of health services. The research questions respectively focus on the supply-side and demand-side of service provision. • First, on the supply side, we examine whether improving service providers’ and CODESA members’ management capacities through a CB training intervention results in better health￾related knowledge, attitudes, and behaviors among citizens, and in enhanced health service delivery and health outcomes. • Second, on the demand side, we examine whether disseminating information and promoting community mobilization to improve health service delivery through a CSC intervention enhances the impacts of the CB training intervention. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 7 2.3 CAPACITY BUILDING TRAINING AND PUBLIC SERVICE DELIVERY Development practitioners see CB as essential in improving performance in the public sector.10 The underlying assumption of CB training interventions is that government officials and service providers lack the necessary knowledge and skills to perform their management duties well. Even in resource￾constrained environments, CB trainings can improve service delivery by enhancing government officials’ and service providers’ ability to gather information about citizen needs, monitor front-line personnel for efficacy and effectiveness, ensure adequate resources to meet the public’s needs, and so forth. Put another way, public servants do not willfully choose to under-provide public services, but instead work inefficiently given available resources because they lack management knowledge and skills. Even if a large share of development assistance goes to providing CB training to government agencies and service providers, there is a dearth of rigorous research and evaluations on the effectiveness of such interventions.11 Non-experimental research from the health sector finds CB interventions have positive effects. In a study conducted in Liberia, a CB intervention led to improved health facility management, as measured in surveys asking about self-reported management skills.12 In another study of decentralization of health services in Pakistan, a CB intervention increased “institutional capacities,” which was defined as having adequate resources and processes to deliver health services, as measured in an average of 21 survey questions covering service delivery management.13 However, these and other studies of CB interventions in the health sector14 rely on self-reported survey measures or analysis of only districts or health workers that received interventions, and thus lack comparison. Still, there is promising evaluative research showing that management training in the health sector can empower participants with problem solving and teamwork that can lead to improved health program implementation15 and adherence to health facility performance standards.16 In related research beyond the health sector, several studies have measured the value of technical skill development—especially financial and business training—for firms and small-to-medium enterprises.17 These studies find that such training improves private sector performance, suggesting capacity development in the public sector may similarly strengthen public service delivery. 10 See, for example, World Bank Operations Evaluation Department. 2005. “Capacity Building Capacity Building in Africa: An OED Evaluation of World Bank Support.” OED No. 34351. Washington, DC: The International Bank for Reconstruction and Development / The World Bank. 11 Yeager, V.A. and Bertrand, J. 2016. “Putting management capacity building at the forefront of health systems strengthening: comment on ‘Management matters: A leverage point for health systems strengthening in global health.’” International Journal of Health Policy and Management, 5(2), 129. 12 Rowe, L.A., Brillant, S.B., Cleveland, E., Dahn, B.T., Ramanadhan, S., Podesta, M., and Bradley, E.H. 2010. “Building capacity in health facility management: guiding principles for skills transfer in Liberia.” Human Resources for Health, 8(1), 5. 13 Bossert, T.J., Mitchell, A.D., and Janjua, M.A. 2015. “Improving health system performance in a decentralized health system: capacity building in Pakistan.” Health Systems & Reform, 1(4), 276–284. 14 See, for example, Conn, C.P., Jenkins, P., and Touray, S.O. 1996. “Strengthening health management: experience of district teams in The Gambia.” Health Policy and Planning, 11(1), 64–71. 15 Umble, K.E., Brooks, J., Lowman, A., Malison, M., Huong, N.T., Lademarco, M., and Laserson, K. 2009. “Management training in Vietnam’s National Tuberculosis Program: an impact evaluation.” The International Journal of Tuberculosis and Lung Disease, 13(2), 238–246. 16 Kebede, S., Mantopoulos, J., Ramanadhan, S., Cherlin, E., Gebeyehu, M., Lawson, R., and Bradley, E.H. 2012. “Educating leaders in hospital management: a pre-post study in Ethiopian hospitals.” Global Public Health, 7(2), 164–174. 17 See, for example, Karlan, D., Knight, R., and Udry, C. 2015. “Consulting and capital experiments with microenterprise tailors in Ghana.” Journal of Economic Behavior & Organization, 118, 281–302. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 8 Our IE goes beyond this scholarship by evaluating the impact of a CB training intervention on intermediate outcomes related to health service delivery, citizens’ perceptions of health care provision and governance, and citizens’ health seeking behaviors, and, more importantly, downstream impacts on household and community health. We test the following hypotheses: • Hypothesis 1a (H1a): Increasing health officials’ capacity in managing resources will improve health service delivery. • Hypothesis 1b (H1b): Increasing health officials’ capacity in managing resources will improve citizens’ perceptions of health care provision and governance. • Hypothesis 1c (H1c): Increasing health officials’ capacity in managing resources will improve citizens’ health seeking behaviors. • Hypothesis 1d (H1d): Increasing health officials’ capacity in managing resources will improve household and community health outcomes. CB training may be insufficient to improve outcomes, however. Even if a CB intervention was successful in increasing government officials’ and service providers’ ability to carry out their management responsibilities, they must also be motivated to carry out their duties. In politically competitive environments, elected officials that fail to deliver face the prospect of losing political office and therefore have incentives to hold agents accountable for service provision. Similarly, in a competitive health market, service providers have incentives to provide good service if they want to retain clients seeking care. In a context like rural Eastern Congo, however, where political competition is low and health facilities have captive users, CB training might need to be paired with an intervention that increases citizens demand for improved service provision and sparks a virtuous accountability cycle. An information and mobilization intervention is a particularly attractive option to encourage accountability from the bottom up. 2.4 INCREASING AWARENESS AND MOBILIZATION THROUGH COMMUNITY SCORECARDS Interventions aimed at increasing community awareness of and mobilization around public service provision have yielded mixed results, but in some contexts have improved the quality of public services and governance in developing countries.18 Some studies have found that information alone can be insufficient to motivate citizens to action. In a randomized controlled trial (RCT) in Kenya, providing information to parents about their children’s test performance and materials about how to become more involved had no impact on parent action. 19 Likewise, in an RCT in India, informing parents on the status of education and participatory institutions was insufficient to encourage involvement in public schools.20 In their assessment, the authors wrote, “Parents may be too pessimistic about their ability to influence the system even if they are willing to take an active role, or parents may not be able to coordinate to exercise enough pressure to influence the system.” 21 Put simply, increasing information does not always solve coordination problems. 18 See, for example, Kosack, S. and Fung, A. 2014. “Does transparency improve governance?” Annual Review of Political Science, 17, 65–87. 19 Lieberman, E. S., Posner, D. N., and Tsai, L. L., 2014. “Does information lead to more active citizenship? Evidence from an education intervention in rural Kenya.” World Development, 60, 69-83. 20 Banerjee, A. V., Banerji, R., Duflo, E., Glennerster, R., and Khemani, S., 2008. Pitfalls of participatory programs: Evidence from a randomized evaluation in education in India. The World Bank. 21 Ibid, p. 5. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 9 We believe the method of communicating information to citizens could be critical in explaining how increased information can lead to improved service delivery and development outcomes. We expect citizens will act based on how they expect others might act. Particularly in a setting with a long history of state negligence and even absence in public service provision, individually informing citizens about their service entitlements and how they can intervene to receive those services may seem an empty gesture. Community meetings, however, are different. Through the public ritual of a community meeting, citizens can experience first-hand an increased understanding of service entitlements and potential actions they can take, while also witnessing others in their community learning the same things they have learned. Chwe (2003) elaborates on the power of a public ritual as it is “not just about the transmission of meaning from a central source to each member of an audience; it is also about letting audience members know what other audience members know.” 22 Citizens may be more likely to act and bureaucrats may be more likely to respond to citizen action in a state of “common knowledge.” Chwe (2015) defines common knowledge as occurring among a group of people when “everyone knows that everyone knows it, everyone knows that everyone knows that everyone knows it, and so on.”23 Common knowledge might affect how citizens understand and process newly disseminated information. An RCT among members of parliament in Benin found that providing information about politician performance had a positive impact only when the information was provided as part of a widely-disseminated civics campaign.9F 24 An implication of this is the importance of common knowledge in shaping how citizens receive informational interventions. Lack of common knowledge or inaccurate prior beliefs about the proper role of politicians might decrease public engagement due to alienation by confirming cynical depictions of politics.10F 25 Björkman and Svensson (2009) is perhaps the most promising research measuring the impact of information and mobilization interventions on health service delivery and outcomes. Through community meetings, local non-governmental organizations encouraged communities in Uganda to be more involved in health service provision and strengthened communities’ capacity to hold their local health care providers accountable. More specifically, these community meetings provided a venue to collect and disseminate information on the status of health service delivery in the respective areas. This led to the creation of an action plan, which acted as a ‘contract’ between health care providers and communities on “what needs to be done, and how, when, and by whom.”26 The researchers found large increases in health care utilization and improved health outcomes (the latter measured as reduced child mortality and increased child weight). A follow-up found that these benefits were long-lasting.27 There are two parts to the Björkman and Svensson intervention, namely information sharing and collective action, which together give us our second set of hypotheses to test: 22 Chwe, M. S., 2003. Rational ritual: culture, coordination, and common knowledge. Princeton University Press. 23 Chwe, M. S., 2015. “Mark Zuckerberg wants people to understand common knowledge. What’s common knowledge?” The Monkey Cage at The Washington Post. https://www.washingtonpost.com/news/monkey-cage/wp/2015/04/08/mark￾zuckerberg-wants-people-to-understand-common-knowledge-whats-common-knowledge/. 24 Adida, C., Gottlieb, J., Kramon, E., & McClendon, G., 2020. “When Does Information Influence Voters? The Joint Importance of Salience and Coordination.” Comparative Political Studies, 53(6), 851-891. 25 Arias, E., Larreguy, H., Marshall, J., and Querubin, P., 2018. “Priors rule: When do malfeasance revelations help or hurt incumbent parties?” Working Paper No. w24888. National Bureau of Economic Research. 26 Björkman, M. and Svensson, J. 2009. “Power to the people: evidence from a randomized field experiment on community￾based monitoring in Uganda.” The Quarterly Journal of Economics, 124(2), 735–769. 27 Björkman Nyqvist, M., Walque, D., and Svensson, J. 2014. “Information is power: experimental evidence of the long run impact of community based monitoring.” World Bank Group Policy Research Working Paper 7015. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 10 • Hypothesis 2a (H2a): Increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness will improve health service delivery. • Hypothesis 2b (H2b): Increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness will improve citizens’ perceptions of health care provision and governance. • Hypothesis 2c (H2c): Increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness will improve citizens’ health seeking behaviors. • Hypothesis 2d (H2d): Increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness will improve household and community health outcomes. To be sure, disseminating information to whole communities can leave citizens with little direction for the way forward. However, we expect that deliberating as a community to craft an action plan based on information shared during the community meeting could serve to mobilize citizens. In principle, one could examine the individual and joint impact of the two mechanisms—increased awareness and mobilization—by implementing separate interventions; however, a recent replication of Bjorkman and Svensson that separated these mechanisms found no impacts on health care utilization rates or health outcomes. 28 Given limited resources, and prioritizing the goal of improving services and measuring impacts, our research design bundles information and mobilization. Moreover, it implements the bundled information and mobilization intervention only in combination with capacity building. The design thus allows us to measure the impact of the capacity building intervention alone and the added marginal impact of increased awareness and community mobilization. 3. METHODOLOGY 3.1 INTERVENTIONS NORC worked with USAID/DRC and DAI to co-design the interventions. For the CB intervention, DAI engaged a consultant to diagnose health care providers’ and CODESA members’ management knowledge and skill gaps and propose a training curriculum to address them. We then worked together to finalize the curriculum. For the CSC intervention, we made an initial proposal based on best practices and current standards in DRC which we then finalized with input from DAI. We describe each intervention in detail below. The CB intervention is a standardized training program that seeks to close gaps in knowledge and skills in the following areas: management of finances and medicine stocks; health care personnel conduct and performance; client service best practices; reporting of public health and service utilization information to HZ; CODESA duties and responsibilities; seeking and responding to citizen feedback; and general public health issues that CODESA can work with providers to address (e.g., use of latrines, hand washing, mosquito control, and epidemics). The goal of the CB intervention is to help health care providers make more efficient use of existing human and material resources, improve health center reporting to the National Health Information System (Système National 28 Raffler, P., Posner, D.N., and Parkerson, D. 2018. “The weakness of bottom-up accountability: Experimental evidence from the Ugandan health sector.” Unpublished manuscript. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 11 d’Information Sanitaire, SNIS), and enhance the functioning of CODESAs and their collaboration with health care providers on health center management and public health issues. The CB training is offered to health care providers (particularly head nurses) and CODESA members in all HAs randomized to the CB only and CB+CSC groups. Volunteers serving as liaisons between the community and health care providers (known locally as relais communautaires) are also invited to the training. IGA delivers the training at centralized locations for participants coming from up to four HAs in a given HZ. Each training lasts five days; the first four days provided training to both health center staff and CODESA members, while the fifth day was reserved for CODESA members and relais communautaires only. IGA offers a second round of training in all locations so that those who are not able to participate in the first round have another chance to do so. The CSC intervention involves generating and disseminating a short document that presents the scores or ratings that the community assigns to various aspects of health care provision (i.e., a community scorecard). In a review of transparency and accountability interventions, Joshi (2013) describes the CSC process as involving “community meetings in which performance of public services is discussed among providers, users, and other stakeholders and includes self-evaluation of performance by providers, as well as the formulation of an action plan based on scorecard outputs.”29 CSCs aim to increase accountability and responsiveness to service users by providing a space for dialogue between users and service providers. In our IE, the CSC intervention consists of three meetings. The first meeting includes CODESA members and the community; together, they generate a scorecard by rating the performance of the health center on a number of areas.30 The second meeting includes health center staff only, who generate a similar scorecard rating their own performance. The third meeting is an interface between all parties; CODESA members, the community, and health center staff come together to discuss their assessments, identify key areas for improvement, and develop a joint action plan. At the joint meeting, facilitators alert participants of a follow-up meeting to happen about six months later to discuss progress made in implementing the action plan. As mentioned earlier, we received reports during the design phase that CODESAs do not operate as intended. To increase the likelihood that our CSC intervention is successful in this context, the CSC intervention includes a ‘CODESA strengthening’ component whereby facilitators attempt to jumpstart accountability relationships between CODESA members and the community. Specifically, facilitators hold a public CODESA-community meeting where the duties and responsibilities of the CODESA are discussed, with a particular emphasis on the CODESA’s role vis-à-vis the community, which includes representing the community’s interests and expectations to the health center and promoting the community’s use of the health center. The CODESA strengthening component is delivered before the three CSC meetings. Figure 1 summarized the CSC intervention. 29 Joshi, A. 2013. “Do they work? Assessing the impact of transparency and accountability initiatives in service delivery.” Development Policy Review, 31(s1), s29–s48. 30 In line with the CSC methodology of the Access to Primary Health Care program (known locally as Accès aux Soins de Santé Primaires, ASSP), facilitators suggested the following performance areas: availability of essential/necessary medicines; availability of equipment; appearance of the building; cleanliness of the building and its surroundings; possibility of being treated without being seen by other patients (intimacy/privacy); technical competence of providers; attendance and punctuality of providers; customer service orientation of providers; discretion and confidentiality of service provider; ability of providers to gain the respect and the trust of the population; and affordability of fees. Participants were given the option to modify this list to reflect what they deemed to be most relevant. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 12 Figure 1. CSC Intervention Summary Our CSC intervention aligns with government guidelines regarding CODESAs and community engagement in health care provision. Currently, the guidelines have not been fully implemented in the areas where our study takes place; therefore, our CSC intervention helps make progress toward guideline uptake. The CB intervention is implemented ahead of the CSC intervention, as it provides the knowledge and skills that health care providers and CODESA members need to implement change. 3.2 SCOPE Our IE takes place in two provinces in Eastern Congo: Haut Katanga and Sud Kivu. The pool of eligible HAs comprised all those where IGA could operate and that had received assistance from the Integrated Health Program (IHP), USAID/DRC’s flagship health sector program in the past. 31 To determine the eligible HAs, we started with a list of all areas in Haut Katanga and Sud Kivu. We excluded HAs located in urban settings, in settings determined to be insecure or inaccessible by USAID/DRC, DAI, or our data collection partner, or in HZs reserved for multi-level governance programming from IGA.32 To avoid contamination, we also omitted HAs in HZs where CordAid (in Sud Kivu) and the World Bank (in Haut Katanga) were implementing similar interventions, either involving CSCs or community engagement activities mixed with performance-based financing programs. In doing so, we sought to avoid any potential interference between those interventions and ours, and preserve our ability to make valid inferences. In addition, following a request from USAID/DRC and DAI to only work with public or faith-based facilities, we excluded HAs whose lead health centers were privately operated and those for which we had no information on who operated the lead health center. Lastly, in an effort to include only HAs that had a health center as a clear lead facility on which CODESA efforts would focus, we further excluded HAs that had a hospital-level facility, HAs that had more than one health center￾31 The original IHP that started in 2010 was extended until the beginning of 2017 via a bridge program called IHP Plus. The program was not active during most of IGA’s first year of operations (2017). IHP Mega is a successor program that was procured in 2018. NORC, DAI, and the USAID/DRC’s DRG Office have coordinated with USAID/DRC’s Health Office to minimize the possibility that IHP Mega activities would inadvertently jeopardize the IE. The State Department’s 2018 Trafficking in Persons Report placed the DRC in Tier 3 and therefore subject to certain restrictions on assistance. This affected the ability of IHP Mega to fully operate until received a waiver for restrictions in 2019. 32 These are a handful of health zones selected to participate in more intensive programming, where IGA would provide some of the same activities used in our study at the HA level, but also work with HZ- and DPS-level officials, civil society organizations, and other stakeholders. CODESA Strengthening Discuss duties and responsibilities of the CODESA Meeting Participants Focus 1 - CODESA Members - Community Members Generate a scorecard by rating the performance of the health center on a number of areas 2 - Health Center Staff Generate a similar scorecard or bulletin, rating their own performance 3 - CODESA Members - Community Members - Health Center Staff Discuss all assessments, identify key areas for improvement, and develop a joint action plan CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 13 level facility, and HAs where a health post (poste de santé) was designated as the lead health center despite the presence of a health center in the HAs. 33 This left us with 182 HAs across 16 HZs; seven HZs in Haut Katanga and nine HZs in Sud Kivu. 34 We determined the number of HAs to be studied based on discussions with USAID/DRC and DAI and the power calculations described in Appendix A. 3.3 RANDOMIZATION We randomly assigned the 182 HAs to one of three groups: (1) CB intervention (CB only); (2) CB intervention plus CSC intervention (CB+CSC); or (3) control. This design allows identification of the impact of CB alone by comparing outcomes across HAs assigned to CB and control. In addition, by comparing outcomes across the CB and the CB+CSC treatment arms, the IE will address whether CB activities produce better results when paired with an information and mobilization intervention. Resource constraints prohibited us from using a full factorial design which would have allowed us to gauge the impact of CSC interventions alone. Figure 2 presents a diagram of the design and includes the breakdown of HAs in each arm, by province. Figure 2. Impact Evaluation Design To achieve balance across the treatment and control groups along important dimensions, we used blocked randomization, also known as stratified random assignment. To implement the randomization we first created seven ‘zonal groups’ based on location (i.e., proximity on a map) and accessibility (i.e., location with respect to roads). Then, within each zonal group, we created blocks of six or more HAs and subsequently assigned the HAs within a block to the experimental groups (CB, CB+CSC, and control). Each of these blocks comprises the HAs within a given zonal group that are more similar to each other, where similarity was determined by the Mahalanobis measure of multivariate distance. Random assignment produced balanced groups. Appendix B lists the variables we used to implement the blocking procedure and presents the results of balance checks we conducted. 3.4 DATA This study consists of three waves of data collection—baseline, midline, and endline. During each wave, we collect objective and subjective indicators of interest using three survey instruments: 33 In the hierarchy of health care provision in DRC, health posts fall below health centers because health posts provide fewer services. 34 The eligible HZs in Haut Katanga are: Kafubu, Kambove, Kapolowe, Kilele Balanda, Lukafu, Mitwaba, and Mufunga Sampwe. The eligible HZs in Sud Kivu are: Bunyakiri, Kalonge, Kamituga, Kaniola, Kaziba, Kitutu, Minova, Mwana, and Mwenga. Treatment: CB+CSC 23 in Haut Katanga 38 in Sud Kivu 182 HAs 68 in Haut Katanga 114 in Sud Kivu Treatment: CB Only 22 in Haut Katanga 37 in Sud Kivu Control: No CB or CSC 23 in Haut Katanga 39 in Sud Kivu CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 14 • Household panel survey: A minimum of 20 households per HA are randomly sampled to complete a survey covering household health, health seeking behavior, experiences with the local health center, and other relevant topics. The target of the interview is a woman of childbearing age. Our goal is for the survey to take less than an hour to complete. The same respondent is interviewed across all three waves. • Lead health center audits: A resource audit of the lead health center for each HA covering inventory (e.g., drugs, test kits, etc.), staffing (e.g., absence of essential personnel), condition of the facility, and other relevant topics. The informant for the audit is the head nurse or her designee. • CODESA member survey: At least one (and where possible two) CODESA member(s) for each HA complete a survey covering CODESA activities, the quality of health services, and other relevant topics. Baseline data collection took place between April and August 2018 and midline data collection between February and August 2019. Endline data collection is tentatively scheduled for the second half of 2020 due to the COVID-19 pandemic. All data collection is carried out by Forcier Consulting under NORC supervision. All instruments were prepared in English and subsequently translated by Forcier for administration in French and two Swahili dialects—Haut Katanga and Sud Kivu. All interviews are conducted using tablet-enabled computer-assisted personal interviewing (CAPI) software.35 Before each round of data collection, the instruments went through several rounds of pre-testing to ensure question appropriateness, check skip patterns and programmed constraints, and identify any technical problems. 3.4.1 BASELINE DATA COLLECTION We conducted the primary training exercise for the field personnel in Bukavu (Sud Kivu) and Lubumbashi (Haut Katanga team) from March 19-25, 2018. The first four days were in-class trainings, which involved reviewing each questionnaire, interviewing techniques, field protocols, and mock interviews, while the final two days involved an in-field pilot and reconciliation. The start of fieldwork had to be delayed because of issues related to tablet procurement and programming and instrument translation. Data collection began on April 29 in both regions after a two day training refresher where field personnel got reacquainted with the instruments. The primary wave of fieldwork ended June 2, but we redeployed a small field team from August 3-26 to collect additional surveys and achieve design quotas in all HAs. Field teams reached 158 HAs and collected a total of 4,227 household surveys, 153 health center audits, and 272 CODESA surveys. We excluded 23 of the 182 HAs included in the IE because we deemed them to be inaccessible due to poor road conditions. Field teams therefore set out to collect data in 159 HAs and reached all but one HA (Ngando, Sud Kivu) due to a security incident.36 35 For baseline data collection, the survey team used the ONA-collect platform built on open data kit (ODK) software programmed onto seven-inch Lenovo Android tablets. For midline data collection, the team used the SurveyCTO platform also built on ODK and programmed onto the same tablets used at baseline. We will continue using SurveyCTO and the same tablets for endline data collection. 36 The team assigned to collect data in Ngando was assaulted by three men dressed in military uniforms when they were en route to this HA, which was the last one they were supposed to cover. This second major security incident marked the end of the primary wave of data collection in Sud Kivu. In addition, another team encountered an armed group in one HA (Culwe, Sud Kivu) and had to leave the area before completing all planned data collection activities. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 15 3.4.2 MIDLINE DATA COLLECTION We conducted the primary training exercise for the field personnel in Lubumbashi (Haut Katanga team) from February 4-8, 2019 and in Bukavu (Sud Kivu) from February 12-16, 2019. Similar to baseline training, the first four days were in-class trainings, which involved reviewing each questionnaire, interviewing techniques, field protocols, and mock interviews, and the final day involved a pilot in the field and a reconciliation. Data collection began on February 25 in Haut Katanga and February 26 in Sud-Kivu. Data collection proceeded in a staggered way and was spread out over a period of five months, finalizing in July 2019 in both provinces.37 Field teams set out to collect data in all 182 HAs, but did not reach four HAs in Sud Kivu (Bisisi, Culwe, Lumbishi, and Shanje) due to insecurity.38 Field teams reached 178 HAs and collected a total of 4,450 household surveys, 178 health center audits, and 356 CODESA surveys. In each HA that was visited at baseline, household survey enumerators attempted to interview the baseline respondent or another respondent in the same household. In each of the 23 HAs that were excluded from the baseline and one additional HA that was not reached due to a security incident, enumerators conducted interviews in a random sample of households. 3.5 OUTCOMES Our outcomes of interest are grouped into four categories: health service delivery; citizens’ perceptions of health care provision and health governance; health seeking and promoting behaviors; and health outcomes. Health outcomes is our main category of downstream impacts; the other categories comprise intermediate outcomes. 3.5.1 HEALTH SERVICE DELIVERY • Service offering and utilization o Proportion of minimum service package (PMA) services provided at health center39 o Index of service utilization reflecting the number of PMA services provided in the past two months40 • Health center financing and resource management and availability 37 Given that implementing the interventions in all HAs assigned to the CB only and CB+CSC groups took nine months (July 2018-March 2019), data collection occurred between three to eight months after intervention implementation in any given HA. 38 Teams could not visit Bisisi and Culwe health areas because they were occupied by the Raïa Mutomboki, a local Congolese militia. Teams were unable to visit Lumbishi and Shanje health areas because of the presence of rebels from the Democratic Forces for the Liberation of Rwanda being tracked by Rwandan troops. 39 In addition to outpatient care consultations, services included in the PMA are: “preschool” consultations (i.e., check-ups for under-five children), vaccinations, family planning consultations, antenatal care consultations, services for the prevention of mother-to-child transmission of human immunodeficiency virus (HIV), vaginal birth deliveries, malaria diagnosis and treatment, sexually-transmitted infections (STIs) diagnosis and treatment, tuberculosis diagnosis and treatment, HIV counseling and testing services, HIV antiretroviral treatment, diagnosis and management of non-communicable diseases, minor surgical services, bacteriologic testing, parasitic testing, and hematologic testing. We do not include outpatient care consultations in the outcome measure, because all health centers provide this service. 40 The services included in this index are outpatient care consultations, “preschool” consultations, vaccinations, family planning consultations, antenatal care consultations, vaginal birth deliveries, malaria diagnosis and treatment, STI diagnosis and treatment, tuberculosis diagnosis and treatment, HIV counseling and testing services, HIV antiretroviral treatment, bacteriologic testing, parasitic testing, and hematologic testing. We create an index following Kling, J.R., Liebman, J.B., and Katz, L.F. 2007. “Experimental analysis of neighborhood effects.” Econometrica, 75(1), 83-119. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 16 o Health center charges a flat fee that covers all services, supplies, and drugs o Amount of flat fee in Congolese francs (CDF) o Extent to which community is involved in setting fee o Proportion of financial management tools41 that are up to date o Health center has inventory or stock control system for drugs that is up to date o Proportion of essential drugs42 that are available at health center • Health center staffing, equipment, and facilities o Health center has a staff schedule that is up to date o Proportion of staff scheduled to work on the day before the audit who were in attendance according to duty schedule o Proportion of staff scheduled to work on day of audit whose attendance was verified o Index of general condition of health center43 • Other health center-related outcomes o Client (patient) communication materials44 displayed in health center o Suggestion box for client feedback was observed in health center • Health service governance o CODESA members are elected by the community o Frequency of meetings between head nurse and CODESA • Household survey respondents’ experiences visiting local health center45: o Wait time (in hours) o Received care from healthcare professional o Medical test was performed during visit o Diagnosis was clearly explained o Medicine was available (if prescribed) o Total amount paid during visit 41 The tools included reflect the guidelines of the MSP. The tools are cash book, receipts booklet, invoice ledger, withdrawal orders booklet, cash flow statement/forecast, expenditures record, debts record, purchase order book, inventory book, list of prices, and operating account statement. 42 This includes Ampicillin (adults), Oxytocin (adults), Albendazole (adults), Stavudine (adults), Norfloxacin (adults), Hepatitis B vaccine (adults & infants), Meningococcal vaccine (A+B) (adults & infants), Isoniazid + Rifampicin (adults & infants), BCG vaccine (adults & infants), Tenofovir (adults & infants), Amoxocillin + clavulanic acid quinine (infants), Quinine (infants), DTC-HepB-Hib (pentavalent) (infants), Zidovudine + Lamivudine (infants), and Sulfadiazine (infants). 43 The index is calculated as the average of the responses to items asking health center auditors to rate the privacy, cleanliness, and condition of the furniture of three rooms (waiting room, consultation room, and maternity room), the smell of the facility, and the cleanliness of the grounds. All items use Likert scales to record responses. 44 The materials are: poster displaying hours of operations, list of prices charged for medical services, poster listing CODESA roles and responsibilities, and poster informing clients of their rights and obligations. 45 Outcomes are available for respondents who reported having visited their local health center to seek care. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 17 o Had to make voluntary payment (i.e., bribes) during visit46 • Household survey respondent reports having received information about contraception • Household survey respondent reports having received information about the importance of seeking care and avoiding self-treatment 3.5.2 PERCEPTIONS ABOUT HEALTH CARE PROVISION AND GOVERNANCE • Household survey respondents’ assessments about health care services47: o Ease in obtaining care during visit48 o Satisfaction with quality of services during visit49 • Household survey respondents’ opinions about health center and health governance: o Index of general service quality50 o Index of general community empowerment vis-à-vis health center staff51 o Belief in community’s ability to improve health care at local health center52 • CODESA and household survey respondents’ opinions on CODESA performance53 3.5.3 HEALTH SEEKING AND PROMOTING BEHAVIORS • Under-five immunization o Child has an immunization record o Proportion of recommended immunizations that child has received54 46 This is a dichotomous variable indicating whether the respondent had to make any voluntary payments in cash or in kind in addition to regular fees. 47 Outcomes are available for respondents who reported having visited their local health center to seek care. 48 This is a dichotomous variable indicating whether the respondent believes it was very easy or easy—as opposed to neither easy nor difficult, difficult, or very difficult—to obtain health care during most recent visit to health center. 49 This is a dichotomous variable indicating whether the respondent was very satisfied or somewhat satisfied—as opposed to neither satisfied nor dissatisfied, somewhat dissatisfied, or very dissatisfied—with the quality of services received during most recent visit to health center. 50 The index is calculated as the average of the responses to the following four items: (1) How satisfied are you with the technical competence of the staff at your local health center? (2) How much would you say you trust the staff at your local health center? (3) How much would you say you respect the staff at your local health center? And, (4) what do you think about the quality of the services currently offered at your local health center? All items use Likert scales to record responses. 51 The index is calculated as the average of the responses to the following two items: (1) If community members found out about a health worker not reporting for work at your local health center, what do you think are the chances that they would be able to pressure that health worker to report for work? And, (2) if community members found out about a health worker at your local health center not providing the effort that he/she should in caring for his/her patients, what do you think are the changes that they would be able to pressure that health worker to apply more effort? Both items use Likert scales to record responses. 52 This is a dichotomous variable indicating whether the respondent believes people in their community have a lot or some power—as opposed to little or no power—to improve the quality of health care. 53 The index is calculated as the average of the responses to four items asking respondents to state their level of agreement with a statement related to CODESA performance. The statements are “The CODESA does a good job representing the community’s health needs;” “The CODESA does a good job promoting health programs through sensitization of the community;” “The CODESA does a good job monitoring the performance of the health center;” and “The CODESA does a good job mobilizing the community to improve health infrastructure.” 54 Immunizations included are five that are recommended by the World Health Organization (WHO). They are the polio vaccine, which protects against the poliovirus; the Bacille Calmette-Guérin vaccine, which prevents tuberculosis; the CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 18 • Health seeking behavior in response to diarrhea in under-five child55 o Did they seek health care? o How many days after the illness onset? o Did they seek care at the local health center? • Health seeking behavior in response to illness in five-and-over household member (available for households with at least one five-and-over member ill within the two weeks prior to the household survey)56 o Did they seek health care? o How many days after the illness onset? o Did they seek care at the local health center? • Antenatal and postnatal care57 o Pregnant woman sought antenatal care o Pregnant woman received antenatal care at the local health center o Woman who recently gave birth sought antenatal care o Woman who recently gave birth received antenatal care at the local health center o Woman who recently gave birth delivered at the local health center o Woman who recently gave birth received postnatal care o Woman who recently gave birth received postnatal care at the local health center 3.5.4 HEALTH OUTCOMES58 • Child health: o Neonatal mortality rate59 o Child mortality rate60 pentavalent vaccine, which prevents diphtheria, tetanus, pertussis (whooping cough), hepatitis B, and Haemophilus influenza type B; the pneumococcal vaccine, which protects against the bacteria Streptococcus Pneumoniae; and the MMR vaccine, which prevents measles, mumps, and rubella. Available for all children who are dependent on the household survey respondent. 55 This is available for all children who were reported to have had diarrhea in the two weeks prior to the household survey and who are dependent on the household survey respondent. 56 This is available for households where the survey respondent indicated that at least one five-and-over member had been ill in the two weeks prior to the survey. If the respondent indicated that two or more household members had been ill, follow-up questions about health seeking behavior were asked about the member whose birthday was coming up next. We defined being ill as feeling too sick to do normal activities. 57 In each surveyed household we included questions to identify women who were pregnant at the time of survey and women who had given birth in the 12 months prior to the survey. We then attempted to interview one woman of each type (i.e., pregnant women and women who recently gave birth). For each type, if more than one woman was available for interviewing, we interviewed the one whose birthday was coming up next. 58 All health outcomes are reported at the HA level. 59 The neonatal mortality rate is calculated as the number of neonate (i.e., child younger than 29 days) deaths per 1,000 live births. The reference period for the indicator is the 12 months immediately preceding the household survey. 60 The child mortality rate is calculated as the number of under-five child deaths per 1,000 live births. The reference period for the indicator is the 12 months immediately preceding the household survey. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 19 o Diarrhea incidence61 • Maternal health o Stillbirth rate62 o Maternal mortality rate63 • Community health o Illness incidence in five-and-over population64 3.6 ESTIMATION We estimate the intent-to-treat effects of the interventions via a least squares regression of the following equation: Yijk,midline = α + β1CBj + β2 CSCj + γYijk,baseline + δk + ε ijk, where Yijk,midline is a given outcome measured at midline, where i refers to the household, j to the HA, and k to the randomization block, α is a constant; CBj and CSCj respectively indicate whether the HA was assigned to the CB only or to the CB+CSC treatment arms; Yijk,baseline is the outcome measured at baseline, and δk are block fixed effects. The coefficients of the treatment indicators (β1 and β2) are the impact estimates. We use standard errors clustered by HA given that treatment is assigned at this level. For dichotomous outcome variables, we estimate equivalent equations via logistic regression and report marginal effects. We expect to find that outcomes are better in HAs assigned to the CB treatment than HAs assigned to the control (β1 > 0). Given our experimental design, we cannot assess whether the CSC treatment alone improves outcomes. We can, however, assess whether outcomes are better in HAs assigned to the CB+CSC treatment than HAs assigned to the control (β2 > 0) and HAs assigned to the CB treatment (β2 > β1). This last comparison allows us to examine whether CB training produces better results when paired with an information and mobilization intervention encouraging accountability from the bottom up. We estimate the equation above using the full sample of 177 HAs for which we have midline data65. As mentioned in Subsection 3.4.1, it was not possible to collect baseline data in 24 HAs and we therefore have missing baseline values for these areas. Our approach to deal with these missing 61 Diarrhea incidence is calculated as the number of children younger than five years who had diarrhea in the two weeks prior to the household survey divided by the total number of children. 62 The stillbirth rate is calculated by dividing the number of stillbirths by the total number of births. The number of stillbirths corresponds to the 24-month period preceding the household survey, while the number of live births corresponds to the 12-month period preceding the household survey 63 The maternal mortality rate is calculated as the number of maternal deaths per 100,000 live births. The number of deaths corresponds to the 24-month period preceding the household survey, while the number of live births corresponds to the 12-month period preceding the household survey. 64 Illness incidence is calculated as the number of five-and-over persons who were ill in the two weeks prior to the household survey divided by the total number of five-and-over persons. 65 We collected data from 178 HAs, but only use the information of 177 HAs because the wrong health center and catchment area was visited in one case. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 20 values is to recode the missing values to an arbitrary constant and include a missingness dummy as an additional covariate.66,67 To estimate causal effects on outcomes at the HA level (i.e., those for which we have one observation per HA and come from the health center audit), we rely on equations similar to the one presented above, but dropping the i subscript. 4. FINDINGS The subsections below present the main findings for each of our four outcome families: health service delivery; citizens’ perceptions of health services and health governance; health seeking behaviors; and household and community health outcomes. Before presenting these findings, we examine treatment uptake and implementation in the following subsection. Since we conducted midline data collection between two to twelve months after intervention implementation in any given HA, the analysis below estimates short-term impacts. We will examine medium-term impacts with endline data to be collected in the second half of 2020. We present the detailed estimation results in Appendix C. 4.1 TREATMENT UPTAKE AND IMPLEMENTATION The instruments for our household survey, health facility audit, and CODESA member survey included questions aimed at measuring treatment uptake and implementation. These measures, along with DAI’s implementation monitoring records, allow us to check whether participating HAs received treatments according to the randomization schedule and treatments were implemented as designed. We also included some simple measures to assess whether the capacity building training improved the knowledge of intended participants. According to DAI implementation monitoring data, the CB activities were implemented in the 120 HAs (45 in Haut Katanga and 75 in Sud Kivu) assigned to the CB only and CB+CSC groups. The training sessions took place between July 2018 and February 2019. DAI’s records also indicate that no medical staff or CODESA members from the 62 control HAs (23 in Haut Katanga and 39 in Sud Kivu) received training. Analyses of health facility audit and the CODESA member survey data indicate successful CB training uptake. In the CB only and CB+CSC groups, about eight in ten audit informants reported that staff from their health center had received training in the last six months, while in the control group, less 66 Gerber, A.S. and Green, D.P. 2012. Field Experiments: Design, Analysis, and Interpretation. WW Norton. 67 For robustness, we estimate the same equations using only the sample of HAs for which we have panel data (i.e., the HAs that we were able to reach both at baseline and midline). In addition, we estimate the following equation using only the sample of HAs for which we have panel data: 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖,𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚 = 𝛼𝛼 + 𝛽𝛽1𝐶𝐶𝐶𝐶𝑖𝑖 + 𝛽𝛽2𝐶𝐶𝐶𝐶𝐶𝐶𝑖𝑖 + 𝛾𝛾𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖,𝑏𝑏𝑏𝑏𝑏𝑏𝑚𝑚𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚 + 𝜃𝜃′𝑋𝑋𝑖𝑖𝑖𝑖𝑖𝑖 + 𝛿𝛿𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 where Yijk,midline is a given outcome measured at midline, where i refers to the household, j to the HA, and k to the randomization block, α is a constant; CBj and CSCj respectively indicate whether the HA was assigned to the CB only or to the CB+CSC treatment arms; Yijk,baseline is the outcome measured at baseline, Xijk is a vector of pre-treatment (i.e., baseline) covariates, and θk are block fixed effects. The coefficients of the treatment indicators (β 1 and β 2) are the impact estimates. We use standard errors clustered by HA given that treatment is assigned at this level. As before, we estimate logistic regressions for dichotomous outcomes and estimate outcomes at the HA level using equations similar to the one presented above, but dropping the i subscript. We do not report the results of these estimations—panel sample with baseline outcome only and panel with baseline outcome and additional covariates—as they are largely consistent with the results described below. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 21 than one in ten informants reported the same. The results are similar for CODESA survey respondents. Other results offer some insights into CB training implementation: • About five in ten audit informants that attended the training correctly mentioned the five main topics that were covered. • About seven in ten CODESA member survey respondents that attended the training correctly mentioned the five main topics that were covered. • About seven in ten audit informants that attended the training correctly indicated that it had lasted five days. • About six in ten CODESA member survey respondents that attended the training correctly indicated that training had lasted five days. Turning to the CSC intervention, and according to DAI’s implementation monitoring data, the intervention was implemented as planned in the 61 HAs (23 in Haut Katanga and 38 in Sud Kivu) assigned to the CB+CSC group. The CSC meetings took place between September 2018 and April 2019, following the completion of the first round of CB trainings in all cases. According to DAI’s records, no CSC intervention events tool place in the 121 HAs (45 in Haut Katanga and 76 in Sud Kivu) in the CB only and control groups. Analyses of data from our three instruments suggest that CSC intervention uptake was not successful. We asked respondents of the health facility audit, the CODESA members survey, and the household survey whether the performance of the local health center had been assessed by the community through a participatory procedure in the past six months. • Among audit informants, those from the CB+CSC group were no more likely to report that performance had been assessed than informants from the control group. • Among CODESA survey respondents, while just under five in ten from the CB+CSC group reported that performance had been assessed, about three in ten respondents from the control group reported the same. • Among household survey respondents, those from the CB+CSC and CB only groups were more likely to report that performance had been assessed than respondents from the control group. While just over 11 in 100 respondents reported that the performance had been assessed in the CB+CSC group and just over 12 in 100 respondent reported the same in the CB only group, only about six in 100 household survey respondents reported the same in the control group. Given that DAI’s implementation tracking sheets indicate that HAs received the CSC intervention according to the randomization schedule, these results suggest two interpretations. On the one hand, they indicate that the CSC intervention might not have been implemented as strongly as designed. Specifically, the CSC events might have failed to be adequately publicized and include all head nurses and acting head nurses, all CODESA members, and a large number of citizens.68 This seems to have been an issue particularly among nurses, where we found no differences between the CB+CSC and the control group. On the other hand, the fact that reports of assessment having taken 68 As described in Section 3.1, the CSC intervention consists of three meetings. The first meeting includes CODESA members and the community; the second meeting includes health center staff only; and the third meeting is an interface between all parties—CODESA members, the community, and health center staff. As many people as possible were supposed to participate in the meetings. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 22 place increased among household respondents in the CB only groups might be indicative of some form of spillovers, whereby some CB training participants from CB-only HAs somehow were encouraged to organize participatory performance assessments.69 Finally, we examine some outcomes that should have had immediate impact attributable to the interventions, if successfully implemented. The first measure indicates whether the audit informant was able to respond correctly to a question about health center data reporting, which was a topic covered in the CB training (head nurses are supposed to report public health and other data to the HZ once a month). The next three indicators are the number of recognized CODESA roles that audit informants, CODESA survey respondents, and household survey respondents are able to mention. These roles were covered during the CB training as well as during the CODESA strengthening activity offered as part of the CSC intervention. We only find impacts consistent with expectations for the recognized CODESA roles among CODESA members. Altogether, the results presented thus far provide a mixed picture of intervention uptake. First, the CB training seems to have been implemented roughly in line with the design; however, the CB training did not produce expected changes in simple knowledge questions among health center staff. Second, CSC intervention implementation seems to have deviated from the design; specifically, it appears that CSC events were not adequately publicized and failed to include all head nurses and acting head nurses, along with other health center staff, all CODESA members, and a large number of citizens. The fact that the CB+CSC treatment did not produce improvements in simple knowledge questions among CODESA and household survey respondents is also puzzling. These knowledge questions, however, only constitute one family of outcomes we study to measure the intervention’s impact; while important, they are not essential to see an impact on other families of outcomes, including health service delivery, which we turn to next. 4.2 HEALTH SERVICE DELIVERY The midline findings offer little support for the hypothesis that increasing health officials’ capacity in managing resources improves health service delivery in the short term. Likewise, we find little evidence that increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness in combination with CB training yields better health service delivery. Specifically, we examine: • Two outcomes related to service offering and utilization70 and find no significant effects. • Seven outcomes related to health center financing and resource management71 and find two significant effects: (1) the CB only treatment resulted in an 18 percent reduction in the flat fee charged to cover services, supplies, and drugs, compared to the control group; and (2) the 69 We included some questions about attendance to the various CSC intervention meetings in both survey instruments and the audit instrument, but do not report results here given the very small number of respondents who reported having attended the meetings. 70 This includes the following indicators: proportion of PMA services provided at health center and index of service utilization reflecting the number of PMA services provided in the past two months. 71 This includes the following indicators: health center charges a flat fee that covers all services, supplies, and drugs; amount of flat fee in Congolese francs (CDF); extent to which community is involved in setting fee; extent to which community is involved in setting fee; proportion of financial management tools used at health center that are up to date (verified); health center has inventory or stock control system for drugs that is up to date (verified); and proportion of essential drugs that are available at health center (verified). CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 23 CB+CSC treatment increased the likelihood of having an updated inventory or stock control system for drugs compared to the control group by 43 percent. • Four outcomes related to health center staffing, equipment, and facilities72 and find no significant effects. • Two other health center-related outcomes73 and find one significant effect: the likelihood of health centers in the CB+CSC group having client communication materials was 32 percent higher than in the control group. • Four outcomes related to health service governance74 and find two significant effects: (1) according to the health center staff, the CB only and CB+CSC interventions increased the likelihood of the community electing CODESA members by 31 percent, compared to the control group; and (2) based on responses from CODESA members, the CB only treatment increased the likelihood of the community electing CODESA members by 22 percent, compared to the control group. • Nine outcomes related to households’ experiences visiting the local health center75 and find two significant effects: (1) the CB+CSC intervention reduced the total amount paid by households by 16 percent, compared to the control group; and (2) the CB only treatment increased the likelihood of receiving information about contraception by 27 percent, compared to the control group. 4.3 PERCEPTIONS ABOUT HEALTH CARE PROVISION AND GOVERNANCE The midline results do not support the hypothesis that increasing health officials’ capacity in managing resources improves citizens’ perceptions of health care provision and governance. Likewise, increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness in combination with CB training did not improve perceptions of health care provision and governance by the time midline data was collected. Specifically, we examine: • Two outcomes related to households’ assessments about health care services76 and find no significant effects. • Five outcomes related to opinions about the health center, health governance, and the performance of the CODESA77, and find one significant effect: CODESA respondents from the 72 This includes the following indicators: health center has a staff schedule that is up to date (verified); proportion of staff scheduled to work on the day before the audit who were in attendance (according to staff schedule); proportion of staff scheduled to work on day of audit whose attendance was verified; and cleanliness and condition of health center (enumerator observation). 73 This includes the following indicators: client (patient) communication materials are present in health center (verified); and suggestion box for client feedback is available at health center (verified). 74 This includes the following indicators: CODESA members are elected by the community; and frequency of meetings between head nurse and CODESA—each from the Audit and CODESA surveys. 75 The following indicators were assessed: wait time; received care from healthcare professional; medical test was performed during visit; diagnosis was clearly explained; medicine was available (if prescribed); total amount paid during visit; had to make voluntary payment (i.e., bribes) during visit; received information about family planning methods / contraception; and received information about the importance of seeking care and avoiding self-treatment. 76 This includes the following indicators: households’ assessments of ease in obtaining care; and satisfaction with the quality of services during visit. 77 This includes the following indicators: index of general health center quality; index of general community empowerment vis-à-vis health center staff; belief in community’s ability to improve health care at local health center; and respondents’ opinions on CODESA performance. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 24 CB+CSC group have a more favorable opinion on the CODESA performance (measure is ten percent higher) than CODESA respondents from the control group. 4.4 HEALTH SEEKING AND PROMOTING BEHAVIORS The midline findings do not support the hypothesis that increasing health officials’ capacity in managing resources improves citizens’ health seeking behaviors. Similarly, increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness in combination with CB training did not promote health seeking or health-promoting behaviors among citizens in the short term. Specifically, we examine: • Two outcomes related to under-five immunization 78 and find no significant effects. • Six outcomes related to health-seeking behavior79 and find no significant effects. • Seven outcomes related to antenatal and postnatal care80 and find one significant effect: the CB only and CB+CSC treatments increased the likelihood of women who recently gave birth to seek antenatal care by four and five percent, respectively, compared to the control group. 4.5 HEALTH OUTCOMES The midline results do not support the hypothesis that increasing health officials’ capacity in managing resources improves child, maternal, or community health indicators. Likewise, increasing citizens’ awareness of health service delivery and mobilizing communities to act on that increased awareness in combination with CB training did not improve health outcomes by the time midline data was collected. Specifically, we examine: • Three outcomes related to child health 81 and find no significant effects consistent with expectations.82 • Two outcomes related to maternal health 83 and find no significant effects. • One outcome related to community health 84 and find no significant effects. 78 This includes the following indicators: likelihood of a child having an immunization record; and proportion of immunizations that a child has received. 79 This includes the following indicators: sought care; number of days between illness onset and care was sought; and sought care at local health center—each when a household member under five years old has diarrhea and when a household member five years old or older is sick. 80 This includes the following indicators: pregnant women sought antenatal care; pregnant women received antenatal at the local health center; woman who recently gave birth sought antenatal care; woman who recently gave birth received antenatal care at the local health center; woman who recently gave birth delivered at the local health center; woman who recently gave birth received postnatal care; and woman who recently gave birth received postnatal care at the local health center. 81 This includes the following indicators: infant mortality rate; child mortality rate; and diarrhea incidence. 82 Surprisingly, we find that the incidence of diarrhea is 31 percent higher in the CB only group than in the control group, and 21 percent higher in the CB+CSC group than in the control group. 83 This includes the following indicators: stillbirth rate; and maternal mortality rate. 84 This includes the following indicator: incidence of illnesses in the over-five population. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 25 4.6 HETEROGENEOUS EFFECTS We analyze if short-term treatment effects vary across different groupings of study participants.85 Stated differently, we assess whether the interventions have a greater impact among households (within HAs) and HAs with certain characteristics. We can imagine, for example, that an information and mobilization intervention may not be sufficient to mobilize people from marginalized groups. All characteristics were measured at baseline. We examine the following heterogeneous effects: • Household characteristics o Social engagement: We expect the CB+CSC treatment to have a larger effect among those who are more engaged in their community. o Political efficacy: We expect the CB+CSC treatment to have a larger effect among those with higher levels of political efficacy. o Minority status: We expect the CB+CSC treatment to have a larger effect among those who identify as members of a majority group, whether ethnic, linguistic, or religious. o Household wealth: We expect both the CB and CB+CSC treatments’ effects to vary according to household wealth, particularly among the relatively rich (highest tertile on asset ownership index) and the relatively poor (lowest tertile on asset ownership index). o Religious service attendance: We expect the CB+CSC treatment to have a larger effect among those who report attending religious services more often. • Health area characteristics o Social engagement: We expect the CB+CSC treatment to have a larger effect in HAs with higher levels of social engagement. o Political efficacy: We expect the CB+CSC treatment to have a larger effect in HAs with higher levels of political efficacy. o Fragmentation: We expect the CB+CSC treatment to have a larger effect in HAs that are less socially heterogeneous along ethnic, linguistic, or religious lines. o Wealth inequality: We expect the CB+CSC treatment to have a larger effect in HAs with greater wealth inequality. o Value of participation: We expect the CB+CSC treatment to have a larger effect in HAs that place a higher value on citizen participation in oversight of health services. o Health service quality: We expect both the CB and CB+CSC treatments to have larger effects in HAs where health delivery was rated below the median. o Health outcomes: We expect both the CB and CB+CSC treatments to have larger effects in HAs with worse health outcomes. o Health facility type: We expect both the CB and CB+CSC treatments to have larger effects in HAs with public—as opposed to faith-based—health facilities. 85 To examine heterogeneous effects, we estimate the following equation: 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖,𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚 = 𝛼𝛼 + 𝛽𝛽1𝐶𝐶𝐶𝐶𝑖𝑖 + 𝛽𝛽2𝐶𝐶𝐶𝐶𝑖𝑖𝑍𝑍𝑖𝑖𝑖𝑖𝑖𝑖 + 𝛽𝛽3𝐶𝐶𝐶𝐶𝐶𝐶𝑖𝑖 + 𝛽𝛽4𝐶𝐶𝐶𝐶𝐶𝐶𝑖𝑖𝑍𝑍𝑖𝑖𝑖𝑖𝑖𝑖 + 𝛾𝛾𝑍𝑍𝑖𝑖𝑖𝑖𝑖𝑖 + θ 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖,𝑏𝑏𝑏𝑏𝑏𝑏𝑚𝑚𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚 + 𝛿𝛿𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖,𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚𝑖𝑖𝑚𝑚𝑚𝑚, where 𝑍𝑍𝑖𝑖𝑖𝑖𝑖𝑖 is an indicator variable that takes the value of one if the baseline value of covariate 𝑧𝑧𝑖𝑖𝑖𝑖𝑖𝑖 is above the median. 𝛽𝛽2 and 𝛽𝛽4 estimate the change in the effects of the CB and CB+CSC treatments associated with a change in 𝑍𝑍𝑖𝑖𝑖𝑖𝑖𝑖. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 26 o Supervision of health facility: We expect both the CB and CB+CSC treatments to have larger effects in HAs where health facilities receive supervision visits from health zone officials more often. o Health facility funding: We expect both the CB and CB+CSC treatments to have larger effect in HAs where patient fees are a more important source of health facility funding.86 We examine effects on 30 outcomes for each of five household characteristics and 57 outcomes for each of ten HA characteristics, estimating more than 720 regressions. We find a number of significant effects scattered throughout this analysis, but this is not surprising given the number of statistical tests we performed.87 As in the main analyses, taken together, these results do not suggest that treatment effects varied in line with our expectations. Lastly, because we were curious about whether our findings may have been driven by survey timing – e.g., that we have little evidence of effects because the impact “decayed” since the intervention – we examine whether short-term treatment effects vary with time elapsed between intervention implementation and midline data collection. Since we conducted midline data collection between two to twelve months after intervention implementation in any given HA, we can assess whether effects increase, decrease, or do not vary with time elapsed since intervention implementation. However, we find that short-term treatment effects do not vary with time. 5. CONCLUSIONS This IE aims to contribute to our broader understanding of how to improve the human condition in contexts of great need. More specifically, we seek to add to the growing body of literature examining whether governance interventions can contribute to improving service provision and development prospects. We seek to answer two research questions: Whether improving service providers’ and CODESA members’ management capacities through a CB training intervention results in better health-related knowledge, attitudes, and behaviors among citizens, and in enhanced health service delivery and health outcomes? And whether disseminating information and promoting community mobilization to improve health service delivery through a CSC intervention enhances the impacts of the CB training intervention. In this report we examine intervention uptake and implementation, and present estimates of the interventions’ short-term impacts. Our analysis offers a mixed picture of intervention uptake. The CB training seems to have been implemented roughly in line with the design, but it did not produce expected changes in simple knowledge questions among health center staff. In turn, implementation of the CSC intervention seems to have deviated from the design in that they do not appear to have been adequately publicized and widely attended. Moreover, the fact that the CB+CSC treatment did not produce improvements in simple knowledge questions among CODESA and household survey respondents is also puzzling. 86 Other sources of funding include government transfers; private funds; transfers from other health centers; and funds from donors or CSOs. Some health center audit informants reported that health center personnel provide funding. 87 The one potentially interesting finding emerging from the analysis of heterogeneous effects is that the CB+CSC treatment had some different effects among those who attend religious services twice or more times per week. Among women who recently gave birth and attend religious services often, those in HAs assigned to the CB+CSC treatment were more likely to (1) receive antenatal care and (2) deliver at the local health center than those in HAs assigned to the control group. We observe a similar effect on respondents’ likelihood of receiving information about the importance of seeking care and avoiding self-treatment: among respondents who attend religious services often, those in HAs assigned to the CB+CSC treatment were more likely to report having received this information than those in HAs assigned to the control group. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 27 Regarding the interventions’ short-terms impacts, our results indicate that neither the CB training alone nor the CB training combined with the CSC intervention substantively improved health service delivery, citizens’ perceptions about health care provision and governance, citizens’ health seeking and promoting behaviors, or health outcomes. Our analysis also indicates that short-term treatment effects did not vary across different groupings of study participants. Contrary to our expectations, we find no evidence that treatments produced short-term improvements in HAs where health care services and outcomes were particularly defective at baseline, or in HAs with lower levels of social fragmentation and wealth inequality, as these are impediments to social coordination and mobilization. As we prepare for endline data collection, we remain hopeful that our analysis of endline data will show that the CB training alone and the CB training combined with the CSC intervention did have effects in the medium term. If that were the case, we would conclude that treatment effects needed additional time to materialize. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 28 REFERENCES Adida, C., Gottlieb, J., Kramon, E., and McClendon, G., 2020. “When does information influence voters? The Joint Importance of Salience and Coordination.” Comparative Political Studies 53(6): 851-891. Arias, E., Larreguy, H., Marshall, J., and Querubin, P., 2018. “Priors rule: When do malfeasance revelations help or hurt incumbent parties?” Working Paper No. w24888. National Bureau of Economic Research. Banerjee, A. V., Banerji, R., Duflo, E., Glennerster, R., and Khemani, S., 2008. Pitfalls of participatory programs: Evidence from a randomized evaluation in education in India. The World Bank. Björkman Nyqvist, M., Walque, D., and Svensson, J. 2014. “Information is power: experimental evidence of the long run impact of community based monitoring.” World Bank Group Policy Research Working Paper 7015. Björkman, M. and Svensson, J. 2009. “Power to the people: evidence from a randomized field experiment on community-based monitoring in Uganda.” The Quarterly Journal of Economics, 124(2), 735–769. Bold, T., Svensson, J., Gauthier, B., Maestad, O., and Wane, W. 2011. “Service Delivery Indicators: Pilot in Education and Health Care in Africa.” CMI Report 2011/08. Bossert, T.J., Mitchell, A.D., and Janjua, M.A. 2015. “Improving health system performance in a decentralized health system: capacity building in Pakistan.” Health Systems & Reform, 1(4), 276– 284. Brinkerhoff, D.W. 2005. “Rebuilding governance in failed states and post‐conflict societies: core concepts and cross‐cutting themes.” Public Administration and Development: The International Journal of Management Research and Practice, 25(1), 3-14. Chaudhury, N., Hammer, J., Kremer, M., Muralidharan, K., and Rogers, F.H. 2006. “Missing in action: teacher and health worker absence in developing countries.” Journal of Economic perspectives, 20(1), 91-116. Chwe, M. S., 2003. Rational ritual: culture, coordination, and common knowledge. Princeton University Press. Chwe, M. S., 2015. “Mark Zuckerberg wants people to understand common knowledge. What’s common knowledge?” The Monkey Cage at The Washington Post. Available at: https://www.washingtonpost.com/news/monkey-cage/wp/2015/04/08/mark-zuckerberg-wants￾people-to-understand-common-knowledge-whats-common-knowledge. Conn, C.P., Jenkins, P., and Touray, S.O. 1996. “Strengthening health management: experience of district teams in The Gambia.” Health Policy and Planning, 11(1), 64–71. Englebert, P. 2014. “Democratic Republic of Congo: Growth for All? Challenges and Opportunities for a New Economic Future.” The Brenthurst Foundation Discussion Paper 6/2014. Gates, S., Hegre, H., Nygård, H.M., and Strand, H., 2012. “Development consequences of armed conflict.” World Development, 40(9), 1713-1722. Gerber, A.S. and Green, D.P. 2012. Field Experiments: Design, Analysis, and Interpretation. WW Norton. Joshi, A. 2013. “Do they work? Assessing the impact of transparency and accountability initiatives in service delivery.” Development Policy Review, 31(s1), s29–s48. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 29 Karlan, D., Knight, R., and Udry, C. 2015. “Consulting and capital experiments with microenterprise tailors in Ghana.” Journal of Economic Behavior & Organization, 118, 281–302. Kebede, S., Mantopoulos, J., Ramanadhan, S., Cherlin, E., Gebeyehu, M., Lawson, R., and Bradley, E.H. 2012. “Educating leaders in hospital management: a pre-post study in Ethiopian hospitals.” Global Public Health, 7(2), 164–174. Kling, J.R., Liebman, J.B., and Katz, L.F. 2007. “Experimental analysis of neighborhood effects.” Econometrica, 75(1), 83-119. Kosack, S. and Fung, A. 2014. “Does transparency improve governance?” Annual Review of Political Science, 17, 65–87. Lieberman, E. S., Posner, D. N., and Tsai, L. L., 2014. “Does information lead to more active citizenship? Evidence from an education intervention in rural Kenya.” World Development, 60, 69-83. Ministère du Plan et Suivi de la Mise en œuvre de la Révolution de la Modernité, Ministère de la Santé Publique, and ICF International, 2014. “Democratic Republic of Congo Demographic and Health Survey 2013-14: Key Findings.” Rockville, Maryland, USA: MPSMRM, MSP, and ICF International. Raffler, P., Posner, D.N., and Parkerson, D. 2018. “The weakness of bottom-up accountability: Experimental evidence from the Ugandan health sector.” Unpublished manuscript. Reinikka, R. and Svensson, J., 2004. “Local capture: evidence from a central government transfer program in Uganda.” The Quarterly Journal of Economics, 119(2), 679-705. Rowe, L.A., Brillant, S.B., Cleveland, E., Dahn, B.T., Ramanadhan, S., Podesta, M., and Bradley, E.H. 2010. “Building capacity in health facility management: guiding principles for skills transfer in Liberia.” Human Resources for Health, 8(1), 5. Tetra Tech ARD. 2012. “Democracy, Human Rights, and Governance Assessment of the Democratic Republic of the Congo: Final Report.” Report commissioned by USAID. Burlington, Vermont. Umble, K.E., Brooks, J., Lowman, A., Malison, M., Huong, N.T., Lademarco, M., and Laserson, K. 2009. “Management training in Vietnam’s National Tuberculosis Program: an impact evaluation.” The International Journal of Tuberculosis and Lung Disease, 13(2), 238–246. World Bank Operations Evaluation Department. 2005. “Capacity Building Capacity Building in Africa: An OED Evaluation of World Bank Support.” OED No. 34351. Washington, DC: The International Bank for Reconstruction and Development / The World Bank. Yeager, V.A. and Bertrand, J. 2016. “Putting management capacity building at the forefront of health systems strengthening: comment on ‘Management matters: A leverage point for health systems strengthening in global health.’” International Journal of Health Policy and Management, 5(2), 129. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 30 APPENDIX A: POWER CALCULATIONS CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 31 The power analysis whose results we present below are based on estimating difference-in￾differences regressions of the form: where Yict is a continuous outcome variable for household i in HA c at time t; Tc are HA-level treatment indicators; tt is a period fixed effect; and αi are household fixed effects. The coefficient of the interactions between the treatment indicators and the period fixed effect estimates the impact of the treatments. Regressions for outcomes at the HA level (i.e., outcomes for which there is a single observation per cluster, such as characteristics of the lead health center) drop the i subscript and include HA instead of household fixed effects. We estimated these regressions using 1000 simulated datasets. The simulations assumed a sample of 180 HAs assigned to three groups—two treatment and one control—and 20 households per HA. In addition, the simulations conservatively assumed household- and HA-level autocorrelations of 0.10 and an intra-cluster correlation coefficient (ICC) of 0.10. Consistent with conventions in RCT evaluations, we set the significance level at 0.05 and power at 0.80. Table A.1 presents the results of the power analysis. For both household- and HA-level outcomes, we computed the minimum detectable effect sizes (MDESs) (i.e., effect size expressed as a multiple of the standard deviations of the original outcome variable) for two tests: (1) whether the effect of a given treatment, CB or CB+CSC is different from zero and (2) whether the effects of the CB and CB+CSC treatments are different from each other. These estimates have not been adjusted for multiple hypothesis testing. With many outcomes, this could increase the minimum detectable effects. Blocked random assignment and covariate adjustment, which are not reflected in these power calculations, will improve precision and reduce the MDESs. Table A.1. Estimated Minimum Detectable Effect Sizes TYPE OF OUTCOME ASSUMPTIONS MDES FOR TEST 1 MDES FOR TEST 2 HA-level outcome (one observation per HA) HA-level autocorrelation=0.10 0.69 0.68 Household-level outcome (20 observations per HA) HA -level autocorrelation=0.10 Household-level autocorrelation=0.10 ICC=0.10 0.26 0.27 Note: All simulations assume a sample of 180 HAs and 20 households per HA, a significance level of 05, and power of 0.8. Test 1 contrasts whether the effect of a given treatment is different from zero; Test 2, whether the effects of two given treatments are different from each other. Standard errors are adjusted for clustering at the health area level in regressions of household-level outcomes. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 32 APPENDIX B: RANDOMIZATION AND BALANCE CHECKS CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 33 The variables used to implement the blocking are listed in Table B.1 and include information from USAID, SNIS, and the household baseline survey. Table B.1. Variables Used for Blocking VARIABLEa SOURCE USAID/DRC priorityb USAID Records Type of facility (public or faith-based) SNIS On-time facility compliance with health zone reporting protocols SNIS Number of other facilities in the HA SNIS Previous experience with a community scorecard process Baseline Household Survey Health center quality Baseline Household Survey Community contact with CODESA members Baseline Household Survey An indicator of child and infant mortality Baseline Household Survey a The data collection team was not able to reach 23 HAs in two zonal groups due to poor road conditions. Blocks in these zonal groups were created using only the first four variables. Also, due to a security incident, the data collection team was not able to reach one HA in an otherwise accessible zonal group. This HA was added to the block that contained its most similar area using only the first four variables. b All HZs included in our IE have received health programming support from USAID; while some received only minimal support, other “priority” health zones received full support. Table B.2 presents results from examining the balance on important background characteristics and outcome indicators across HAs by assignment to treatment and control groups. More specifically, we tested whether there are statistically significant differences between CB and CB+CSC and control areas in the mean values for responses to questions asked in the household survey. Examining balance across random assignment on measures central to our study, we found good balance on all measures, though some imbalance on citizens’ evaluations of the quality of their local health center. We also saw a significant difference in education level of respondents in the CB+CSC arm when compared to the CB treatment arm. The substantive differences between these values are small and we are not concerned about threats to the IE. Table B.2. Balance across Groups VARIABLE (1) (2) (3) T-TEST DIFFERENCE T-TEST DIFFERENCE T-TEST CONTROL CB CB+CSC DIFFERENCE N MEAN [SE] N MEAN [SE] N MEAN [SE] (1)-(2) (1)-(3) (2)-(3) Level of Education 1467 1.055 [0.026] 1353 1.071 [0.026] 1399 0.999 [0.025] -0.016 0.055 0.072** Religiosity 1389 4.060 [0.042] 1257 4.117 [0.044] 1323 4.038 [0.043] -0.057 0.022 0.079 Household Reported U5 dead 1472 0.003 [0.002] 1353 0.007 [0.002] 1403 0.004 [0.002] -0.003 -0.001 0.002 CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 34 VARIABLE (1) (2) (3) T-TEST DIFFERENCE T-TEST DIFFERENCE T-TEST CONTROL CB CB+CSC DIFFERENCE Health Center Evaluation 1122 2.028 [0.022] 986 2.129 [0.023] 1078 1.943 [0.021] -0.101*** 0.084*** 0.185*** Empowerment Overall 1406 3.363 [0.026] 1320 3.367 [0.026] 1361 3.359 [0.026] -0.005 0.004 0.009 Empowerment WRT Health 1350 3.064 [0.027] 1224 3.067 [0.029] 1290 3.013 [0.028] -0.003 0.051 0.054 Note: The values displayed for t-tests are the differences in the means across groups; ***, **, and * indicate significance at the 1, 5, and 10 percent critical level. We conducted an additional balance check to confirm that the randomization produced groups with balanced control covariates. The check consisted of estimating a multinomial logistic regression of treatment assignment with block fixed effects and all covariates in the table above included as independent variables. Standard errors were clustered at the health area level. We then tested the hypothesis that all covariates’ coefficients are jointly zero, but could not reject the null (p=0.1430). We conclude that random assignment produced balanced groups. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 35 APPENDIX C: DETAILED ESTIMATION RESULTS CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 36 C.1 TREATMENT UPTAKE AND IMPLEMENTATION Table C.1.1 presents estimation results for two CB training uptake measures. The first measure indicates whether the audit informant reported that staff from their health center had participated in a training offered by USAID and IGA in coordination with the DPS in the past six months. In the control group, the likelihood of this being the case was just below 0.07. In the CB only and CB+CSC groups, the likelihoods were statistically significant higher and 0.85 and 0.90, respectively. Similarly, the second measure indicates whether the CODESA survey respondent reported that members of their CODESA attended the training. Among CODESA survey respondents in the control group, the likelihood that a respondent reported that members of their CODESA had participated in a training was just below 0.10. In the CB only and CB+CSC groups, the likelihoods were statistically significantly higher and 0.84 and 0.91, respectively. Table C.1.1. CB Intervention Uptake Estimation Results OUTCOME CB Only CB+CSC CONTROL Medical staff received training in the last 6 months (Audit) 0.783*** 0.833*** 0.068 (0.057) (0.066) (0.033) CODESA members received training in the last 6 months (CODESA) 0.745*** 0.817*** 0.094 (0.056) (0.048) (0.027) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household and CODESA member surveys. Parentheses next to outcome indicate source. Regressions only include data from HAs where data collection took place within the six months immediately after intervention implementation. The following figures use information from HAs in the CB only and CB+CSC groups to provide information about how the CB intervention was implemented. According to the design, head nurses and acting head nurses were the main target groups of the trainings among health staff. Figure C.1.1 shows the percentage of audit informants—who are head nurses or acting head nurses—that reported having attended the CB training themselves.88 More than four out of five (81 percent) audit informants attended the training, which indicates that the CB intervention was successful in reaching its intended audience among health staff. 88 At midline, 69 percent of audit informants identified themselves as head nurses and 31 percent as acting head nurses. In HAs that were assigned to receive the CB training—i.e., HAs in both the CB only and CB+CSC groups, 65 percent of audit informants identified themselves as head nurses and 35 percent as acting head nurses. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 37 Figure C.1.1. Percentage of Audit Informants that Attended Training Note: Figures present data from HAs in CB only and CB+CSC groups where data collection took place within the six months immediately after intervention implementation. Figures C.1.2 and C.1.3 provide additional information about CB training implementation. Figure C.1.2 shows that just under one in two (47.1 percent) audit informants that attended the training correctly mentioned the five main topics that were covered (i.e., CODESA roles and attributions; personnel performance, attendance, and attitude toward patients; HZ reporting; financial management principles; and management of medicine stocks). Figure C.1.2. Number of Topics Mentioned by Audit Informants Note: Figures present data from HAs in CB only and CB+CSC groups where data collection took place within the six months immediately after intervention implementation. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 38 Figure C.1.3 shows that just over three in four (76.5 percent) audit informants that attended the training indicated that it had lasted five days. According to the design, while each training session was supposed to last five days, only the first four days were supposed to provide training for both health center staff and CODESA members—the fifth day was reserved for CODESA members only. One possibility is that audit informants generally attended four days of training, but were aware of the fifth day and interpreted our question (“How many days did the training last?”) as referring to the duration of the whole training session. Another possibility is that they indeed attended all five days. Figure C.1.3. Length of Training (in Days) According to Audit Informants Note: Figures present data from HAs in CB only and CB+CSC groups where data collection took place within the six months immediately after intervention implementation. Figures C.1.4, C.1.5, and C.1.6 present information about the CB intervention implementation from the perspective of CODESA survey respondents. First, Figure C.1.4 shows the percentage of respondents that reported having attended the CB training themselves. Just over two in three (69.2 percent) attended the training, which indicates that the CB intervention was successful in reaching its intended audience among CODESA members.89 In turn, Figure C.1.5 shows that 30 percent of CODESA survey respondents that attended the training correctly mentioned the five main topics that were covered. This suggests that CODESA members might have had greater difficulty than health center staff to absorb training materials, which is consistent with a report we received from DAI during implementation.90 Last, Figure C.1.6 shows that three in five (60 percent) CODESA member survey respondents that attended the training correctly indicated that training had lasted five days. 89 At midline, 35 percent of CODESA survey respondents identified themselves as presidents, 16 percent as vice￾presidents, 12 percent as secretaries, and the remaining 37 percent as adjunct secretaries, treasurers, or members of one of four CODESA commissions. In HAs that were assigned to receive the CB training—i.e., HAs in both the CB only and CB+CSC groups, 36 percent of CODESA survey respondents identified themselves as presidents, 17 percent as vice￾presidents, 11 percent as secretaries, and the remaining 36 percent as adjunct secretaries, treasurers, or members of one of four CODESA commissions. 90 Specifically, during a regular check-in call at the beginning of the implementation period, DAI reported that trainers were having to spend more time than originally planned to cover some topics because CODESA members were having difficulties in understanding the materials. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 39 Figure C.1.4. Percentage of CODESA Survey Respondents that Attended Training Note: Figures present data from HAs in CB only and CB+CSC groups where data collection took place within the six months immediately after intervention implementation. Figure C.1.5. Number of Topics Mentioned by CODESA Survey Respondents Note: Figures present data from HAs in CB only and CB+CSC groups where data collection took place within the six months immediately after intervention implementation. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 40 Figure C.1.6. Length of Training (in Days) According to CODESA Survey Respondents Note: Figures present data from HAs in CB only and CB+CSC groups where data collection took place within the six months immediately after intervention implementation. Table C.1.2 presents estimation results for three uptake measures indicating whether respondents reported that the performance of the local health center had been assessed by the community through a participatory procedure in the past six months. Finding that all three measures are higher in the CB+CSC group than in the other groups would be indicative of successful treatment uptake (and implementation). We find that this is not the case, however. The only results that conform to this expectation are those pertaining to CODESA members. Among CODESA survey respondents in the control group, the likelihood that a respondent indicated that the performance of the health center had been assessed was 0.31. The likelihood was statistically significantly higher in the CB+CSC group (estimated likelihood was 0.57). Said in other words, while almost three in five CODESA members reported that performance had been assessed in the CB+CSC group, less than one in three CODESA members reported the same in the control group. The likelihood that an audit informant in the control group indicated that the performance of the health center had been assessed was just under 0.41. The likelihood in the CB+CSC group was not statistically significantly higher. Among household survey respondents in the control group, the likelihood that a respondent indicated that the performance of the health center had been assessed was 0.07. In the CB+CSC group, but also in the CB only group, the likelihood was statistically significantly higher. The likelihood was 0.11 in the CB+CSC group and 0.12 in the CB only group. While just over 11 and 12 in 100 household survey respondents reported that the performance had been assessed in the CB+CSC and CB only group, respectively, only about six in 100 household survey respondents reported the same in the control group. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 41 Table C.1.2. CSC Intervention Uptake Estimation Results OUTCOME CB ONLY CB+CSC CONTROL CSC event occurred in the last 6 months (Audit) 0.098 0.152 0.407 (0.088) (0.086) (0.064) CSC event occurred in the last 6 months (CODESA) 0.137 0.257*** 0.313 (0.070) (0.069) (0.043) CSC event occurred in the last 6 months (Household) 0.045** 0.042* 0.071 (0.016) (0.017) (0.009) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household and CODESA member surveys. Parentheses next to outcome indicate source. Regressions only include data from HAs where data collection took place within the six months immediately after intervention implementation. Table C.1.3 presents estimation results for some outcomes that should have been immediately impacted by the interventions if successfully implemented. The first measure indicates whether the audit informant was able to respond correctly to a question about health center data reporting, which was a topic covered in the CB training (head nurses are supposed to report public health and other data to the health zone once a month). Quite surprisingly, we find no changes in this measure. The next three indicators are the number of recognized CODESA roles that audit informants, CODESA survey respondents, and household survey respondents are able to mention. Consistent with expectations, we find that the CB only treatment had a positive impact among CODESA members. Surprisingly, we find no statistically significant impacts among audit informants in either treatment group or among CODESA members in the CB+CSC group. More surprisingly perhaps, we find that the CB only treatment had a negative impact among household survey respondents. Table C.1.3. Manipulation Checks OUTCOME CB Only CB+CSC CONTROL Health center staff responds correctly to questions about health center data reporting (Audit) 0.034 -0.077 0.831 (0.092) (0.096) (0.049) Number of CODESA roles mentioned by health center staff (Audit) 0.449 0.303 2.534 (0.229) (0.226) (0.171) Number of CODESA roles mentioned by CODESA member (CODESA) 0.519** 0.195 2.305 (0.185) (0.170) (0.113) Number of CODESA roles mentioned by household respondent (Household) -0.240** 0.110 2.594 (0.086) (0.080) (0.050) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household and CODESA member surveys. Parentheses next to outcome indicate source. Regressions only include data from HAs where data collection took place within the six months immediately after intervention implementation. Altogether, the results presented thus far provide a mixed picture of intervention uptake. First, the CB training seems to have been implemented roughly in line with the design; however, the CB CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 42 training did not produce expected changes in simple knowledge questions among health center staff. Second, CSC intervention implementation seems to have deviated from the design; specifically, it appears that CSC events were not adequately publicized and failed to include all head nurses and acting head nurses, along with other health center staff, all CODESA members, and a large number of citizens. The fact that the CB+CSC treatment did not produce improvements in simple knowledge questions among CODESA and household survey respondents is also puzzling. C.2 HEALTH SERVICE DELIVERY Table C.2.1 reports the estimated effects of the CB only and the CB+CSC treatments on four groups of health service delivery outcomes: (1) service offering and utilization, (2) health center financing and resource management and availability, (3) health center staffing, equipment, and facilities, and (4) other health center-related outcomes. We find no significant effects of the CB only and CB+CSC interventions on service offering (proportion of PMA services offered at health center) and service utilization (index of the number of PMA services provided). We find a couple of significant effects on outcomes related to health center financing and resource management and availability. First, the CB only treatment resulted in an 18 percent reduction in the flat fee charged to cover services, supplies, and drugs, compared to the control group. Second, the CB+CSC treatment increased the likelihood of having an updated inventory or stock control system for drugs compared to the control group by 43 percent. In each case, there is no positive impact of the other treatment on the outcome variable—i.e. there are no statistically significant differences between the flat fee charged at health centers in the CB+CSC group and the control group, and between the likelihood of having an updated inventory or control systems for drugs between the CB only group and the control group. Likewise, there are no significant effects of the CB only and CB+CSC treatments on variables such as the health center’s likelihood of charging a flat fee and the extent to which the community is involved in setting the fee, as well as the proportion of updated financial management tools used and the proportion of essential drugs that are available at the health center. Neither treatment had a significant effect on the health center staffing, equipment, and facilities outcomes at midline or on the availability of suggestion boxes for client feedback. However, the likelihood of health centers in the CB+CSC group having client communication materials was 32 percent higher than in the control group. We find no statistically significant differences between the CB only group and the control group. Table C.2.1. Health Service Delivery OUTCOME CB Only CB+CSC CONTROL Service offering and utilization Proportion of minimum service package (PMA) services provided at health center (Audit) -0.020 -0.008 0.622 (0.022) (0.022) (0.023) Index of service utilization reflecting the number of PMA services provided in the past two months (Audit) -0.029 -0.008 -0.067 (0.074) (0.074) (0.081) CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 43 OUTCOME CB Only CB+CSC CONTROL Health center financing and resource management and availability Health center charges a flat fee that covers all services, supplies, and drugs (Audit) -0.001 0.081 0.576 (0.102) (0.096) (0.064) Amount of flat fee in Congolese francs (CDF) (Audit) -496.358* -281.616 2830.603 (220.105) (218.693) (245.794) Extent to which community is involved in setting fee (Audit) 0.030 0.028 0.576 (0.050) (0.050) (0.036) Extent to which community is involved in setting fee (CODESA) 0.048 0.064 0.398 (0.045) (0.042) (0.028) Proportion of financial management tools used at health center that are up to date (verified) (Audit) -0.088 -0.078 0.357 (0.189) (0.187) (0.043) Health center has inventory or stock control system for drugs that is up to date (verified) (Audit) 0.135 0.234** 0.542 (0.085) (0.081) (0.065) Proportion of essential drugs that are available at health center (verified) (Audit) -0.051 0.011 0.340 (0.030) (0.030) (0.024) Health center staffing, equipment, and facilities Health center has a staff schedule that is up to date (verified) (Audit) -0.118 0.031 0.729 (0.089) (0.083) (0.058) Proportion of staff scheduled to work on the day before the audit who were in attendance (according to staff schedule) (Audit) 0.017 0.030 0.888 (0.028) (0.028) (0.024) Proportion of staff scheduled to work on day of audit whose attendance was verified (Audit) 0.020 0.007 0.831 (0.033) (0.032) (0.026) Cleanliness and condition of health center (enumerator observation) (Audit) 0.004 -0.011 0.506 (0.023) (0.023) (0.018) Other health center-related outcomes Client (patient) communication materials are present in health center (verified) (Audit) 0.051 0.099* 0.305 (0.043) (0.043) (0.036) Suggestion box for client feedback is available at health center (verified) (Audit) 0.082 0.150 0.136 (0.085) (0.086) (0.045) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regression of outcome from the CODESA member survey. Parentheses next to outcome indicate source. Table C.2.2 shows some limited effects of the treatments on health service governance outcomes at midline. Based on answers from the health center audit, the likelihood of the community electing CODESA members is 31 percent higher in the CB only and CB+CSC groups than in the control CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 44 group. The difference between the effects of the two treatments is not statistically significant. Responses from CODESA members differ slightly. According to the data collected through the CODESA survey, there is a 22 percent increase in the likelihood of the community electing CODESA members in the CB only group compared to the control group, but there are no differences between the CB+CSC group and the control group. Furthermore, the results indicate that the CB only and CB+CSC interventions did not produce improvements in the frequency of meetings between the head nurse and the CODESA. Table C.2.2. Health Service Governance OUTCOME CB Only CB+CSC CONTROL CODESA members are elected by the community (Audit) 0.183* 0.186* 0.596 (0.083) (0.082) (0.065) CODESA members are elected by the community (CODESA) 0.148* 0.060 0.667 (0.075) (0.068) (0.044) Frequency of meetings between head nurse and CODESA (Audit) -0.030 -0.029 0.662 (0.021) (0.021) (0.014) Frequency of meetings between head nurse and CODESA (CODESA) 0.020 0.011 0.785 (0.012) (0.010) (0.009) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the CODESA member survey. Parentheses next to outcome indicate source. Table C.2.3 indicates that the CB training alone and the CB training combined with the CSC intervention did not have statistically significant effects on most outcome measures related to households’ experiences visiting the local health center. We find no differences between the treatment groups and the control group regarding wait time, and the likelihoods of receiving care from a healthcare professional, having a medical test performed, receiving a clear diagnosis, finding that medicines prescribed are available, and having to make voluntary payments during visits. Similarly, there are no significant effects at midline of the CB only and the CB+CSC treatment on receiving information about the importance of seeking care and avoiding self-treatment. However, the total amount paid by households in the CB+CSC group is 16 percent lower than households in the control group, and households in the CB only group are 27 percent more likely to receive information about contraception than households in the control group. We did not find significant differences between the CB and control groups, and between the CB+CSC and the control groups in terms of amount paid during visit, and receiving information on family planning methods, respectively. Table C.2.3. Households’ Experiences Visiting Local Health Center OUTCOME CB Only CB+CSC CONTROL Wait time (Household) 0.023 -0.020 0.248 (0.022) (0.017) (0.011) Received care from healthcare professional (Household) 0.019 0.013 0.935 (0.011) (0.012) (0.008) CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 45 OUTCOME CB Only CB+CSC CONTROL Medical test was performed during visit (Household) 0.016 -0.011 0.567 (0.027) (0.028) (0.016) Diagnosis was clearly explained (Household) 0.046 -0.009 0.692 (0.026) (0.026) (0.015) Medicine was available (if prescribed) (Household) 0.037 0.033 0.711 (0.031) (0.031) (0.016) Total amount paid during visit (Household) -1153.141 -2210.842* 13949.604 (1090.524) (1092.853) (559.751) Had to make voluntary payment (i.e., bribes) during visit (Household) -0.001 -0.004 0.015 (0.008) (0.008) (0.004) Received information about family planning methods / contraception (Household) 0.081*** 0.025 0.285 (0.023) (0.024) (0.013) Received information about the importance of seeking care and avoiding self-treatment (Household) 0.034 0.008 0.454 (0.027) (0.026) (0.015) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household survey. Parentheses next to outcome indicate source. C.3 PERCEPTIONS ABOUT HEALTH CARE PROVISION AND GOVERNANCE Table C.3.1 shows that the CB only treatment and CB+CSC treatment did not produce any improvements in households’ assessments of ease in obtaining care and satisfaction with the quality of services during visit. Table C.3.1. Households’ Assessments about Health Care Services OUTCOME CB Only CB+CSC CONTROL Ease in obtaining care during visit (Household) 0.002 0.018 0.750 (0.023) (0.023) (0.014) Satisfaction with quality of services during visit (Household) -0.002 -0.014 0.791 (0.020) (0.020) (0.013) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household survey. Parentheses next to outcome indicate source. Table C.3.2 reports the estimated effects of the CB only treatment and CB+CSC treatment on households’ opinions about the health center and health governance, as well as the opinions of CODESA and household survey respondents’ on CODESA performance. We find no significant effects for any household-level indicator evaluated. Nevertheless, CODESA respondents from the CB+CSC group have a more favorable opinion on the CODESA performance (measure is ten percent higher) than CODESA respondents from the control group. The CB only treatment did not produce similar results. CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 46 Table C.3.2. Opinions about Health Center, Health Governance, and CODESA Performance OUTCOME CB ONLY CB+CSC CONTROL Index of general health center quality (Household) 0.017 -0.001 0.218 (0.012) (0.012) (0.005) Index of general community empowerment vis-à-vis health center staff (Household) 0.012 -0.006 0.523 (0.017) (0.016) (0.010) Belief in community’s ability to improve health care at local health center (Household) 0.009 0.023 0.261 (0.020) (0.021) (0.013) Respondents’ opinions on CODESA performance (Household) -0.041 -0.089 2.813 (0.065) (0.068) (0.035) Respondents’ opinions on CODESA performance (CODESA) 0.033 0.069** 0.663 (0.024) (0.022) (0.014) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household and CODESA member surveys. Parentheses next to outcome indicate source. C.4 HEALTH SEEKING AND PROMOTING BEHAVIORS Table C.4.1 and Table C.4.2 show that the treatments did not have statistically significant effects on under-five immunization and health seeking behavior, respectively. Table C.4.1 shows results for the likelihood of a child having an immunization record and the proportion of immunizations they have received. Table C.4.2 shows results for the likelihood of seeking care when a household member under five years old has diarrhea and when a household member five years old or older is sick. Table C.4.1. Under-five Immunization OUTCOME CB ONLY CB+CSC CONTROL Child has an immunization record (Household) 0.000 -0.010 0.150 (0.021) (0.019) (0.009) Proportion of immunizations that child has received (Household) 0.021 0.028 0.691 (0.020) (0.020) (0.010) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household survey. Parentheses next to outcome indicate source. Table C.4.2. Health Seeking Behavior OUTCOME CB Only CB+CSC CONTROL Following diarrhea of household member under 5 y.o. Sought care (Household) 0.017 0.045 0.708 (0.044) (0.041) (0.028) CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 47 OUTCOME CB Only CB+CSC CONTROL Number of days between illness onset and care was sought (Household) 0.220 0.090 1.679 (0.131) (0.127) (0.088) Sought care at local health center (Household) -0.020 0.030 0.630 (0.049) (0.049) (0.036) Following illness of household member (5 y.o. or older) Sought care (Household) -0.007 0.008 0.831 (0.027) (0.031) (0.020) Number of days between illness onset and care was sought (Household) -0.029 -0.091 2.655 (0.193) (0.204) (0.129) Sought care at local health center (Household) 0.031 0.069 0.593 (0.040) (0.040) (0.029) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household survey. Parentheses next to outcome indicate source. Table C.4.3 shows the effects of the CB only and CB+CSC treatments on antenatal care for pregnant women and women who recently gave birth, as well as on postnatal care for women who recently gave birth. The CB only treatment did not produce significant impacts on any of the outcomes we considered. The same is true for the CB+CSC treatment, with the exception of one indicator: women who recently gave birth in the CB only and CB+CSC groups are four and five percent, respectively, more likely to seek antenatal care than women in the control group. Table C.4.3. Antenatal and Postnatal Care OUTCOME CB Only CB+CSC CONTROL Pregnant women sought antenatal care (Household) 0.071 0.054 0.396 (0.047) (0.047) (0.034) Pregnant women received antenatal at the local health center (Household) 0.081 0.025 0.800 (0.074) (0.099) (0.045) Woman who recently gave birth sought antenatal care (Household) 0.034* 0.048*** 0.954 (0.015) (0.014) (0.009) Woman who recently gave birth received antenatal care at the local health center (Household) 0.003 0.067 0.778 (0.042) (0.040) (0.019) Woman who recently gave birth delivered at the local health center (Household) 0.012 0.079 0.562 (0.045) (0.047) (0.022) Woman who recently gave birth received postnatal care (Household) 0.025 0.008 0.500 (0.030) (0.031) (0.022) CONTRACT NO. GS-10F-0033M/AID-0AA-M-13-00013 / DRG-LER TASKING N035 USAID.GOV IMPACT EVALUATION OF USAID/DRC IGA – MIDLINE REPORT | 48 OUTCOME CB Only CB+CSC CONTROL Woman who recently gave birth received postnatal care at the local health center (Household) 0.016 0.084 0.565 (0.054) (0.059) (0.031) Note: ***p<0.001, **p<0.01, *p<0.05. Standard errors are adjusted for clustering at the HA level in regressions of outcomes from the household survey. Parentheses next to outcome indicate source. C.5 HEALTH OUTCOMES Table C.5.1 displays commonly used health outcome indicators at the health area level. We report infant mortality, child mortality, and diarrhea incidence to gauge child health; the stillbirth and maternal mortality rates to measure maternal health; and incidence of illnesses in the over-five population as an indicator of general community health. We find no statistically significant impacts associated with the treatments, except for one indicator. The incidence of diarrhea is 31 percent and 21 percent higher in the CB only group and the CB+CSC group, respectively, than in the control group. This result, not driven by outliers, is surprising and counterintuitive. Table C.5.1. Health Outcomes OUTCOME CB Only CB+CSC CONTROL Child health Infant mortality (Household) 12.629 -26.552 101.036 (28.056) (27.496) (19.359) Child mortality (Household) 111.254 -4.822 282.239 (68.907) (67.347) (35.707) Diarrhea incidence (Household) 0.048*** 0.032* 0.153 (0.014) (0.014) (0.010) Maternal health Stillbirth rate (Household) -1.912 5.420 28.559 (11.798) (11.677) (7.056) Maternal mortality rate (Household) -461.760 -135.532 621.469 (527.910) (517.183) (437.499) Community health Incidence of over-five illness (Household) 0.011 -0.004 0.074 (0.006) (0.006) (0.006) Note: ***p<0.001, **p<0.01, *p<0.05. All of these outcomes are at the HA level. Parentheses next to outcome indicate source. U.S. AGENCY FOR INTERNATIONAL DEVELOPMENT 1300 Pennsylvania Avenue, NW Washington, DC 20523