Cost-effectiveness Analysis Comparing Integrated and Malaria-only Social and Behavior Change Programming in Nigeria: Final report TECHNICAL REPORT MAY 2023 Acknowledgments We would like to thank the Breakthrough ACTION partners for contributing their time and data for this report and our Breakthrough RESEARCH USAID management team and colleagues at the U.S. President’s Malaria Initiative who provided suggestions and guidance throughout. Breakthrough RESEARCH is funded by the U.S. Agency for International Development (USAID) and U.S. President’s Malaria Initiative under the terms of Cooperative Agreement No. AID-OAA-A-17-00018. The contents of this document are the sole responsibility of Breakthrough RESEARCH and Population Council and do not necessarily reflect the views of USAID or the United States Government. Breakthrough RESEARCH catalyzes social and behavior change (SBC) by conducting state-of-the-art research and evaluation and promoting evidence-based solutions to improve health and development programs around the world. Breakthrough RESEARCH is a consortium led by the Population Council in partnership with Avenir Health, ideas42, Institute for Reproductive Health at Georgetown University, Population Reference Bureau, and Tulane University. The Population Council confronts critical health and development issues—from stopping the spread of HIV to improving reproductive health and ensuring that young people lead full and productive lives. Through biomedical, social science and public health research in about 50 countries, the Council works with our partners to deliver solutions that lead to more effective policies, programs, and technologies to improve lives worldwide. Established in 1952 and headquartered in New York, the Council is a nongovernmental, nonprofit organization with an international board of trustees. Avenir Health was founded in 2006 as a global health organization that works to enhance social and economic development by providing tools and technical assistance in policy, planning, resource allocation and evaluation. Its focus is on developing and implementing demographic, epidemiological and costing models for long-range planning to assist with setting goals, strategies, and objectives. Avenir Health assists in both developing and implementing programs in HIV/AIDS, reproductive health, maternal health and other programming areas. Avenir Health works with government agencies, foundations, corporations, and nongovernmental organizations around the world. ©2023 The Population Council. All rights reserved. Cover photo USG works Suggested Citation Avenir Health. 2023. "Cost-effectiveness analysis comparing integrated and malaria-only social and behavior change programming in Nigeria: Final report," Breakthrough RESEARCH Technical Report. Washington, DC: Population Council. Contact 4301 Connecticut Avenue NW, Suite 280 | Washington, DC 20008 +1 202 237 9400 | BreakthroughResearch@popcouncil.org breakthroughactionandresearch.org Cost-effectiveness Analysis Comparing Integrated and Malaria-only Social and Behavior Change Programming in Nigeria: Final report TECHNICAL REPORT MAY 2023 Avenir Health List of Acronyms ACT artemisinin-based combination therapy ANC antenatal care BSS behavioral sentinel surveillance CCP Johns Hopkins Center for Communication Programs COVID-19 coronavirus disease 2019 DALY disability-adjusted live year DPT3 diptheria-pertussis-tetanus series GDP gross domestic product GHSA Global Health Security Agenda ICER incremental cost-effectiveness ratio IHP Integrated Health Program IPTp intermittent preventive treatment in pregnancy ITN insecticide treated net LiST Lives Saved Tool ORS oral rehydration salts SBC social and behavior change TB tuberculosis USAID United States Agency for International Development US$ United States Dollar WHO World Health Organization YLD years of life lost to death YLL years of life lived with disability ii COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT Table of Contents List of Acronyms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Estimating impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Estimating expenditures/costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Calculating cost-effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 RESULTS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Expenditures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Cost-effectiveness.................................................................................... 14 DISCUSSION. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Question 1—What are the total program expenditures incurred during the total study time period from program initiation (April 2018) through October 2022? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Question 2—Are the expenditures required for an integrated SBC program, compared to malaria-only SBC program, a cost-effective investment? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Question 3—How do malaria-specific outcomes perform within an integrated SBC program compared to a malaria-only program? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 REFERENCES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 APPENDICES Appendix A: Population data on Breakthrough ACTION intervention coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Appendix B: State-specific results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Appendix C: Further SBC expenditures details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Appendix D: Examining ICER with varying levels of impact attribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 BREAKTHROUGH RESEARCH | MAY 2023 iii iv COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT Executive Summary In Nigeria, the Breakthrough ACTION project, funded by the United States Agency for International Development (USAID)/Nigeria, is working closely with the Nigerian government to improve the practice of priority health behaviors, with a focus on maternal, newborn and child health and nutrition, family planning and reproductive health, malaria, and tuberculosis. Within the broader program, Breakthrough ACTION uses different social and behavior change (SBC) approaches in different states. In northern Nigeria, Breakthrough RESEARCH compared the cost-effectiveness of the integrated SBC approach addressing multiple health areas in Kebbi and Sokoto states to a malaria-only approach in Zamfara. This cost-effectiveness study examined whether the additional cost required for an integrated SBC approach (as compared to a malaria￾only approach) is a cost-effective investment. Additionally, this study examined malaria-specific impacts within an integrated SBC program compared to a malaria-only program. Methods There are three fundamental steps for estimating the relative cost-effectiveness of the two different SBC approaches: 1) estimate the impact, 2) estimate the costs, and 3) calculate the relative incremental cost￾effectiveness ratio (ICER) using a difference-in-difference approach. The foundation for the impact analysis is the Breakthrough RESEARCH baseline (2019) and endline (2022) behavioral sentinel surveillance (BSS) surveys, which include 16 health behavioral outcomes related to Breakthrough ACTION’s SBC programming that are available for modeling in the Lives Saved Tool (LiST). LiST is a linear deterministic causal model that calculates how changes in population coverage of specific interventions result in lives saved over a specified time period. Two scenarios were generated: one that used all 16 outcomes, and another that only used the outcomes that showed a statistically significant difference between the integrated state(s) and malaria-only state from baseline to endline, based on a difference-in-differences analysis. The number of lives saved from the LiST scenarios were then translated into disability-adjusted life years (DALYs) averted using data obtained from the Global Burden of Disease (GBD) Results Tool for Nigeria. Program and personnel expenditure data were provided by Breakthrough ACTION. SBC expenditures were summarized for each of the three study states, examining all SBC expenditures and malaria-only expenditures. In addition to SBC expenditures, service delivery costs associated with changes in the behavioral health outcomes were estimated using default values from the LiST costing module and were also included. All costs were adjusted to 2022 US dollars (US$) using the U.S. gross domestic product (GDP) deflator. Once the impacts and costs were obtained, the ICER was calculated as the cost per DALY averted, measuring the additional cost needed to achieve the additional impact from the integrated states vs. the malaria-only state. The cost per DALY averted was then compared to the GDP per capita to assess cost-effectiveness, where health interventions with a cost per DALY averted that are less than the GDP per capita are considered “highly cost￾effective” and those between one- and three-times GDP per capita are “cost-effective”.1 The most recent GDP per capita estimates were used and adjusted to 2022 US$, resulting in a national threshold of $2,252 and a three￾state GDP average of $860. Results The results of the scenario using all health outcomes measured by the BSS resulted in a net of 967 lives saved in the integrated SBC states and 555 lives lost in the malaria-only SBC state. When looking at the outcome￾specific results, one of the biggest changes is from a reduction in insecticide treated net (ITN) ownership from baseline to endline in the integrated states, resulting in a loss of 1,631 lives in the integrated states. This contrasts a gain of 403 lives in Zamfara, where there was an increase in ITN ownership. Antibiotic use for respiratory illness also showed substantial gains and losses, with a gain of 1,590 lives in the integrated states and a loss of 1,211 lives in the malaria-only state. The change in the use of oral rehydration salts (ORS) and zinc for diarrhea BR E A K THROUGH R ESE A RCH | M AY 2023 1 from baseline to endline results in lives lost in a gain of 443 lives in the integrated states and a loss of 762 in the malaria-only state. When translating the number of lives saved to the additional DALYs averted in the integrated SBC states, compared to the malaria-only state, the scenarios in Kebbi and Sokoto yielded 47,605 more DALYs averted than the malaria-only scenario in Zamfara. For malaria-specific DALYs, however, the malaria-only SBC approach in Zamfara yielded a net DALY advantage of over 66,000 DALYs compared to integrated SBC, due to a drop in ITN ownership in the integrated SBC states from baseline to endline. The SBC expenditures during the study timeframe totaled $8.1 million in Kebbi (integrated SBC), $7.2 million in Sokoto (integrated SBC), and $3.0 million in Zamfara (malaria-only SBC). The SBC expenditures are then combined with the service delivery costs associated with changes in the behavioral outcomes to get total costs. When combined with impact, the ICER (meaning the cost per additional DALY averted in the integrated states compared to the malaria-only state) is $278 for the scaled-up scenario and $426 for the limited scaled-up scenario. When compared to the national GDP per capita of $2,252 and the three-state average GDP per capita of $860, both results are below the “highly cost-effective” threshold. Because the SBC interventions are not delivered in a vacuum and it is likely that some of the impact captured by the BSS is attributable to factors other than SBC, further analysis examined the proportion of the impact (the additional DALYs averted in the integrated SBC areas) that would need to be attributed to SBC interventions for the investment in integrated SBC states to be cost￾effective. The most difficult threshold to reach is that for the three-state average highly cost-effective at $860. Using this threshold in the scaled-up scenario, 33% of the impact seen in the BSS from baseline to endline would need to be due to SBC to be considered highly cost-effective. This increases to 50% using the limited scaled-up scenario. To be considered cost-effective, however, based on the three-state average threshold, only 11% of impact needs to be attributed to SBC in the scaled-up scenario or 17% in the limited scaled-up scenario. Reaching cost-effectiveness is easier using the national thresholds, where only 13% of impact needs to be attributed to SBC to be considered highly cost￾effective in the scaled-up scenario; 19% is required in the limited scaled-up scenario. Finally, using the national threshold for cost-effectiveness, only 5% of impact must be attributed to SBC under the scaled-up scenario, which increases to 7% in the limited scaled-up scenario. Discussion The overall ICER calculations indicate that the additional investments for integrated SBC relative to malaria-only SBC are highly cost-effective based on both national and regional thresholds. The primary drivers of the positive results are the increased use of antibiotics for respiratory infections and the use of ORS and zinc for diarrhea in the integrated states versus the drop in use in the malaria￾only state. While these priority health behaviors are addressed in Breakthrough ACTION’s SBC programming in the integrated states in both community activities (household visits, community dialogues, and community meetings) and mass media programming, in addition to Breakthrough ACTION programming, there are other differences between the integrated and malaria-only states that are likely contributing to these impacts. These include antibiotic stockouts in the malaria-only state of Zamfara and the presence of the Integrated Health Program (IHP), which is working in Kebbi and Sokoto on a complementary project to improve these outcomes through improved primary care services. In contrast to the overall findings, the malaria-specific outcomes do not appear to be well served by integrated SBC. However, there is also an important caveat to consider when interpreting these results, as an ITN distribution campaign was conducted in the malaria￾only state during the study time frame but not in the integrated SBC states. It is notable, that while there was a drop in ITN ownership, ITN use among pregnant women and young children in the integrated states remained stable from baseline to endline, while increasing substantially in Zamfara. Unfortunately, the ITN use figures cannot be used in LiST due to the underlying parameters operating in the model. Still, while a change to ITN use instead of ownership would be more favorable to the integrated states, it would not change the conclusions that the integrated SBC investments were deemed highly cost-effective but malaria-only results fared better in Zamfara. The primary limitation of this study is that Zamfara does not appear to be a good comparator district for Kebbi and Sokoto, due to the stockouts of antibiotics in Zamfara, the IHP program presence in Kebbi and Sokoto, and the distribution of ITNs in Zamfara but not in Kebbi and Sokoto during the study period. However, despite 2 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT this and other limitations, the findings indicate that even if only a relatively small proportion of the overall impact modeled is attributed to the SBC activities, the SBC investments would be cost-effective. As such, these findings are promising for the cost-effectiveness of integrated SBC, but not definitive. Future research should continue to explore the cost-effectiveness of integrated SBC. BR E A K THROUGH R ESE A RCH | M AY 2023 3 Background In the last decade, many social and behavior change (SBC) programs in health have shifted from working in a single health area to a more integrated approach that includes multiple health areas and communication channels.2 While it is hypothesized that integrated SBC will result in programs that are more reflective of clients’ needs and thus more cost-effective, these claims are largely unproven given the lack of robust studies of integrated SBC approaches.3,4 Additionally, while some studies have examined the costs of integrated SBC programs, no studies were identified that compared the costs of an integrated SBC approach to a stand-alone approach.5 To address these knowledge gaps, Breakthrough RESEARCH compared the cost-effectiveness of the integrated SBC approach versus malaria-only SBC approach in three states (Kebbi, Sokoto and Zamfara) of northwestern Nigeria. In Nigeria, the Breakthrough ACTION project, funded by the United States Agency for International Development (USAID)/Nigeria, is working closely with the Nigerian government to improve the practice of priority health behaviors, with a focus on maternal, newborn and child health and nutrition, family planning and reproductive health, malaria, and tuberculosis (TB). Breakthrough ACTION began implementation in April 2018, led by Johns Hopkins Center for Communication Programs (CCP) with partners Save the Children International, ThinkPlace, Ideas42, and Viamo.a Breakthrough ACTION uses different SBC approaches in different states. In northern Nigeria, an integrated SBC approach started in Bauchi, Kebbi, and Sokoto, where SBC activities focus on multiple health behaviors (malaria, family planning, maternal and child health, and nutrition) for women of reproductive age who are either currently pregnant or are within the 1,000-day window following childbirth.b In contrast, a malaria-only SBC approach is being used in Zamfara state. Some of the SBC interventions that have been conducted by Breakthrough ACTION in Nigeria include advocacy efforts involving opinion leaders and community influencers, community health dialogues with individual and group interpersonal communication, radio programming, mobile digital interventions, and provider behavior change focused on addressing barriers to malaria diagnosis and treatment, family planning uptake, positive maternal and child health seeking behavior, as well as nutrition. a Additionally, the Centre for Communication and Social Impact is a Nigerian non-governmental organization that is a subaward under CCP and oversees community activities in malaria-only intervention states. b Integrated programming extended to Ebonyi state and the Federal Capital Territory starting in late 2020. Breakthrough RESEARCH led an evaluation of Breakthrough ACTION’s integrated (malaria, family planning, maternal and child health, and nutrition) and vertical (malaria-only) SBC programming in northern Nigeria, including a study that explored the relative cost-effectiveness of the two approaches. The cost￾effectiveness activity consists of three consecutive reports. First, the initial phase costing report examined expenditures from the program inception in April 2018 through December 2019, differentiating between start-up and implementation expenditures during the initial phase of the program.6 A midline report was aligned to the midline behavioral sentinel surveillance (BSS) survey and reported SBC expenditures at midline (December 2020) and an analysis of the impact of the COVID-19 pandemic on SBC expenditures.7 This third and final report is focused on calculating the cost-effectiveness of the integrated SBC approach vs. malaria-only SBC approach interventions from April 2018 through October 2022, when the endline BSS survey was conducted. Note that the Breakthrough ACTION program has been extended beyond the Breakthrough RESEARCH evaluation timeframe to 2025; however, no further BSS is planned at this time. Figure 1 details the timeline for the cost data collection and BSS surveys. The overall study period includes the beginning of the coronavirus disease 2019 (COVID-19) global pandemic, which disrupted Breakthrough ACTION Nigeria’s community SBC activities for several months starting February 2020 through January 2021.6 Additionally, local unrest and security concerns in northwest Nigeria escalated in 2021, resulting in a temporary cessation of program implementation in parts of Sokoto and Zamfara states. Despite these program disruptions, the continued 4 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT collection of SBC-related expenditures and data from the endline BSS allow for an examination of the relative cost-effectiveness of the programming between the integrated (Kebbi and Sokoto) and malaria-only (Zamfara) states as of October 2022. There are three primary research questions addressed in this report: 1. What are the total program expenditures incurred during the total study time period from program initiation (April 2018) through October 2022? 2. Are the investments needed for an integrated SBC program cost-effective, compared to a malaria-only approach? 3. How do malaria-specific investments perform within an integrated SBC program compared to a malaria￾only program? FIGURE 1 EVALUATION TIMELINE Initial costing phase April 2018– December 2019 Only design￾related expenditures are included in this period Baseline BSS Midline costing phase January 2020– December 2021 Midline BSS Final costing phase January– October 2022 Final BSS Impact timeframe Costing timeframe BR E A K THROUGH R ESE A RCH | M AY 2023 5 Methods There are three fundamental steps for estimating the relative cost-effectiveness of the two different SBC approaches: 1) estimate the impact of each, 2) estimate the costs of each, and 3) calculate the relative incremental cost-effectiveness ratio (ICER) using a difference-in-difference approach. Estimating Impact The foundation for the impact analysis is the Breakthrough RESEARCH Nigeria BSS study, which is a multi-round quasi-experimental study designed to assess the impact of Breakthrough ACTION programming implemented in two integrated SBC states (Kebbi and Sokoto) compared to a malaria-only SBC approach in Zamfara. The baseline, midline, and endline BSS involve interviews with women with a child under two years of age in Breakthrough ACTION programming areas and assess the ideational factors, behaviors, and outcomes associated with key health areas. This analysis relies on the behavioral health outcome findings from the women’s sample, with 3,020 women interviewed among the three states at baseline in December 2019 and 3,144 in the endline BSS in October 2022.8 Further details on the Breakthrough RESEARCH Nigeria study and the BSS methodology can be found at: [insert link to endline report on website]. To examine the relative cost-effectiveness of an integrated vs. malaria-only SBC approach, a common impact measure is needed that can be compared across multiple health areas. To achieve this, we used the health behavioral outcomes from the BSS as inputs into the Lives Saved Tool (LiST). LiST is a linear deterministic causal model that was initially developed in 2003 to estimate the impact of clinical, hospital, and community-based interventions on mortality for children under five years and expanded to include maternal mortality.9 LiST is widely used by donors, policy makers, academics, and health system stakeholders to examine the impact associated with scaling-up key maternal and child health, nutrition, and water, sanitation and hygiene interventions, including the Bill & Melinda Gates Foundation, Gavi, the World Bank Group, Johns Hopkins, and UNICEF (www.livessavedtool.org/projects). The tool examines how changes in population-level coverage of specific interventions translate into the number of lives saved over a specified time period. For example, Figure 2 shows how the ownership and use of insecticide treated nets (ITNs) results in decreased deaths among children and mothers. Underlying parameters in the tool use validated research studies that capture the impact of ITN ownership, which is then translated to ITN use through an efficacy factor, to malaria deaths for children and maternal deaths from hemorrhage due to maternal anemia.10 Table 1 details the 16 behavioral health outcomes assessed by the baseline and endline surveys in each state that are available to be used in LiST modeling and the percentage point change for each outcome in each state. Three outcomes are specific to malaria— ownership of an ITN, at least two doses of intermittent preventive treatment in pregnancy (IPTp), and the use of artemisinin-based combination therapy (ACT) for fever FIGURE 2 MAPPING IMPROVEMENTS IN ITN OWNERSHIP TO REDUCED DEATHS IN LIST Source: www.Listvisualizer.org 6 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT in children under two years old. Other reproductive, maternal, and child health behavioral outcomes include the use of modern contraception among married/ in-union women, attending at least one antenatal (ANC) visit and/or at least four ANC visits during the most recent pregnancy, breastfeeding behaviors, completing the diptheria-pertussis-tetanus series (DPT3) and measles vaccines, the use of oral antibiotics for child respiratory infections, and the use of oral rehydration salts (ORS) and zinc for childhood diarrhea. While several changes occurred between the baseline and endline BSS, only four of the behavioral health outcomes had statistically significant changes (p<0.05) when comparing the integrated SBC states to the malaria-only SBC state using a difference-in-differences analysis, with one of the indicators having a negative significant change.8 As the name implies, difference-in￾differences analyses examine two sets of differences at the same time. The first difference is the change in an indicator (e.g., current modern family planning use) from baseline to endline in an integrated state. This difference is then compared with the difference from baseline to endline in the same indicator in the comparison state. In looking at the statistically significant results from the difference-in-differences analyses, first, ITN ownership declined substantially in the integrated states of Kebbi and Sokoto (32 and 19 percentage points, respectively), while it increased by 13 percentage points in the malaria-only state of Zamfara. Conversely, the use of oral antibiotics increased in Sokoto by 13 percentage points and decreased in Zamfara by 22 percentage points. The use of ORS also increased in Sokoto by 9 percentage points and decreased in Zamfara by 14 percentage points. Finally, the use of zinc increased in all three states, but more so in Sokoto (16 percentage points) compared to Zamfara (5 percentage points). These changes are shown in Figure 3. TABLE 1 BEHAVIORAL HEALTH OUTCOME CHANGES FROM BASELINE TO ENDLINE BSS AVAILABLE FOR USE IN LIST BEHAVIORAL HEALTH OUTCOMES KEBBI SOKOTO ZAMFARA BASELINE ENDLINE % PT CHANGE BASELINE ENDLINE % PT CHANGE BASELINE ENDLINE % PT CHANGE Owns at least one ITN 77.9 45.6 -32.3* 79.7 61.2 -18.5* 74.4 87.7 13.3 IPTp among pregnant women (at least 2) 33.5 47.0 13.5 25.6 33.1 7.5 38.3 42.3 4 ACT for children under 2 28.8 23.9 -4.9 17.9 27 9.1 26.1 34.5 8.4 Modern contraceptive prevalence 8.6 15.0 6.4 10.7 8.7 -2 14.7 18.1 3.4 At least 1 ANC visit 42.1 45.9 3.8 24.6 31.9 7.3 38.2 48.5 10.3 At least 4 ANC visits 23.6 32.0 8.4 17 21.9 4.9 26.1 37.7 11.6 Facility-based birth 14.8 22.7 7.9 13.8 13.6 -0.2 16.3 26.6 10.3 Early initiation of breastfeeding 41.6 39.2 -2.4 31.6 24.2 -7.4 46.1 43.2 -2.9 Exclusive breastfeeding (under 1 month) 8.9 19.8 10.9 27.3 9.2 -18.1 43.6 22.3 -21.3 Exclusive breastfeeding (under 6 months) 20.3 16.6 -3.7 29.3 11 -18.3 45.9 37.4 -8.5 Any breastfeeding (6+ months) 97.9 96.3 -1.6 97.1 98.6 1.5 94.9 96.6 1.7 DPT vaccine 5.7 12.2 6.5 9.6 6.5 -3.1 10.7 18.1 7.4 Measles vaccine 15.9 18.9 3.0 16.5 12 -4.5 19.2 22.5 3.3 Oral antibiotics for respiratory illnesses 38.3 58.7 20.4 20.7 33.3 12.6* 45.1 22.9 -22.2 ORS for diarrhea 51.8 47.7 -4.1 27.2 37 9.8* 56 42.4 -13.6 Zinc for diarrhea 30.4 45.0 14.6 18.7 34.8 16.1* 36.8 41.4 4.6 *Indicates a statistically significant difference (p<0.05) when compared to changes in Zamfara using a difference-in-differences analysis. BR E A K THROUGH R ESE A RCH | M AY 2023 7 Other outcomes related to ITNs that are not available in LiST but are of interest in terms of the effectiveness of malaria SBC interventions are shown in Table 2. Note that while ITN ownership declined substantially in Kebbi and Sokoto from baseline to endline as shown in Table 1, the actual use of ITNs by pregnant women and children did not decline substantially from baseline to endline, but rather made small improvements, albeit a very slight reduction (0.4 percentage points) for children under two years in Kebbi. These small changes, however, still contrast with the substantial gains in net use in Zamfara. While it would be preferred to use the proportion of pregnant women and young children sleeping under an ITN as reported in the BSS for the LiST modeling, unfortunately, this is not feasible due to the underlying parameters that link the behaviors to the impacts, as shown in Figure 2. The percentage change values shown in Table 1 were used to populate a LiST file for each of the three states. Each state-specific file uses inputs that have been previously validated by stakeholders at the state￾level. To begin, the default population data from LiST were adjusted to reflect the population of those living in the Breakthrough ACTION intervention wards for community-level interventions at midline, as provided by Breakthrough ACTION. Appendix A details the intervention wards and population estimates used for each state. The proportion of the state population living in the intervention wards amounted to 60% in Kebbi, 53% in Sokoto, and 25% in Zamfara, which translates into over 6 million living in the intervention wards in the integrated states versus approximately 1.3 million in malaria-only state at midline. Next, three LiST scenarios were run for each state: 1. A “baseline” scenario that uses the baseline BSS values for each of the 16 behavioral health outcomes and keeps these values constant from 2019 through 2022. 2. A “scaled-up” scenario that uses the baseline BSS values for 2019, the endline BSS values for 2022, and uses linear interpolation between 2019 (baseline) and 2022 (endline) to estimate outcome values for FIGURE 3 PERCENTAGE POINT CHANGES FROM BASELINE TO ENDLINE FOR HEALTH OUTCOMES THAT WERE STATISTICALLY SIGNIFICANTLY DIFFERENT WHEN COMPARING KEBBI AND/ OR SOKOTO TO ZAMFARA ORS for diarrhea -32.3 Zinc for diarrhea ORS for diarrhea Oral antibiotics for respiratory infections Owns at least 1 ITN -18.5 Kebbi Percentage point change from baseline to endline Sokoto Zamfara 13.3 20.4 12.6 -22.2 -4.1 9.8 -13.6 14.6 16.1 4.6 TABLE 2 ADDITIONAL INDICATORS RELATED TO ITNS NOT AVAILABLE IN THE LIST MALARIA INDICATOR KEBBI SOKOTO ZAMFARA Pregnant women sleeping under a long-lasting insecticide treated net (LLIN) 22.7%–27.0% (+4.3) 24.0%–26.4% (+2.4) 30.5%–70.5% (+40.0%) Children under 2 sleeping under an LLIN 31.0%–30.6% (-0.4) 37.1%–39.8% (+2.7) 40.1%–78.5% (+38.4%) 8 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT 2020 and 2021, when BSS data were not collected for all the behavioral health outcomes measured in the BSS. 3. A “limited scaled-up” scenario similar to the scaled-up file, but this scenario only uses values for the behavioral health outcomes that had statistically significant changes from baseline to endline in the BSS. The number of lives saved was estimated for each state based on comparing both the scaled-up and limited scaled-up scenarios to the baseline scenario. Due to underlying population dynamics, the best approach for calculating the number of lives saved based on changes in the behavioral health outcomes differs for mothers and children.c,11 To calculate maternal lives saved, the number of deaths from the two scaled-up scenarios measuring program impact was subtracted from the baseline scenario for each cause of death. For children, the number of lives saved were estimated for each intervention in the scaled-up scenarios. The number of lives saved from LiST were then translated into disability-adjusted life years (DALYs) averted using data obtained from the Global Burden of Disease (GBD) Results Tool for Nigeria. Total DALYs averted are the years of life lost to death (YLL) and the years of life lived with disability (YLD). Their value lies in being able to aggregate outcomes along a common metric versus using a variety of disparate outcomes across different health c Increases in contraception averts maternal deaths, which are best captured when subtracting the deaths from the scaled-up scenario from the deaths in the baseline scenario. Using this approach for children however, results in an overestimate of deaths averted because it includes the prevention of deaths of children who were never born due to contraception increases. As such, using the “lives saved” estimates from LiST is preferable when examining impacts on children. domains, such as the number of new users of modern contraception, the number of fully vaccinated children, or the number of health facility visits for treatment of fever. Using DALYS also permits the aggregation of both mortality and morbidity from health causes. Figure 4 shows an illustrative example of how changes in health behaviors map to DALYs averted.12 To estimate the number of DALYs averted due to changes in behavioral health outcomes in Nigeria, we calculated the number of DALYs per death for each relevant cause of death and applied it to the estimated number of lives saved from LiST. The resulting DALYs averted were then discounted at 3%, following standard practices for cost-effectiveness evaluation.13 The DALYs averted for Kebbi and Sokoto states were aggregated to compare integrated SBC to the malaria-only approach in Zamfara. See Appendix B for state-specific results. Estimating expenditures/costs What follows is a brief synopsis of the methods employed to arrive at estimates of expenditures for the cost-effectiveness analysis of Breakthrough ACTION’s integrated and malaria-only SBC programs. For a more detailed and in-depth examination of the methodology and results, please refer to Appendix C. SBC program implementation and personnel expenditured data were provided through Breakthrough ACTION’s financial and accounting system for the three costing phases of the evaluation timeline as outlined in Figure 1: initial d Program implementation expenditures include all funding expended on training, equipment, supplies, travel, utilities, and other overheads, etc., for the implementation of each program area. Personnel costs were isolated from these other program expenditures due to the way in which they were captured and presented in the data provided by Breakthrough ACTION. FIGURE 3 PERCENTAGE POINT CHANGES FROM BASELINE TO ENDLINE FOR HEALTH OUTCOMES THAT WERE STATISTICALLY SIGNIFICANTLY DIFFERENT WHEN COMPARING KEBBI AND/ OR SOKOTO TO ZAMFARA ORS for diarrhea FIGURE 4 ILLUSTRATIVE EXAMPLE OF AGGREGATING HEALTH IMPACTS INTO DALYS AVERTED SPECIFIC HEALTH IMPACTS malaria infections maternity complications vaccine-preventable diseases COMMON HEALTH IMPACT disability-adjusted life years averted (DALY) HEALTH BEHAVIORS use of insecticide-treated bed nets and preventive treatment for pregnant women modern contraception child vaccinations } # BR E A K THROUGH R ESE A RCH | M AY 2023 9 costing phase (April 2018 to December 2019); midline costing phase (January 2020 to December 2021); and the endline costing phase (January 2022 to October 2022). Once expenditure data for each phase were received, they were evaluated and validated through direct communication with Breakthrough ACTION. Expenditure analyses for the initial and midline costing reports were each discussed with Breakthrough ACTION to agree on the approach and findings. SBC program expenditures were aggregated across all three time periods. Since the radio program was delivered state-wide and the impacts are being assessed among the intervention areas, the mass media expenditures were adjusted by multiplying the total mass media expenditures to the percent of the population living in the community SBC program intervention wards in each state as compared to the total population. Malaria-only SBC expenditures in each state were also examined, using the same approach for allocating mass media expenditures. Expenditures for personnel at site-level (the point of implementation) and above-site (Abuja and organizational headquarters) were aggregated, along with partner expenditures to arrive at a total expenditure for personnel for each of the study states. Program design expenditures were included as an important investment throughout the life of the project. Design expenditures were aggregated for each of the initial, midline, and endline costing phases. These were then allocated across the life of the project starting from the year the investment was first made, to the end year of the project in 2025. In addition to SBC-related program and personnel expenditures, we included additional service delivery expenditures that could be associated with the desired changes in behavioral health outcomes. Using the LiST costing module, we estimated the potential change in total intervention service delivery expenditure based on the two modeled scenarios: scaled-up and limited scaled-up. These service delivery expenditures were then added to all other SBC-related expenditures to present an aggregated total. Finally, all costs were adjusted to 2022 US dollars (US$) using the GDP deflator as published by the Federal Reserve Bank of St. Louis, on the FRED Economic Data website (https://fred.stlouisfed.org/series/GDPDEF#). Calculating cost-effectiveness Using the total costs and total impact described above, the incremental cost-effectiveness ratio (ICER) was calculated by examining the additional costs of integrated SBC relative to malaria-only SBC needed to achieve an additional DALY averted.12 This was calculated by dividing the additional costs by the additional impacts: The resulting ICER is the cost per DALY averted for integrated SBC relative to malaria-only SBC. To determine if the cost per DALY averted falls within ranges considered cost-effective by the World Health Organization, the ICER is compared to both the national gross domestic product (GDP) per capita and the average GDP per capita across the three study states. According to World Health Organization’s guidelines, health interventions with a cost per DALY averted that are less than one-times GDP per capita are considered “highly cost-effective” and those with a cost per DALY averted between one- and three-times GDP per capita are “cost-effective”.1 For Nigeria, the most recent estimate for GDP per capita is $2,066 in 2021.14 Because this study is conducted in three specific states in northern Nigeria with lower GDPs per capita than the national GDP per capita, an average of the three state-level GDP per capita values were also included as a regional threshold for cost-effectiveness.e To ensure comparability with the costs, the GDPs per capita were adjusted to 2022 US$; the resulting thresholds are shown in Table 3. A comparison of the ICER with the thresholds in Table 3 was used to determine whether the additional e Kingmakers - State of States - GDP Size Ranking TABLE 3 THRESHOLDS FOR COST- EFFECTIVENESS THREE STATE AVERAGE NATIONAL Cost-effective $2,580 $6,755 Highly cost-effective $ 860 $2,252 Incremental cost￾effectiveness ratio (ICER) = Integrated costs – Malaria only costs Integrated DALYs averted – Malaria only DALYs averted 10 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGR AMMING IN NIGERIA: FINAL REPORT investments needed for integrated SBC are cost-effective based on the health outcomes captured at baseline and endline in the BSS. Further analyses were conducted to see what proportion of the overall impact from baseline to endline would need to be attributed to SBC differences for the additional investments in integrated SBC to be cost￾effective. To estimate these proportions, the ICER was calculated keeping the costs constant but multiplying the total DALYs averted by each percentage point between 1 and 100 and compared to the thresholds. BR E A K THROUGH R ESE A RCH | M AY 2023 11 RESULTS Impact Breakthrough ACTION-Nigeria’s SBC programming aims to improve health behaviors and thus improve the health and wellbeing of the people in northern Nigeria. As such, positive impact in the key health behaviors would facilitate progress toward the Sustainable Development Goals by reducing maternal mortality and preventable deaths in newborns and children under five years.15 The number of lives saved based on the changes in behavior between the baseline and endline BSS in each state as modeled in LiST are shown in Table 4, disaggregated by reproductive, maternal, and child interventions. The number of lives saved in Table 4 corresponds to Table 1, so that increases in coverage of outcomes generate positive lives saved and decreases in coverage result in lives lost (expressed as negative numbers). For the scaled-up scenario, the changes from baseline to endline yield a net of 967 lives saved in the integrated SBC states, and 555 lives lost in the malaria-only SBC state. When looking at the outcome￾specific results, some of the biggest changes are from ITN coverage, resulting in a loss of 1,631 lives in the integrated SBC states where ITN coverage decreased from baseline to endline, respectively, and a gain of 403 lives in Zamfara. Antibiotic use for respiratory illness is another outcome where there are substantial gains and losses in life estimated, resulting in 1,590 lives saved in the integrated SBC states and a loss of 1,211 lives in the malaria-only SBC state. The change in the use of ORS and zinc for diarrhea from baseline to endline results in lives lost in a gain of 443 lives in the integrated SBC states and a loss of 762 in the malaria-only SBC state. The limited scaled-up scenario uses only values for the behavioral health outcomes where there was a statistically significant difference in changes between an integrated SBC and malaria-only SBC state. These include changes in ITN ownership, changes in antibiotics for respiratory infections, ORS/zinc for diarrhea.f In this scenario, there is a net loss of 571 lives in the integrated SBC states due to the elimination of the non-statistically f The difference-in-difference analyses found statistically significant (p<0.05) changes in ITN ownership when comparing both Kebbi and Sokoto states to Zamfara; however, only the Sokoto changes were statistically significant for changes in antibiotics for respiratory infections, ORS use, and zinc use. TABLE 4 LIVES SAVED FROM INTEGRATED SBC AND MALARIA-ONLY SBC SCENARIOS (2020–2022) SCALED-UP LIMITED SCALED-UP INTEGRATED MALARIA-ONLY INTEGRATED MALARIA-ONLY Maternal Pregnancy 23 15 0 0 Childbirth 75 101 0 0 Contraception 117 103 0 0 Subtotal 215 219 0 0 Child Prenatal care 140 44 0 0 ITN coverage -1631 403 -1654 408 Childbirth 313 420 0 0 Breastfeeding -265 -19 0 0 Vaccines (DPT and measles) 3 25 0 0 ACT use 159 326 0 0 ORS and zinc for diarrhea 443 -762 521 -768 Antibiotics for respiratory illness 1590 -1211 562 -1226 Subtotal 752 -774 -571 -1586 12 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGR AMMING IN NIGERIA: FINAL REPORT significant outcome variables. The malaria-only SBC state saw an increase in lives lost, resulting in 1,586 lives lost. When converting the number of lives saved in the scaled-up scenarios to the additional DALYs averted in the integrated SBC states relative to the malaria-only SBC state, the integrated states yielded 47,605 more DALYs averted than the malaria-only state (Figure 5). The limited scaled-up scenario that considers just the statistically significant outcomes results in fewer DALYs averted, 31,814. Figure 5 also shows the number of malaria-specific DALYs averted in the integrated states compared to the malaria-only state, where the difference between the integrated and malaria-only states is negative because there were substantially more DALYs averted in the malaria-only state than in the two integrated states. The negative DALYs averted—or DALYs lost—was 66,474 for the scaled-up scenario and 64,626 for the limited scaled-up scenario. Expenditures The total expenditure for all SBC programming for the total evaluation period, by component, for the integrated and malaria-only program study states is shown in Figures 6 and 7. Over the total evaluation period, more than $18 million was spent in the study states. Looking at the individual study states, the integrated program in Kebbi had the largest expenditure over the total evaluation period at around $8.1 million. Sokoto’s integrated program had the next largest total expenditure at just over $7.2 million, and Zamfara, with its stand-alone malaria program, had the lowest expenditure overall with just under $3 million. Table 5 shows the total SBC expenditures and the expenditures per person in the intervention areas for each of the integrated and malaria-only study states. In FIGURE 5 ADDITIONAL DALYS AVERTED IN THE INTEGRATED SBC STATES COMPARED TO MALARIA-ONLY STATES (2020–2022) All DALYs averted Malaria DALYs averted (66,474) (64,626) Scaled-up scenario Limited scaled-up scenario (47,649) (31,814) FIGURE 6 EXPENDITURES FOR ALL SBC PROGRAMMING BY COMPONENT FOR INTEGRATED STUDY STATES Program Personnel Partner Program $5,466,824 Design Personnel $5,255,448 Partner $1,915,435 Design $2,687,801 FIGURE 7 EXPENDITURES FOR ALL SBC PROGRAMMING BY COMPONENT FOR MALARIA-ONLY STUDY STATES Program $638,987 Personnel $1,150,324 Partner $775,947 Design $419.190 BR E A K THROUGH R ESE A RCH | M AY 2023 13 total, the two study states implementing an integrated SBC program (Kebbi and Sokoto) expended over $15.3 million, of which, about 14% ($2.1 million) on average was allocated for malaria-focused SBC and the remaining 86% ($13.2 million) went to all other SBC programs. In the malaria-only study state (Zamfara), around $3 million was spent on all SBC with 71% ($2.1 million) of this being for malaria focused programming. The other 29% of expenditures were for TB and the Global Health Security Agenda (GHSA) and their associated personnel expenses as well as above-site costs. The expenditure per person for all SBC programs was $2.51 per person in integrated program areas, and $2.25 per person in the malaria￾only program areas in Zamfara. The malaria-specific expenditures per person living in the integrated areas was $0.35 and $1.59 per person living in the malaria-only intervention areas. To estimate the total cost of the SBC interventions, we calculated the additional service delivery costs using default values from the LiST costing module associated with the anticipated expanded coverage and then added them to the SBC expenditures in each state. The differences between the scaled-up scenarios and the baseline scenario were then used in the cost￾effectiveness analysis. Table 6 shows the effect of the additional service delivery costs when added to the malaria-only SBC costs and all other SBC costs for each of the three study states. Note that service delivery costs are negative in some instances due to a reduction in health services. More state-specific details on the SBC expenditures can be found in Appendix C. Cost-effectiveness The impact and cost findings combine to calculate the ICER as the additional costs for integrated SBC in Kebbi and Sokoto relative to the additional costs for malaria￾only programming in Zamfara divided by the additional impact for integrated SBC relative to the additional impact for malaria-only programming (see Equation above). Figure 8 displays the ICER results compared to the national and three state average GDP per capita benchmarks. The scaled-up scenarios resulted in an ICER of $278 per DALY averted, while the scaled-up scenario yielded $426 per DALY averted, both of which are “highly cost-effective” using either the national GDP per capita threshold of $2,252 or the three-state average GDP per capita threshold of $860. As described in the methods section, further analysis examined what proportion of the impact (additional DALYs averted in the integrated SBC areas) would need to be attributed to SBC, given other concurrent factors in each state that could be affecting the results during the study period, for the investments in integrated SBC to be considered cost-effective. Appendix D shows the cost TABLE 5 EXPENDITURE PER PERSON REACHED FOR INTEGRATED AND MALARIA-ONLY SBC STUDY STATES INTEGRATED MALARIA ONLY Total expenditure all SBC $15,325,508 $2,984,448 Total expenditure malaria SBC $2,120,488 $2,112,597 Total population 10,757553 5,066,557 Target population 6,123,722 1,327,297 Target population as % of total population 56.3% 25.0% Expenditure per person all SBC $2.51 $2.25 Expenditure per person malaria SBC $0.35 $1.59 TABLE 6 ADDITIONAL SERVICE DELIVERY EXPENDITURES FOR INTEGRATED & MALARIA ONLY STATES FOR THE SCALED-UP AND LIMITED SCALED-UP SCENARIOS INTEGRATED MALARIA ONLY SCALED-UP SCENARIO LIMITED SCALED-UP SCENARIO SCALED-UP SCENARIO LIMITED SCALED-UP SCENARIO Total SBC expenditures $15,325,508 $15,325,508 $2,984,448 $2,984,448 Added service delivery expenditures $1,192,883 $339,859 $425,864 -$873,565 Total expenditures (2022) $16,518,391 $15,665,366 $3,410,312 $2,110,883 Malaria SBC expenditures $2,120,488 $2,120,488 $2,112,597 $2,112,597 Added service delivery expenditures -$592,455 -$909,129 $425,864 $108,377 Total malaria expenditures (2022) $1,528,033 $1,211,359 $2,538,461 $2,220,973 14 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGR AMMING IN NIGERIA: FINAL REPORT per DALY averted when multiplying the DALYs averted to each percentage point possible, from 1 to 100%, that could be attributed to the SBC activities. Table 7 displays the comparison of those values to the four cost-effectiveness thresholds under the scaled-up and limited scaled-up scenarios. The most difficult threshold to reach is the three-state average highly cost-effective threshold at $860. Using this threshold in the scaled-up scenario, 33% of the impact seen in the BSS from baseline to endline would need to be due to SBC to be considered highly cost-effective. This increases to 50% using the limited scaled-up scenario. To be considered cost-effective, however, based on the three￾state average threshold, only 11% of impact needs to be attributed to SBC in the scaled-up scenario or 17% in the limited scaled-up scenario. Reaching cost-effectiveness is easier using the national thresholds, where only 13% of impact needs to be attributed to SBC to be considered highly cost-effective in the scaled-up scenario, and 19% is required in the limited scaled-up scenario. Finally, using the national threshold for cost-effectiveness, only 5% of impact must be attributed to SBC under the scaled-up scenario, which increases to 7% in the limited scaled-up scenario. TABLE 7 PROPORTION OF IMPACT ATTRIBUTED TO SBC NEEDED TO REACH COST-EFFECTIVENESS THRESHOLDS THREE STATE AVERAGE NATIONAL SCALED-UP SCENARIO LIMITED SCALED-UP SCENARIO SCALED-UP SCENARIO LIMITED SCALED-UP SCENARIO Cost-effective 11% 17% 5% 7% Highly cost-effective 33% 50% 13% 19% FIGURE 8 COST-EFFECTIVENESS RESULTS FOR ADDITIONAL COST PER DALY AVERTED FOR INTEGRATED SBC (2020–2022) $0 $1000 $2000 $3000 $4000 $5000 $6000 $7000 Scaled-up scenario $6,755—National; cost-effective Limited scaled-up scenario $2,580—Three state average; cost-effective $2,252—Three state average; highly cost-effective $ 860—National; highly cost-effective $ 278 $ 426 BR E A K THROUGH R ESE A RCH | M AY 2023 15 DISCUSSION To answer the three primary research questions, we explored SBC program expenditures, the relative cost￾effectiveness of integrated SBC, and how malaria-specific outcomes fared within integrated SBC. Question 1—What are the total program expenditures incurred during the total study time period from program initiation (April 2018) through October 2022? Among the study states, the total SBC expenditures in the integrated SBC states were higher when compared to the malaria-only SBC state, approximately US$15 million versus US$3 million. However, the expenditure per target population was similar across both approaches with integrated SBC at $2.53 per person and $2.36 per person in the malaria-only state. When SBC expenditures focused only on malaria were examined, $0.35 per person was spent in the integrated states vs. $1.67 in the malaria-only state. Question 2—Are the expenditures required for an integrated SBC program, compared to malaria-only SBC program, a cost￾effective investment? The ICER calculation indicates that the investments for integrated SBC, as compared to malaria-only SBC, are highly cost-effective based on both the national and the three-state average thresholds. The primary driver of this result is the increased use of antibiotics for respiratory infections in the integrated states versus the drop in use in the malaria-only state, resulting in a net difference between the integrated and malaria-only programs of 2,801 lives in the scaled-up scenarios. Similarly, the changes in use patterns for ORS and zinc for childhood diarrhea result in a net difference of 1,205 lives in favor of integrated programming (scaled-up applications). Improving these health behaviors are priority objectives addressed in Breakthrough ACTION’s SBC programming in the integrated states in both community activities (household visits, community dialogues, and community meetings) and mass media programming. In addition to Breakthrough ACTION programming, however, there are other differences between the integrated and malaria-only states that could be contributing to these behavioral outcomes. Specifically, a recent report noted that Zamfara experienced antibiotic stockouts during the study period that likely contributed to the drop in antibiotic use for respiratory infections.16 Additionally, the USAID-funded Integrated Health Program (IHP) is working in Kebbi and Sokoto on a complementary project to reduce maternal and child mortality by working at the health system and health facility level to improve the provision of essential primary care services.17 For these reasons, the results from Table 6 on the proportion of the impact that would need to be generated by SBC interventions to be considered cost-effective are informative. Based on the national benchmark, a modest 5% to 7% of the total impact seen from baseline to endline in the BSS needs to be attributed to the SBC interventions to be cost-effective and 13% to 19% to be highly cost-effective. The three￾state average threshold increases the proportion required to be considered highly cost-effective but still only 11% (scaled-up) to 17% (limited scaled-up) of the total impact is needed to be considered cost-effective. These results indicate that while it is unlikely that the different approaches in SBC is the primary driver in the different outcomes between the integrated and malaria￾only states, the SBC contributions from the intervention activities to improving antibiotic use for respiratory infections and ORS/zinc for childhood diarrhea are likely cost-effective. Question 3—How do malaria-specific outcomes perform within an integrated SBC program compared to a malaria-only program? In contrast to the overall findings, malaria-specific outcomes do not appear to be well served by integrated SBC. However, there are important caveats to consider when interpreting these results. The malaria-specific lives saved/lost calculated in the LiST model are primarily due to ITN ownership, where ownership increased in the malaria-only SBC state of Zamfara and decreased in the 16 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGR AMMING IN NIGERIA: FINAL REPORT integrated SBC states of Kebbi and Sokoto. These changes in ownership result in a net gain of 2,034 lives for the malaria-only SBC approach relative to the integrated SBC approach. This substantial impact likely has much more to do with the fact that an ITN distribution campaign was conducted in Zamfara during the study period and not in Kebbi and Sokoto, although an ITN campaign in Kebbi state coincided partly with the endline BSS.g SBC accompanying ITN distribution has been found to be effective in increasing the use of ITNs, but ownership itself is closely tied to distribution campaigns and steady declines are typically seen thereafter.18 For this reason, it would be very appealing to use the ITN use measures in Table 2 instead of the ITN ownership measures in Table 1. However, for reasons explained previously, the underlying modeling structure does not allow for that option. Still, it should be noted that if such a change were possible, it would not alter the overall conclusions that the malaria-only SBC state fared better in terms of malaria lives saved and DALYs averted due to ITN use since the improvements were much greater in the malaria-only areas. In contrast, if ITN distribution campaigns had fully occurred in the integrated SBC states during this time period, it is likely the ITN ownership results would be comparable across all three states like the other two malaria-related outcomes, IPTp use and ACT use. Although the changes from baseline to endline in the BSS were not statistically significant, the percentage of women receiving at least two doses of IPTp increased in all three states, with higher increases in the integrated states (14 and 7 percentage points) than in the malaria-only state (4 percentage points). ACT use dropped in Kebbi by 5 percentage points but increased in Sokoto and Zamfara by 9 and 8 percentage points, respectively, although these changes were also not statistically significant. Note that, since SBC expenditures are not allocated specifically to outcomes, it is not possible to calculate ICERs for these two specific outcomes, but instead the ICERs need to be calculated for SBC investments in malaria. Limitations As noted in the discussion above, the primary limitation of this study is that Zamfara does not appear to be a good comparator for Kebbi and Sokoto due to the stockouts of antibiotics in Zamfara, the IHP program presence in Kebbi and Sokoto, and the distribution g At the time of the endline BSS, ITN distribution had begun in Kebbi state. campaign of ITNs in Zamfara only during the study period. In addition, as with all modeling exercises, the results are based on numerous inputs and assumptions. Among other things, the expenditure data for the initial evaluation period and those for both the midline and endline periods differed slightly in terms of form and content. While each of the reports presented rational estimations of cost, calculating these estimations required assumptions on the allocation of above-site costs and the distribution of personnel costs. On the impact side, while the DALYs-averted metric used in this analysis does allow for comparison across different health areas, it does not fully capture the entire impact of the Breakthrough ACTION’s integrated SBC program, which also influences social norms and attitudes as well as building local SBC capacity that may not have yet translated into measurable health behavior change during the study time period. Where there were observable changes, few were noted to be statistically significant. The LiST modeling also does not allow for the changes in the number of people living in the intervention areas over time, but rather takes the midpoint population as the best proxy for the program. Another clear limitation previously discussed is the use of the ITN ownership variable for this particular study due to the focus on malaria-related outcomes and the different results seen in terms of ownership and use. Conclusions This study is one of the first to examine the cost￾effectiveness of integrated SBC. Overall, the findings indicate that even if only a relatively small proportion of the overall impact modeled is attributed to SBC, the SBC investments would be considered cost-effective. As such, these findings are promising, but not definitive given the challenges of using Zamfara as a comparator to the integrated states during this time period. Leveraging further Breakthrough ACTION work could potentially address some of the challenges in this analysis; future work should continue to explore the cost-effectiveness of integrated SBC programs. BR E A K THROUGH R ESE A RCH | M AY 2023 17 REFERENCES 1. WHO Commission on Macroeconomics and Health & World Health Organization. 2001. “Macroeconomics and health: investing in health for economic development: executive summary/report of the Commission on Macroeconomics and Health.” Geneva: World Health Organization. https://apps. who.int/iris/handle/10665/42463. 2. Compass. 2017. Integrated SBC. Accessed February 11, 2023 at: https://thecompassforsbc.org/trending-topics/integrat￾ed-sbc. 3. FHI 360. 2014. “Integration of global health and other devel￾opment sectors: A review of the evidence.” Washington, DC: FHI 360. https://www.fhi360.org/sites/default/files/me￾dia/documents/sap-integration-ofglobal-health-full.pdf. Accessed March 19, 2021. 4. Velu, S. et al. 2016. “Social and behavior change communi￾cation in integrated health programs: A scoping and rapid review.” HC3 Project and UNICEF. 5. Breakthrough RESEARCH. 2022. SBC cost data repository. Accessed March 28, 2023 at: https://breakthroughactionan￾dresearch.org/creating-sbc-cost-repository/. 6. Initial costing report 7. Avenir Health. 2023. “Cost-effectiveness analysis compar￾ing integrated and malaria-only social and behavior change programming in Nigeria: Midline analysis,” Breakthrough RESEARCH Technical Report. Washington, DC: Population Council. 8. Hutchinson, P. L. et al. (forthcoming). «Behavioral sentinel surveillance survey in Nigeria: Endline technical report,» Breakthrough RESEARCH Technical Report. Washington, DC: Population Council. 9. Walker N, Tam Y, Friberg IK. 2013. “Overview of the Lives Saved Tool (LiST),” BMC Public Health 13(Suppl 3): S1. doi: 10.1186/1471-2458-13-S3-S1 10. Eisele TP, Larsen D, Steketee RW. 2010. “Protective efficacy of interventions for preventing malaria mortality in children in Plasmodium falciparum endemic areas,” Int J Epidemiol. 39(Suppl 1): i88-101. doi: 10.1093/ije/dyq026. 11. Winfrey, W. Personal communication with William Winfrey, January 13, 2023. 12. Breakthrough RESEARCH. 2022. “Breakthrough RESEARCH— Social and Behavior Change Costing Community of Practice Series Brief #4: Are integrated social and behavior change interventions cost-effective? A methodological approach,” Programmatic Research Brief. Washington, D.C.: Population Council. 13. Vassall, A. et al. 2018. “Reference case for estimating the costs of global health services and interventions.” Accessed February 28, 2023 at: GHCC | Global Heath Cost Consortium (ghcosting.org) 14. World Bank. 2021. GDP per capita (current US$) – Nigeria. Accessed February 11, 2023 at: GDP per capita (current US$) - Nigeria | Data (worldbank.org). 15. United Nations. 2022. The sustainable development goals report. Accessed March 28, 2023 at: https://unstats. un.org/sdgs/report/2022/The-Sustainable-Develop￾ment-Goals-Report-2022.pdf. 16. USAID. Personal communication during quarterly meetings. February 13, 2023. 17. Integrated Health Program (IHP). 2022. “Feature: PHC exten￾sion services bring integrated healthcare closer to women, children, and communities in Sokoto.” Accessed February 14, 2023 at: https://medium.com/@NigeriaIHP_/bringing-inte￾grated-healthcare-closer-to-communities-to-reach-wom￾en-and-children-facing-preventable-89cbe650684c. 18. Kilian, A., N. Wijayanandana, and J. Ssekitoleeko. 2010. “Re￾view of delivery strategies for insecticide treated mosquito nets: are we ready for the next phase of malaria control efforts?” TropIKA. Net. 1(1). 19. Bollinger, L.A. et al. 2017. “Lives Saved Tool (LiST) costing: a module to examine costs and prioritize interventions,” BMC Public Health 17(Suppl 4): 782. doi: 10.1186/s12889-017- 4738-1. 18 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGR AMMING IN NIGERIA: FINAL REPORT APPENDIX A POPULATION DATA ON BREAKTHROUGH ACTION INTERVENTION COVERAGE To estimate the population for the LiST applications, the proportion of the total state population living in the intervention areas at midline was used. Table A1 details the 2020 population estimates for each Breakthrough ACTION intervention ward included in the analysis. Next, the percentage of the state population represented in the Breakthrough ACTION areas was applied to the 2022 population estimates in each state, as provided by the Spectrum models, which were previously validated under another program. Table A2 details the population numbers included in the LiST applications. TABLE A1 INTERVENTION STATES, LGAS, WARDS, AND 2020 POPULATION ESTIMATES STATE LGA WARD 2020 POPULATION Kebbi Arewa Bachaka 32,489 Kebbi Arewa Chibike 16,898 Kebbi Arewa Falde 9,479 Kebbi Arewa Feske/Jefeji 22,941 Kebbi Arewa Gorun Dikko 10,967 Kebbi Arewa Lema/Jan Tullu 37,961 Kebbi Arewa Sarka 22,513 Kebbi Arewa Bui 31,706 Kebbi Arewa Gumundai 27,811 Kebbi Arewa Kangiwa 28,270 Kebbi Arewa Yeldu 24,422 Kebbi Bagudo Bani Tsamiya 38,976 Kebbi Bagudo Lafagu Gante 24,836 Kebbi Bagudo Matsinkai Geza 18,995 Kebbi Bagudo Kende Kurgu 23,294 Kebbi Bagudo Zagga Kwasara 27,153 Kebbi Bagudo Kaoje Gwamba 65,501 Kebbi Bagudo Bagudo Tuga 37,220 Kebbi Bagudo Bahindi Khaliel 25,265 Kebbi Bagudo Illo Sabon Gari 28,801 Kebbi Bagudo Lolo Giris 40,749 Kebbi Bagudo Sharabi Kwanguwai 21,336 Kebbi Birnin Kebbi Gawasu Damana 16,228 Kebbi Birnin Kebbi Gulumbe 17,572 Kebbi Birnin Kebbi Kardi 34,877 Kebbi Birnin Kebbi Lagga Randali 15,716 Kebbi Birnin Kebbi Makera 19,743 Kebbi Birnin Kebbi Karyo 25,134 STATE LGA WARD 2020 POPULATION Kebbi Birnin Kebbi Asarara 18,345 Kebbi Birnin Kebbi Zauro 15,031 Kebbi Birnin Kebbi Ambursa 21,970 Kebbi Birnin Kebbi Dangaladima 37,023 Kebbi Birnin Kebbi Gwadangwaji 30,099 Kebbi Birnin Kebbi Kola Tarasa 19,141 Kebbi Birnin Kebbi Marafa 31,304 Kebbi Birnin Kebbi Nassarawa I 45,121 Kebbi Birnin Kebbi Nassarawa II 54,018 Kebbi Fakai Bangu 21,792 Kebbi Fakai Fakai Kukah 16,073 Kebbi Fakai Gulbin Kukah 15,475 Kebbi Fakai Kangi 15,780 Kebbi Fakai Marafa 19,460 Kebbi Fakai Penipeni 14,326 Kebbi Fakai Bajida 18,765 Kebbi Fakai Birnin Tudu 11,239 Kebbi Fakai Mahuta 21,091 Kebbi Fakai Maikende 20,843 Kebbi Gwandu Cheberu 28,048 Kebbi Gwandu Dalijan 27,298 Kebbi Gwandu Gwandu Marafa 21,048 Kebbi Gwandu Masama 20,949 Kebbi Gwandu Gulmare 23,413 Kebbi Gwandu Dodoru 20,044 Kebbi Gwandu Gwandu Dangaladima 20,660 Kebbi Gwandu Kambaza 17,440 Kebbi Gwandu Malisa 20,298 BR E A K THROUGH R ESE A RCH | M AY 2023 19 STATE LGA WARD 2020 POPULATION Kebbi Koko Besse Dada Alelu 12,133 Kebbi Koko Besse Zaria 16,094 Kebbi Koko Besse Takware 21,048 Kebbi Koko Besse Damba Bakoshi 17,536 Kebbi Koko Besse Jadadi 14,711 Kebbi Koko Besse Koko Firchin 22,769 Kebbi Koko Besse Hirini Madacci 13,284 Kebbi Koko Besse Illela Sabon Gari 22,868 Kebbi Koko Besse Besse 19,867 Kebbi Koko Besse Dutsi Mari 25,000 Kebbi Koko Besse Koko Magaji 19,980 Kebbi Koko Besse Lani Shiba 17,720 Kebbi Maiyama Gidiga 23,266 Kebbi Maiyama Giwatazo 21,406 Kebbi Maiyama Kawara 25,182 Kebbi Maiyama Karaye 25,533 Kebbi Maiyama Andarai 23,570 Kebbi Maiyama Liba 23,475 Kebbi Maiyama Gubunkure 23,450 Kebbi Maiyama Maiyama 18,363 Kebbi Maiyama Mungadi 26,628 Kebbi Maiyama Sambawa Mayalo 24,594 Kebbi Maiyama Sarandosa 24,941 Kebbi Wasagu/Danko Dan Umaru 35,739 Kebbi Wasagu/Danko Gwanfi Kele 19,680 Kebbi Wasagu/Danko Kyanbu Kandu 27,705 Kebbi Wasagu/Danko Wasagu 36,719 Kebbi Wasagu/Danko Yalmo Shindy Wari 16,001 Kebbi Wasagu/Danko Ayu 25,305 Kebbi Wasagu/Danko Bena 45,410 Kebbi Wasagu/Danko Danko Maga 25,329 Kebbi Wasagu/Danko Kanya 52,483 Kebbi Wasagu/Danko Ribah Machika 44,885 Kebbi Wasagu/Danko Waje 67,229 Kebbi Shanga Dugu Tsoho 21,016 Kebbi Shanga Kawara 20,863 Kebbi Shanga Rafin Kirya 13,419 Kebbi Shanga Atuwo 15,905 Kebbi Shanga Yarbesse 11,746 Kebbi Shanga Gebbe 22,633 Kebbi Shanga Sakace 27,744 Kebbi Shanga Sawashi 18,508 Kebbi Shanga Shanga 16,484 Kebbi Shanga Takware 15,086 STATE LGA WARD 2020 POPULATION Kebbi Suru Aljannare 31,082 Kebbi Suru Bandan 22,470 Kebbi Suru Ginga 13,712 Kebbi Suru Dandane 16,120 Kebbi Suru Barbarejo 19,191 Kebbi Suru Kwaifa 14,533 Kebbi Suru Bakuwai 24,979 Kebbi Suru Dakin Gari 29,841 Kebbi Suru Dandiya Shema 8,536 Kebbi Suru Giro 9,723 Kebbi Suru Suru 32,784 Kebbi Zuru Daben Seme 18,080 Kebbi Zuru Zodi 15,215 Kebbi Zuru Manga Ushe 32,680 Kebbi Zuru Rikoto 33,017 Kebbi Zuru Senchi 21,294 Kebbi Zuru Bedi 26,085 Kebbi Zuru Dabai 22,187 Kebbi Zuru Isgogo Dago 22,758 Kebbi Zuru Rafin Zuru 31,352 Kebbi Zuru Tadurga 23,587 KEBBI TOTAL 2,950,235 Sokoto Binji Jamali 13,074 Sokoto Binji Tudun Kose 11,352 Sokoto Binji Gawazai 14,233 Sokoto Binji Bunkari 16,757 Sokoto Binji Soro Gabas 16,354 Sokoto Binji Binji 33,823 Sokoto Binji Inname 10,233 Sokoto Binji Maikulki 17,910 Sokoto Binji Samama 12,175 Sokoto Binji Soron Yamma 15,479 Sokoto Bodinga Bagarawa 14,215 Sokoto Bodinga Bangi/Dabaga 33,334 Sokoto Bodinga Dingyadi 28,255 Sokoto Bodinga Sifawa Lukuyawa 17,118 Sokoto Bodinga Tulluwa 20,031 Sokoto Bodinga Badau 10,550 Sokoto Bodinga Bodinga 33,539 Sokoto Bodinga Danchadi 38,970 Sokoto Bodinga Kauramiyo￾Mazan Gari 32,065 Sokoto Bodinga Kwacciyo Lalle 14,050 Sokoto Bodinga Takatuku 28,590 20 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT STATE LGA WARD 2020 POPULATION Sokoto Dange/Shuni Bodai 32,365 Sokoto Dange/Shuni Dange 36,205 Sokoto Dange/Shuni Fajaladu 18,915 Sokoto Dange/Shuni Gere-Gajere 18,915 Sokoto Dange/Shuni Rikina 36,345 Sokoto Dange/Shuni Rudu/ Amanawa 36,205 Sokoto Dange/Shuni Ruggar Gidado 17,810 Sokoto Dange/Shuni Shuni 31,085 Sokoto Dange/Shuni Tsafanade S 16,670 Sokoto Dange/Shuni Tuntube Tsefe 65,555 Sokoto Dange/Shuni Wababe 26,820 Sokoto Gada Kadadi 27,433 Sokoto Gada Kiri 33,194 Sokoto Gada Kwarma 27,414 Sokoto Gada Gilbadi 34,965 Sokoto Gada Tsitse 22,642 Sokoto Gada Dukamaje 17,057 Sokoto Gada Gada 35,546 Sokoto Gada Kadassaka 24,260 Sokoto Gada Kaddi 24,839 Sokoto Gada Kaffe 9,794 Sokoto Gada Kyadawa/Holai 39,326 Sokoto Gwadabawa Asara 40,300 Sokoto Gwadabawa Atakwanyo 22,975 Sokoto Gwadabawa Chimmola 35,395 Sokoto Gwadabawa Gidan Kaya 24,945 Sokoto Gwadabawa Gigane 53,400 Sokoto Gwadabawa Gwadabawa 42,460 Sokoto Gwadabawa Huchi 16,995 Sokoto Gwadabawa Mamman Suka 32,470 Sokoto Gwadabawa Mammande 59,065 Sokoto Gwadabawa Salame 63,280 Sokoto Gwadabawa Tambagarka 11,211 Sokoto Illela Araba 20,935 Sokoto Illela Darna Sabon Gari 21,260 Sokoto Illela Garu 12,798 Sokoto Illela Rungumawar Gatti 17,674 Sokoto Illela Tozai 15,855 Sokoto Illela Darna Tsolawo 45,191 Sokoto Illela Gidan Hamma 73,535 Sokoto Illela Gidan Katta 26,060 Sokoto Illela Damba 72,623 Sokoto Illela Kalmalo 27,500 Sokoto Illela Illela 35,960 STATE LGA WARD 2020 POPULATION Sokoto Kebbe Fakku 59,065 Sokoto Kebbe Girkau 32,325 Sokoto Kebbe Kebbe east 22,410 Sokoto Kebbe Kebbe west 18,790 Sokoto Kebbe Kuchi 31,635 Sokoto Kebbe Margai east 52,790 Sokoto Kebbe Margai west 19,892 Sokoto Kebbe Nasagudu 30,765 Sokoto Kebbe Sangi 25,150 Sokoto Kebbe Ungushi 19,690 Sokoto Kware Basansan 16,854 Sokoto Kware Durbawa 15,418 Sokoto Kware Gandu Modibbo 16,472 Sokoto Kware More Gidan Rugga 17,799 Sokoto Kware Sabon Birni 17,083 Sokoto Kware Bankanu 10,060 Sokoto Kware Hamma Ali 41,095 Sokoto Kware Kabanga 14,095 Sokoto Kware Kware 27,875 Sokoto Kware Tsaki-Walaka’e 26,363 Sokoto Kware Tunga￾Mallamawa 22,834 Sokoto Shagari Dandin/Mahe 32,040 Sokoto Shagari Horo 17,227 Sokoto Shagari Kambama 20,144 Sokoto Shagari Jaredi 19,033 Sokoto Shagari Lambara 26,279 Sokoto Shagari Kajiji 26,685 Sokoto Shagari Mandera 11,576 Sokoto Shagari Sanyin Lawal 29,510 Sokoto Shagari Shagari 21,357 Sokoto Shagari Gangam 18,307 Sokoto Wamakko Arkilla/Gwiwa 43,415 Sokoto Wamakko Bado/Kasarawa 15,965 Sokoto Wamakko Dundaye/ Gumburawa 24,607 Sokoto Wamakko Gidan Bubu 22,770 Sokoto Wamakko Gidan Hamidu 19,450 Sokoto Wamakko Gumbi/Wajake 16,755 Sokoto Wamakko Gwamatse 32,345 Sokoto Wamakko Kalambaina/ Girafshi 27,400 Sokoto Wamakko Kammata 21,431 Sokoto Wamakko Kaurar Gedawa 25,870 Sokoto Wamakko Wamakko 25,870 BR E A K THROUGH R ESE A RCH | M AY 2023 21 STATE LGA WARD 2020 POPULATION Sokoto Wurno Achida 20,257 Sokoto Wurno Kwasare Sissawa 23,198 Sokoto Wurno Tunga 15,983 Sokoto Wurno Dinawa 24,428 Sokoto Wurno Magarya 25,387 Sokoto Wurno Chacho/Marnona 25,387 Sokoto Wurno Dimbiso 24,428 Sokoto Wurno Alkamu 20,390 Sokoto Wurno Kwargaba 14,911 Sokoto Wurno Lahodu 27,790 Sokoto Wurno Marafa 13,803 SOKOTO TOTAL 3,109,812 Zamfara Bakura Dankado 34,547 Zamfara Bakura Yargida 17,818 Zamfara Bakura Yarkofoji 39,358 Zamfara Bakura Rini 27,500 Zamfara Bakura Danmannau 32,050 Zamfara Bukkuyum Bukkuyum 30,074 Zamfara Bukkuyum Kyaram 60,802 Zamfara Bukkuyum Nasarawa 37,125 Zamfara Bukkuyum Yashi 18,704 Zamfara Bukkuyum Zarumai 28,292 Zamfara Bukkuyum Zauma 18,243 Zamfara Gunmi Magaji 54,839 Zamfara Gunmi Gayari 29,614 Zamfara Gunmi Gyalange 27,830 Zamfara Gunmi Birnin Magaji 23,220 Zamfara Gunmi Birnin Tudu 53,952 Zamfara Gunmi Falale 16,071 Zamfara Gusau Galadima 114,078 Zamfara Gusau Madawaki 48,781 Zamfara Gusau Mayana 78,542 Zamfara Gusau Sabon Gari 45,049 Zamfara Gusau Tudun Wada 150,936 Zamfara Gusau Wanke 48,385 Zamfara Maradun Dosara Birnin Kaya 40,209 Zamfara Maradun Faru Magami 70,316 Zamfara Maradun Goran Namaye 23,998 Zamfara Maradun Janbako 26,620 Zamfara Maradun Maradun North 31,544 Zamfara Maradun Maradun South 36,093 ZAMFARA TOTAL 1,264,590 22 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT TABLE A2 AGE-SPECIFIC POPULATION ESTIMATES FOR THE LIST APPLICATIONS KEBBI SOKOTO ZAMFARA MALE FEMALE MALE FEMALE MALE FEMALE 0 to 4 281,004 285,746 291,264 292,463 122,559 124,188 5 to 9 231,101 237,797 258,419 262,528 118,316 120,788 10 to 14 197,353 200,058 229,952 230,709 91,058 91,445 15 to 19 177,484 172,783 173,936 166,700 74,482 71,705 20 to 24 133,795 125,339 132,307 120,015 56,079 51,898 25 to 29 96,673 91,016 95,838 86,379 40,358 37,754 30 to 34 75,378 89,296 73,279 85,985 31,059 37,782 35 to 39 60,616 92,167 57,323 90,424 24,590 39,561 40 to 44 52,975 71,159 50,497 71,512 21,987 30,393 45 to 49 47,282 49,704 45,887 51,804 20,221 21,009 50 to 54 38,219 36,887 38,075 38,543 16,292 15,421 55 to 59 29,593 25,942 30,572 26,863 12,491 10,648 60 to 64 22,412 18,405 23,671 19,379 9,534 7,572 65 to 69 15,865 12,450 17,106 13,415 6,821 5,176 70 to 74 10,513 8,261 11,282 8,768 4,456 3,333 75 to 79 6,094 5,021 6,410 5,159 2,519 1,933 80+ 4,938 4,383 5,083 4,469 2,117 1,751 TOTAL 1,481,293 1,526,413 1,540,900 1,575,116 654,939 672,359 BR E A K THROUGH R ESE A RCH | M AY 2023 23 APPENDIX B STATE-SPECIFIC RESULTS While the primary objective of the cost-effectiveness analysis is to compare the results from the integrated states (Kebbi and Sokoto) to the malaria-only state (Zamfara), there are some additional interesting patterns to examine when looking at the individual state results. Using the scaled-up scenarios, the number of lives saved in each state is shown in Table B1. As discussed in the overall report, the are two primary drivers of the cost￾effectiveness results: 1) the loss of lives in the integrated states due to ITN coverage compared to gains in Zamfara and 2) the loss of lives in Zamfara due to reduced use of antibiotics for respiratory infections compared to gains in the integrated states. These results do not change when looking state-specific results, however, there are some interesting differences between Kebbi and Sokoto to consider. First, Kebbi shows a net 269 maternal lives saved in mothers, primarily due to improvements in modern contraceptive use and more facility-based births whereas Sokoto has a net loss of 54 lives. Zamfara results were more in line with Kebbi, seeing an overall gain of 219 lives due to improvements in contraception and facility￾based birth. Facility-based birth results are also reflected among the child lives saved, showing 317 lives saved for Kebbi and 420 for Zamfara, but a loss of 4 lives in Sokoto. In contrast, Sokoto performed better for other outcomes, with many lives saved due to ACT use and ORS and zinc in Sokoto (429 and 536, respectively) but losses in Kebbi (-270, and -93, respectively). It is unclear to what extent these differences in commodities use are being driven by differential SBC successes in each state or rather external supply chain issues or program disruptions. On the cost side, Table B2 shows a detailed breakdown of the SBC expenditures per person living in the Breakthrough ACTION community SBC intervention areas for each of the study states. Among the study states, the highest expenditures occurred in the two integrated programs, Kebbi (US$8.1 million) and Sokoto (US$7.2 million), while the stand-alone malaria SBC program in Zamfara spent approximately US$3 million. When adding the additional service delivery costs associated with improvements in health behaviors, the total program costs were $9.2m for Kebbi, $7.3m for Sokoto, and $3.4m for Zamfara. When combining the impact and costs to examine cost￾effectiveness, it is possible to make two comparisons: 1) Kebbi vs. Zamfara and 2) Sokoto vs. Zamfara. Box B1 TABLE B1 NUMBER OF LIVES SAVED USING THE SCALED-UP SCENARIOS FROM BASELINE (2019) TO ENDLINE (2022) INTEGRATED MALARIA￾ONLY KEBBI SOKOTO ZAMFARA Maternal Pregnancy 14 9 15 Childbirth 75 0 101 Contraception 180 -63 103 TOTAL maternal lives saved 269 -54 219 Child Prenatal care 82 58 44 ITN coverage -1006 -625 403 Childbirth 317 -4 420 Breastfeeding -78 -187 -19 Vaccines (DPT and measles) 22 -19 25 ACT use -270 429 326 ORS and zinc for diarrhea -93 536 -726 Antibiotics for respiratory illness 1014 576 -1211 TOTAL child lives saved -12 764 -774 TOTAL maternal & child lives saved 257 710 -555 TABLE B2 ALL SBC EXPENDITURE PER PERSON REACHED IN ALL THREE STUDY STATES KEBBI $ SOKOTO $ ZAMFARA $ SBC expenditures 8,117,297 7,208,211 2,984,448 Additional service delivery costs 1,103,633 89.251 425,864 TOTAL 9,220,929 7,297,462 3,410,312 24 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT shows the two separate cost per DALY averted estimates comparing the integrated state to the malaria-only state. Both results are far below the thresholds to be deemed highly cost-effective, using either the national threshold of $2,066 or the two-state average thresholds of $767 for Kebbi/Zamfara and $980 for Sokoto/Zamfara. For the Kebbi vs. Zamfara comparison, the cost per DALY averted is $229.82, while the cost per DALY averted is even lower in the Sokoto vs. Zamfara comparison at $96.63. Interestingly, both estimates are less than what is found when we combined Kebbi and Sokoto, which generates a cost per DALY averted of $278. Due to the differences￾in-differences approach, some of the gains in one state are cancelled out by the losses in another and thus the overall results are more favorable using a single integrated state as a comparison. Still, the primary conclusions, as well as study limitations, remain the same whether examining the relative cost￾effectiveness by state or analyzing them together as the “integrated program”, as originally proposed. Future work in this area can continue to explore these dynamics in cost-effectiveness studies. BOX B1 COST PER DALY AVERTED FOR INTEGRATED VS. MALARIA-ONLY STATES KEBBI VS. ZAMFARA Maternal DALYs averted (Kebbi - Zamfara) 1,419 Child DALYs averted (Kebbi - Zamfara) 23,864 Additional impact for Kebbi 25,283 Additional costs for Kebbi $ 5,810,618.00 ICER = (Additional cost/Additional impact) $ 229.82 SOKOTO VS. ZAMFARA Maternal DALYs averted (Sokoto - Zamfara) (7,881) Child DALYs averted (Sokoto - Zamfara) 48,110 Additional impact for Sokoto 40,229 Additional costs for Sokoto $ 3,887,150.00 ICER = (Additional cost/Additional impact) $ 96.63 BR E A K THROUGH R ESE A RCH | M AY 2023 25 APPENDIX C FURTHER SBC EXPENDITURES DETAILS SBC program implementation and personnel expenditure data were provided by Breakthrough ACTION for the three costing phases of the evaluation timeline as outlined in Figure 1: initial costing phase (April 2018 to December 2019); midline costing phase (January 2020 to December 2021); and the endline costing phase (January 2022 to October 2022). To calculate total SBC expenditures for each phase, we extracted all programmatic expenditures for each of the study states except for mass media. Program expenditures are comprised of all funding expended on the implementation of each program area including direct costs such as those for personnel, training, consulting services, supplies, travel, and indirect costs such as equipment and furniture, vehicles, maintenance, rent, utilities, management/oversight, and other overheads. Given the that the radio programming was conducted statewide while the cost-effectiveness analysis is focused on the Breakthrough RESEARCH intervention wards, we assessed a proportion of mass media expenditures based on the population living in the intervention areas for the study relative to the population of the entire state (Kebbi: 60.1%; Sokoto: 53.1%; Zamfara: 25%). For example, in Kebbi, 60.1% of all expenditure on mass media programs were allocated based on the Breakthrough ACTION intervention wards for community SBC. The same principle was used for mass media expenditure in Sokoto and Zamfara. These state proportions of mass media expenditure were then added to all other SBC expenditures by state, resulting in total program expenditures by state. A similar approach was used to determine malaria￾only expenditures for each state. The SBC program expenditures focused on malaria for each of the study states were extracted from the expenditure data. Expenditures for the mass media component were treated in the same way as mass media for total SBC expenditures, i.e., using the proportion of the intervention area population to the population of the overall state to determine the share of mass media costs to apply in each state. These were then added to all other malaria SBC costs to arrive at malaria-only program costs by state. Personnel expenditures for site-level and above-site (Abuja and organizational headquarters) and partners for each study state were extracted from the data for each costing period. For the initial reporting period (April 2018–December 2019) personnel expenditures are documented in the initial costing report.6 Personnel expenditures for the midline and endline costing periods were estimated based on the proportions of total personnel expenditures borne by each state as provided in the Breakthrough ACTION expenditure data. The total site level and above-site level personnel and partner expenditures, for each state were added to the total SBC program implementation expenditures, to arrive at total program and personnel expenditures by state for each costing period. To determine the proportion of personnel and partner expenditures for the malaria￾only programming in each of the study states, it was first determined what proportion of total program implementation expenditures was made up of malaria￾only programming. This proportion was then used to allocate a percentage of total personnel and partner expenditures per state to malaria-only programming for each period. These expenditures were then added to the malaria-only program implementation expenditures to provide an estimate of total malaria-only program and personnel expenditures by state. To add program design expenditures to the total expenditures for each study state, we extracted what was spent on program design in each of the years of the project. Program design expenditures were largely frontloaded, constituting a significant investment in the initial costing period, but then declining in the subsequent midline and endline periods. Expenditures made on program design have an impact over the lifetime of the project (anticipated end in 2025). Therefore, design expenditures made during the initial costing period were spread from 2018 through 2025. Similarly, design expenditures made over the midline and endline costing periods were spread from 2020 and 2022, respectively, to the end of the project. The investment 26 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT made in program design was then added to the total SBC program and personnel expenditures already calculated for each of the study states. To estimate the design cost for the malaria-only program, the same approach used to derive personnel and partner costs was employed. Using the proportion of total program costs attributable to malaria-only programming, we allocated a percentage of total design costs per state to malaria-only programming for each period. In addition to the SBC expenditures, there are service delivery costs associated with changes in the behavioral health outcomes. For example, increases in modern contraceptive prevalence will result in additional commodity costs. LiST contains a costing module with default values that estimate the total intervention costs associated with health behavior outcomes modeled in the scenarios.19 For each scenario, the total intervention costs relevant to the included health behavior outcomes were examined and the difference between the scaled-up scenarios and the limited scaled-up scenario were used to calculate the changes in service delivery costs associated with the changes in the outcome variables. These changes were added to the SBC expenditures to generate the total costs for each application. All expenditures on COVID-19, such as those for advocacy, community SBC, capacity strengthening, mass media, and operational costs, among others, were excluded from our analysis. Finally, all costs were adjusted to 2022 US$ using the GDP deflator as published by the Federal Reserve Bank of St. Louis, on the FRED Economic Data website (https://fred. stlouisfed.org/series/GDPDEF#). Expenditure Analysis Results The total cost for all SBC programming for the three evaluation periods, by component, in the three study states is shown in Figure C1. Of the states included in the study, Kebbi had the highest expenditures over the three evaluation periods and, with only a stand-alone malaria program, Zamfara had the lowest expenditures overall. Program implementation expenditures as a proportion of total expenditures in each state was fairly consistent, making up 34% and 37% in Kebbi and Sokoto respectively, and around 21% in Zamfara. Interestingly, personnel expenditures were higher in Zamfara (39%) compared with either Kebbi (33%) or Sokoto (36%). Similarly, partner expenditures, which include both personnel and programmatic expenses, in Zamfara were 26% of total expenditures, which was significantly higher than in Kebbi, where partner expenditures were 14% of total costs or in Sokoto, where they were only 11%. Design expenditures were highest in Kebbi, comprising 20% of total expenditures, declining to only 15% of total costs in Sokoto and 14% in Zamfara. Figure C2 displays malaria-only expenditures for each of the three study states disaggregated by cost component. Zamfara, with its stand-alone malaria program, had the highest expenditures for malaria-only SBC over the three evaluation periods; 1.5 times more than Kebbi and nearly three times more than Sokoto. Expenditures on design for all three states were very similar, ranging FIGURE C1 EXPENDITURES FOR ALL SBC PROGRAMMING BY STATE AND COMPONENT Program Personnel Partner $0 $1,000,000 $2,000,000 $3,000,000 $4,000,000 $5,000,000 $6,000,000 $7,000,000 $8,000,000 $9,000,000 Kebbi Sokoto Zamfara Design BR E A K THROUGH R ESE A RCH | M AY 2023 27 from 16% in Sokoto to 18% in Kebbi. Of the total malaria SBC expenditures, the allocation of the expenditures varied by state. In Sokoto only 7% of total malaria SBC expenditures were expended by partners, which increased to 13% in Kebbi and to over 30% in Zamfara. All other personnel costs consumed about a third of total expenditures in each of the three study states. Figure C3 examines the proportion of overall expenditures for malaria-only SBC programming versus all other SBC programming in the two states with integrated programs (Kebbi and Sokoto), and the stand-alone malaria program in Zamfara. The integrated program in Kebbi expended 83% of all resources on all other SBC programs with 17% going to malaria SBC. Sokoto was more skewed with just over 90% of all expenditure going towards all other SBC programs and only 10% going to malaria programs. In Zamfara there were only a few program implementation expenditures that were not malaria-related, such as small interventions for TB and GHSA. The bulk of all other non-malaria expenditures were program design and above-site personnel costs. FIGURE C2 EXPENDITURES FOR MALARIA-ONLY SBC PROGRAMMING BY STATE AND COMPONENT Program Personnel Partner $0 $500,000 $100000 $1,500,000 $2,000,000 $2,500,000 Kebbi Sokoto Zamfara Design FIGURE C3 PROPORTION OF ALL EXPENDITURE FOR MALARIA-ONLY AND ALL OTHER SBC PROGRAMMING BY STATE Malaria only SBC All other SBC programs $0 $1,000,000 $2,000,000 $3,000,000 $4,000,000 $5,000,000 $6,000,000 $7,000,000 $8,000,000 $9,000,000 Kebbi Sokoto Zamfara 28 COST-EFFECTIVENESS ANALYSIS COMPARING SBC PROGRAMMING IN NIGERIA: FINAL REPORT APPENDIX D EXAMINING ICER WITH VARYING LEVELS OF IMPACT ATTRIBUTION PERCENT IMPACT ATTRIBUTED TO SBC COST PER DALY AVERTED SCALED-UP LIMITED 1% $ 27,802 $ 42,605 2% $ 13,901 $ 21,303 3% $ 9,267 $ 14,202 4% $ 6,951 $ 10,651 5% $ 5,560 $ 8,521 6% $ 4,634 $ 7,101 7% $ 3,972 $ 6,086 8% $ 3,475 $ 5,326 9% $ 3,089 $ 4,734 10% $ 2,780 $ 4,261 11% $ 2,527 $ 3,873 12% $ 2,317 $ 3,550 13% $ 2,139 $ 3,277 14% $ 1,986 $ 3,043 15% $ 1,853 $ 2,840 16% $ 1,738 $ 2,663 17% $ 1,635 $ 2,506 18% $ 1,545 $ 2,367 19% $ 1,463 $ 2,242 20% $ 1,390 $ 2,130 21% $ 1,324 $ 2,029 22% $ 1,264 $ 1,937 23% $ 1,209 $ 1,852 24% $ 1,158 $ 1,775 25% $ 1,112 $ 1,704 26% $ 1,069 $ 1,639 27% $ 1,030 $ 1,578 28% $ 993 $ 1,522 29% $ 959 $ 1,469 30% $ 927 $ 1,420 31% $ 897 $ 1,374 32% $ 869 $ 1,331 33% $ 842 $ 1,291 34% $ 818 $ 1,253 35% $ 794 $ 1,217 36% $ 772 $ 1,183 37% $ 751 $ 1,151 38% $ 732 $ 1,121 PERCENT IMPACT ATTRIBUTED TO SBC COST PER DALY AVERTED SCALED-UP LIMITED 39% $ 713 $ 1,092 40% $ 695 $ 1,065 41% $ 678 $ 1,039 42% $ 662 $ 1,014 43% $ 647 $ 991 44% $ 632 $ 968 45% $ 618 $ 947 46% $ 604 $ 926 47% $ 592 $ 906 48% $ 579 $ 888 49% $ 567 $ 869 50% $ 556 $ 852 51% $ 545 $ 835 52% $ 535 $ 819 53% $ 525 $ 804 54% $ 515 $ 789 55% $ 505 $ 775 56% $ 496 $ 761 57% $ 488 $ 747 58% $ 479 $ 735 59% $ 471 $ 722 60% $ 463 $ 710 61% $ 456 $ 698 62% $ 448 $ 687 63% $ 441 $ 676 64% $ 434 $ 666 65% $ 428 $ 655 66% $ 421 $ 646 67% $ 415 $ 636 68% $ 409 $ 627 69% $ 403 $ 617 70% $ 397 $ 609 71% $ 392 $ 600 72% $ 386 $ 592 73% $ 381 $ 584 74% $ 376 $ 576 75% $ 371 $ 568 76% $ 366 $ 561 PERCENT IMPACT ATTRIBUTED TO SBC COST PER DALY AVERTED SCALED-UP LIMITED 77% $ 361 $ 553 78% $ 356 $ 546 79% $ 352 $ 539 80% $ 348 $ 533 81% $ 343 $ 526 82% $ 339 $ 520 83% $ 335 $ 513 84% $ 331 $ 507 85% $ 327 $ 501 86% $ 323 $ 495 87% $ 320 $ 490 88% $ 316 $ 484 89% $ 312 $ 479 90% $ 309 $ 473 91% $ 306 $ 468 92% $ 302 $ 463 93% $ 299 $ 458 94% $ 296 $ 453 95% $ 293 $ 448 96% $ 290 $ 444 97% $ 287 $ 439 98% $ 284 $ 435 99% $ 281 $ 430 100% $ 278 $ 426 BR E A K THROUGH R ESE A RCH | M AY 2023 29 Population Council 4301 Connecticut Ave., NW | Suite 280 Washington, DC 20008 +1 202 237 9400 breakthroughactionandresearch.org