AUGUST 2020 EVALUATING SAVING LIVES AT BIRTH Evaluation Report: Rounds One to Eight (2011-2020) ACKNOWLEDGEMENTS: Evaluating SL@B, a joint program of the Duke Global Health Innovation Center and the Duke Global Health Institute Evidence Lab, is funded by the generous support of the Saving Lives at Birth partners: the United States Agency for International Development (USAID), the Government of Norway, the Bill & Melinda Gates Foundation, Grand Challenges Canada, the UK Government, and the Korea International Cooperation Agency (KOICA) via Cooperative Agreement No. 7200AA18C00019. DISCLAIMER: This product is made possible through the generous support of the Saving Lives at Birth partners: the United States Agency for International Development (USAID), Grand Challenges Canada (GCC) (with funding from the Government of Canada), Norwegian Agency for Development Cooperation (NORAD), the Bill & Melinda Gates Foundation (BMGF), the UK Department for International Development (DFID), and the Korea International Cooperation Agency (KOICA). The contents of this report are the sole responsibility of the Duke Global Health Innovation Center and the Duke Global Health Institute Evidence Lab, and do not necessarily reflect the views of Saving Lives at Birth partners. 2 5 11 14 27 37 46 48 51 55 56 57 62 9 10 15 16 18 19 22 29 34 35 38 38 41 Executive Summary 1: Introduction 2: The SL@B Program in the Global Landscape: Sourcing and Funding MNH Innovations 3: SL@B’s Role in Building Innovator Capacity and Accelerating Innovation 4: SL@B-Facilitated Partnerships for Increased Innovation Impact 5: Quantifying SL@B’s Impact 6: Conclusion Bibliography Appendix I: SL@B’s Theory of Change Appendix II: SL@B’s Impact Area Framework Appendix III: SL@B’s Non-financial Support Appendix IV: Impact Modeling Checklist TABLE OF CONTENTS 17 18 21 28 28 29 30 41 Figure 1: Recommendations for SL@B from the Program Evaluation Figure 2: Causes of Maternal and Neonatal Mortality Globally Figure 3: Number of Innovations, by Innovation Type and Round Figure 4: Funding Portfolios by Target Population Figure 5: IDIA and SL@B Innovation Growth Stage Categories Figure 6: Number of SL@B Innovations, by Stage of Growth and Round Figure 7: Number of SL@B Innovations and Amount of SL@B Investment, by Awardee Location Figure 8: Total Number of Months Spent in Each Stage Across Varying Levels of SL@B Support Figure 9: Non-Financial SL@B Support, by Percent of Innovators Reporting it as Highly Helpful Figure 10: Non-Financial SL@B Support, by Percent of Innovators Reporting it as Insufficient Figure 11: Number of SL@B Innovations with Public Sector Partnerships Figure 12: Number of SL@B Innovations with Private Sector Partnerships Figure 13: SL@B Partnerships Support, by Percent of Innovators Reporting it as Sufficient or Insufficient Table 1: SL@B Portfolio by Causes of Maternal and Newborn Mortality Table 2: Number of Innovations, by Type and Stage Table 3: Top 25 Countries with Highest Burden of Maternal Mortality Ratios, Newborn Mortality Rates, Maternal Deaths, and Newborn Deaths, Compared to Number of SL@B Innovations Targeting those Countries Table 4: Innovator Challenges, by Type of Organization and Type of Award Table 5: Number of Months in Each Growth Stage Table 6: SL@B Funding Level, by Grant Type Table 7: Stage of Innovators Reporting SL@B Accelerated their Achievement, by Activity Table 8: Illustrative Key Partnerships Formed by SL@B Innovators LIST OF FIGURES LIST OF TABLES 3 ANC BLA BMGF CE CHX DCID DevX DFID DGHI ESL@B EWEC GCC GFF HIC IDIA IRB JSI KOICA LMIC MGH MNH MNCH MOH NDA NGO NORAD mHealth ODA+ PIO PMNCH RMNCH RMNCAH SL@B TOC TTS UNAIDS UNICEF USAID WAHO WHO Antenatal care Biologic License Application Bill and Melinda Gates Foundation Conformitè Europëenne Chlorhexidine Duke Center for International Development Development Exchange U.K. Department for International Development Duke Global Health Institute Evaluating Saving Lives at Birth Every Woman Every Child Grand Challenges Canada Global Financing Facility High-income countries International Development Innovation Alliance Institutional Review Board JSI Research & Training Institute, Inc. Korea International Cooperation Agency Low- and middle-income countries Massachusetts General Hospital Maternal and Newborn Health Maternal, Newborn and Child health Ministry of Health New Drug Application Non-governmental organization The Norwegian Agency for Development Cooperation Mobile health Official development assistance, plus grants Public International Organization Partnership for Maternal, Newborn, and Child Health Reproductive, Maternal, Newborn and Child Health Reproductive, Maternal, Neonatal, Child and Adolescent Health Saving Lives at Birth Theory of Change Transition to Scale The Joint United Nations Programme on HIV and AIDS The United Nations Children’s Fund United States Agency for International Development West African Health Organization World Health Organization ACRONYMS 4 Saving Lives at Birth (SL@B): A Grand Challenge for Development brings together a global community working on innovation to combat preventable maternal and newborn deaths and stillbirths on a global scale. Launched in 2011, SL@B is a partnership of key global health stakeholders including the United States Agency for International Development (USAID), the Norwegian Agency for Development Cooperation (NORAD), the Bill and Melinda Gates Foundation (BMGF), Grand Challenges Canada (GCC), the U.K. Department for International Development (DFID), and the Korea International Cooperation Agency (KOICA). The SL@B program has funded a total of 147 innovations,* of which 116 are unique, and 92 organizations addressing critical issues in maternal and newborn health (MNH) in low-resource settings. The SL@B portfolio spans different types of innovations: products, including devices and diagnostics (61%), mHealth solutions (11%), drugs and vaccines (12%), and service delivery approaches (16%). To fund these innovations, SL@B used an open call approach through which it solicited innovative ideas in the MNH field with no restrictions on types of organizations, geographic setting, or specific disease. Most SL@B grantees are academic institutions (40%) and non-profit organizations (38%), followed by for-profit (20%) and public international organizations (PIOs) (2%), with the majority of organizations headquartered in high-income countries (HICs, 83%) versus low- and middle-income countries (LMICs, 17%). Of the HIC-based organizations, 5% had their local country offices as the primary SL@B recipient. Awards were made based on the stage of innovation at three levels: seed, validation, and transition to scale (TTS) with corresponding amount of grant ranging 250,000 USD to 2 million USD. Two SL@B funding partners (USAID and GCC) engaged Duke University in 2018 to design and conduct an evaluation of the program to determine if it was achieving its intended impact and to generate data-driven recommendations for future initiatives. The Evaluating SL@B (ESL@B) research team designed an evaluation around four key evaluation questions, informed by the original call for proposals and SL@B’s theory of change (TOC)1 (see Appendix I for the SL@B TOC diagram): 1. How does SL@B map onto the global landscape of MNH innovation? 2. Does SL@B fill a gap in MNH innovation funding? 3. What has been the impact of SL@B on sourcing and scaling MNH innovations between 2011 and 2020? 4. What is the potential impact of SL@B-funded innovations on MNH mortality and in the MNH ecosystem? This final report draws from multiple data sources and analyses, many of which have been shared through earlier reports and briefs, including the interim synthesis report. This evaluation utilizes a quantitative and qualitative mixed-methods approach using data from multiple sources: 1) program data on the SL@B funding portfolio provided by USAID and GCC and validated by the ESL@B team, 2) extensive desk research of publicly available information on the MNH funding landscape, 3) 80 semi-structured interviews with key informants, and 4) data gathered from SL@B innovators through an online quantitative survey designed by the ESL@B team, and 5) cost data collected through an Excel-based costing tool from five SL@B TTS innovators to conduct cost-effectiveness analysis (CEA).2 These data informed several reports presented to SL@B funding partners within the past year, including the SL@B Portfolio Review and Funding Landscape Analysis, the Acceleration to Impact thematic brief, Summary of Key Qualitative Findings, and the Interim Synthesis Report. The ESL@B team reviewed and synthesized these findings, in addition to incorporating new analyses, for this final report. The Duke University Campus Institutional Review Board (IRB) approved three separate protocols to cover all research related activities [IRB #2018-0370 (existing data), IRB #2018-0617 (interviews), and IRB #2019-0546 (quantitative survey)]. PURPOSE OF THE EVALUATION DATA COLLECTION, METHODS, AND LIMITATIONS EXECUTIVE SUMMARY * At the time of writing this report, one award in Round 8 to Ona Kenya was still under negotiations, and so has been excluded from all analyses in this evaluation. 5 Putting SL@B in context, the program, totaling 77 million USD in innovator support alone* between 2011 and 2020 represents a small amount of funding within the reproductive, maternal, newborn and child health (RMNCH) space. Funding for RMNCH just in 2011, the year in which SL@B was launched, through official development assistance plus grants from the BMGF (ODA+) was 12.2 billion USD, which increased to nearly 16 billion USD in 2017, the latest year for which funding information is available.3,4 The global MNCH space within which SL@B operates is covered by broader cross-sector innovation funding as well as MNCH-specific funding. The SL@B program’s focus on newborns is a distinguishing feature in the field. Because this is relatively rare, the ESL@B team examined the wider space of innovation funding for MNCH. Below is a summary of the key findings for the first two evaluation questions, which have significant overlap. 1. How does SL@B map onto the global landscape of MNH innovation? 2. Does SL@B fill a gap in MNH innovation funding? Broadly speaking, SL@B tended to fund more early-stage and product-based innovations, which historically mapped the program as a unique player within the MNH innovation landscape. While the funding landscape for MNH has changed significantly since the program’s inception in 2011, SL@B continues to fill a funding gap as one of the few open-call programs that funds through innovation growth stages and offers value-added technical support. i. Innovations funded by the SL@B program target 80% of the most common causes of newborn death and 60% of the most common causes of maternal death. ii. Compared to a study sample of 227 funded MNCH programs across 32 funders, the SL@B program was more likely to fund product and technology innovations. The majority of SL@B awards went to product￾based innovations (61%), with the remainder going to service delivery approaches (16%), drugs and vaccines (12%), and mHealth solutions (11%). In interviews, some MNH experts shared a concern that this focus on product and technology innovations creates a risk of developing a portfolio of standalone devices instead of integrated solutions. Interview data also note that the SL@B program should leverage its unique convening power to bring product-based innovations into the fold of integrated health systems. iii. The SL@B program appears to fund a greater proportion of early-stage innovations than most other MNCH funders. Most SL@B awards (86%) have gone to innovations in the first three stages of growth (ideation, research and development, and proof of concept), helping to fill a critical funding gap in the early-middle growth stages. Interviews with MNH experts indicate that SL@B’s focus on early growth stages has provided an opportunity to test ideas and produce viable solutions that otherwise would likely not exist. iv. Analysis of the private equity and venture capital investment landscapes for MNH in Kenya and India (two of the most common target countries for SL@B innovations) indicate this field has dramatically changed since the launch of SL@B in 2011, with increased attention to MNH and willingness to invest in these markets. For transparency, we note several research limitations. The evaluation was largely retrospective in nature, with an emphasis on analysis of existing data collected over the course of the program and a review of publicly available data. Primary data collection included an online survey for innovators, which is subject to missed data from those innovators who did not respond as well as self-report bias; however, our response rate of 54% is considered high for an online survey. Likewise, primary data collection via key informant interviews could have been subject to self-report bias; however, the sample size was high enough to reach data saturation (no new themes emerging) which lends strength to our reported findings. Furthermore, quantifying SL@B’s impact on reducing mortality via cost-effective innovations at a portfolio level was not feasible due to the limitations of effectiveness data for early stage innovations and access to accurate cost data which was innovator-dependent. While CEA for the portfolio was not feasible, CEAs for four selected innovations are being conducted. Lastly, innovation as a concept is not clearly or consistently defined across data and literature sources, and there is limited evidence regarding time to impact or scale of global health innovations, adding difficulty in establishing clear benchmarks for impact metrics of the evaluation. KEY FINDINGS * Between 2011 and 2019, the SL@B partnership invested 77 million USD in direct innovator support, making up the bulk of the partnership funds. This does not include investments towards other non-financial support provided to innovators such as the DevelopmentXChange meetings (travel, conference fees, logistics, etc.), Xcelerator support to innovators, SL@B application, review, and selection processes, grants administration, communications and IT support, etc. 6 v. Interviews with MNH experts indicate that SL@B’s open-call approach has fostered interdisciplinary collaborations, setting it apart from other MNCH funders, including the individual agencies of the SL@B partnership. Data from the SL@B innovator survey indicate that the open call may also attract teams that include members outside of the health field to MNH innovation. vi. Despite the open-call approach, the SL@B review and selection process has tended to fund a higher proportion of innovations originating in high-income countries (HIC) (83%) than low and middle-income countries (LMIC) (17%). Of the innovations from HIC-based organizations, 5% have LMIC-based country offices as SL@B’s prime recipient. No known portfolio balance or target ratio for the SL@B portfolio of funded innovations was established. Interview data with MNH experts suggest that a more balanced LMIC:HIC ratio would be ideal. Interviewees noted that solutions designed by LMIC-based teams may be more likely to be viable, sustainable, and cost-effective. vii. The non-financial support provided by the SL@B program (e.g. Xcelerator workshops,* and targeted connections made by grant managers) is highly valued by innovators, and distinct in the field of MNH innovation funding. Key findings for questions three and four are presented together, as they also have significant overlap. 3. What has been the impact of SL@B on sourcing and scaling MNH innovations between 2011 and 2020? 4. What is the potential impact of SL@B-funded innovations on MNH mortality and the MNH ecosystem? While it is premature for most of these early-stage SL@B innovations to have had a significant impact on global MNH mortality rates, SL@B has been able to source and support innovations toward market entry and position for scale via both financial and non-financial technical support and transformative partnerships. i. Expert interviews demonstrate that the SL@B program is seen as a major driver of innovation and a critical convening platform of both development organizations and a community of innovators. Further, the sourcing strategy has brought in several innovators who are first-time entrants to the MNH arena, proposing novel solutions to address maternal and newborn mortality. ii. Interview data indicate that the SL@B program has leveraged both public and private sectors to increase attention to (and funding for) MNH innovations. Through SL@B’s convening power and networks, the program has catalyzed multi-sectoral collaborations and partnerships for several high-impact innovations that are progressing towards scale, thus improving their path to sustainability even as their SL@B funding period ends. iii. It is difficult to assess SL@B’s potential impact on maternal and newborn mortality, given the early growth stages funded in the SL@B program. Many SL@B innovators are not yet scaling but are still working towards market entry. Of respondents to the SL@B innovator survey, only 28% are operating in at least one market and 15% are operating in at least two markets globally. Longer timeframes are needed for an assessment of the program’s potential impact on mortality. iv. Preliminary data from the CEA indicate that grantees with TTS SL@B awards are interested in having cost￾effective analyses of their innovations in order to leverage further funding to scale. The SL@B program has sourced and scaled some cost-effective innovations; however, impact estimates for most grantees are premature given their early stage of growth (and evidence level) in the scaling pathway. v. Diverse stakeholders value SL@B’s early-stage funding to establish proof-of-concept and the program’s intentional approach to progressively fund an innovation through growth stages in order to support a scaling pathway. vi. The SL@B program appears to accelerate progress through key milestones and growth stages: innovators on average moved through growth stages more quickly during their SL@B funding period than innovators in the same growth stages either before or after their SL@B funding period. vii. Survey data show that innovators credit SL@B’s financial and grants management support (e.g. regular check-ins with SL@B program officers, capacity building workshops, connections) with accelerating their trajectory. Specifically, innovators credit SL@B’s financial support with accelerating the achievement of several key milestones, including validation studies (90%), prototype/program development (89%), and usability studies (88%). * The Xcelerator program is a key element of the non-financial support provided by the SL@B partnership through VentureWell since 2017 (and Duke University in 2018 and 2019) to provide training and capacity building to SL@B innovators to develop and scale their innovation. 7 viii.Interviews with innovators identified several non-financial SL@B supports that innovators credit with accelerating their growth, including the push to think about scaling plans early, having SL@B as a ‘champion’ to attract new funding/partners, introductions to commercialization and implementation partners via the DevelopmentXChange (DevX),* creating a peer network of innovators, and opportunities to participate in Xcelerator workshops and mentorship meetings. ix. The non-financial supports rated as most helpful by respondents to the SL@B innovator survey include the Xcelerator workshops, the DevX events, and pitch support. Interview data indicate that the SL@B program could expand the non-financial scope of support to be even more proactive about facilitating partnerships that support scaling, including those with private sector organizations, national and sub-national public sector health organizations, and organizations with local expertise. TOP RECOMMENDATIONS FOR MOVING FORWARD 1. Each section of this report includes recommendations pertaining to specific thematic areas within the evaluation. The ESL@B team has further distilled and synthesized these data-driven recommendations into those of highest priority for overall program improvement. Define more precisely and publicly SL@B’s program goals, strategy and success metrics over time for funded innovations at both the grantee and portfolio levels. i. Revisit and revise the SL@B Theory of Change (TOC) to update and more clearly define program goals and measurable indicators of success, and use the TOC to align expectations and priorities across SL@B partners. Donor perspectives evolve over time and an agreed-upon TOC will support long-term planning, investment strategies, and stronger impact metrics. a. A key issue for SL@B funding partners to specifically address is the tension between addressing the most pressing health needs, and scalability and sustainability of innovations. Some of the greatest needs in MNH are in settings (e.g., fragile states) and among populations (e.g., low-income and vulnerable populations) where innovations may have transformative impact but would be unlikely to be financially sustainable without long-term public sector and/or donor funding. In contrast, innovations that have the highest potential to be scaled sustainably, may be more likely to prioritize customers or beneficiaries in higher wealth quintiles and operate in more stable (and prosperous) environments. The SL@B program (and other global health innovation programs) can have the most impact by developing clear expectations and targets, at both individual grant and portfolio levels, for health impact, scalability, and sustainability. These important decisions will influence SL@B’s priority markets and the types/stages of innovations sourced. ii. Capture better data prospectively at innovation and portfolio levels over time with clear indicators for outputs and outcomes for each growth stage. Having better data would enhance the ability to benchmark and measure pathways and timelines for milestones, better understand program successes and challenges over time, and facilitate continued adaptive program implementation and learnings. This approach could also help influence the broader MNH and global health innovation fields and lead to stronger data and evidence for innovation beyond the SL@B program. iii. Set and manage stakeholders’ expectations including innovators, external partners, and internal stakeholders about program goals, timelines, target success rates, and potential for sustainability by different types for innovations trying to scale in LMICs. a. Set realistic metrics for three to five years for early-stage innovations, including those that could be transformative, which are unlikely to include mortality reduction as a primary outcome. The unifying theme of “saving lives at birth” is powerful in setting an aspirational vision, but may also unintentionally set unrealistic expectations about what is feasible or achievable in the short to intermediate term for an innovation program. b. Establish success metrics and clear and quantified risk tolerance guidelines grounded in realistic understandings of timelines and failure rates for innovations to facilitate alignment among stakeholders. * The DevelopmentXChange (DevX) is an annual event organized by the SL@B partnership every year since 2011. It has been traditionally held at USAID headquarters in Washington, D.C. and attended by SL@B innovators, SL@B finalists, and potential scaling partners for SL@B innovations. 8 2. 3. Strengthen proactive engagement and buy-in of public and private sector stakeholders in targeted LMICs to prioritize MNH challenges, co-design innovations, co-select innovations, and support scaling of integrated solutions into specific country contexts. Address continued gaps beyond current SL@B program to support sustainable scaled innovations i. Recognize the continued gap between exit from the SL@B program and actual scale. The SL@B program currently serves innovations up to a certain point (TTS being the most advanced stage), but even most successful TTS innovations will not be operating at scale at the end of their SL@B funding period. The SL@B program could address this issue in several ways, including: a. Developing an additional funding category for post-TTS. b. Building strong ecosystem partners in LMIC markets to take on innovations after exiting SL@B. c. Proactively facilitating hand-offs to other funders. Implementation programs supported by SL@B funding partners (including USAID, GCC, and BMGF) could become natural handoff partners. ii. Leverage existing and growing networks and platforms such as Every Woman Every Child (EWEC) and Partnership for Maternal, Newborn and Child Health (PMNCH), as well as the growing number of funders for MNH innovation. Doing so would offer SL@B a significant opportunity to strengthen its approach to later-stage growth and scaling beyond its current approach through TTS funding and support. iii. Facilitate integration of promising innovations. This is a key step in improving the scale and adoption of innovations especially in target LMICs. Innovations are more likely to be used when combined with other successful interventions or existing workflows. The SL@B program has been supportive of these efforts, but creating clear pathways and provisions for innovations to gain acceptability, and be adopted or integrated in health systems will help accelerate time to scale in target countries. Scaling requires alignment across regional, national, and subnational policies, strengthening of sustainable financing, building of appropriate health system capabilities and competencies, and the deployment at scale of multiple complementary innovations that address complex health challenges. The scaling journey is exceedingly difficult for innovators to navigate, and a stronger program emphasis on active facilitation of integration and scaling could be a differentiated value for SL@B or other programs. i. As SL@B considers its geographic prioritization, collaborations with national and sub-national public and private sector stakeholders are important to identify current MNH gaps and opportunities for improvement, as well as garner support and co-lead programs. a. Engage local ministries early, jointly determine priorities, and curate validated innovations that can be scaled in different countries based on need. b. Build on this LMIC engagement to increase in-country support for target markets, to help innovators better understand the context, get real-time insights on the ecosystem, and make targeted connections. ii. Strengthen demand-driven innovation sourcing combined with the open call approach to source diverse innovations and ideas, so that SL@B can build on its strengths and increase its impact in key LMICs. iii. Give some precedence to LMIC-based innovators while strengthening their capacity to implement and scale innovations to re-balance HIC-driven SL@B solutions and to address the issue of sustainably scaling integrated solutions. Achieving stronger local innovation capacity in LMICs would be a positive outcome for the SL@B program to complement the direct impact of the innovations funded. iv. Convene an integrated marketplace for innovations at the LMIC country or regional level with local stewardship and ownership of the platform. a. SL@B’s open call approach could include supplemental targeted calls based on regional priorities and needs. b. Convening a regional marketplace would provide an opportunity for increased visibility for the SL@B program and for countries to select the most impactful innovations for their context. 9 Figure 1. Recommendations for SL@B from the Program Evaluation 10 Saving Lives at Birth (SL@B): A Grand Challenge for Development brings together a global community to combat preventable maternal and newborn deaths and stillbirths on a global scale. Launched in 2011, SL@B is a partnership of key global health stakeholders including Grand Challenges Canada (GCC), the United States Agency for International Development (USAID), the Norwegian Agency for Development Cooperation (NORAD), the Bill and Melinda Gates Foundation (BMGF), U.K. Department for International Development (DFID), and Korea International Cooperation Agency (KOICA).5 As a grand challenge, the approach of SL@B has been to source pioneering innovations to address critical MNH challenges, including newborn survival by accelerating progress towards sustainable solutions.6 SAVING LIVES AT BIRTH: A GRAND CHALLENGE INTRODUCTION Global Maternal and Newborn Health Priorities In 2015, over 300,000 women died due to complications of pregnancy and/or childbirth.7 Nearly all (99%) of those deaths occurred in LMICs, where women from rural, poor communities are the most impacted.8 The majority of maternal deaths are also preventable or treatable; they occur because too many women in low-income countries lack access to emergency obstetric care, skilled birth attendants, high-quality antenatal care (ANC), and family planning services.8 The World Health Organization (WHO) estimates that 2.5 million infants die within the first month of life in a year.9 Stillbirth accounts for additional 2.6million deaths annually.10 Overall, newborn deaths make up 46% of all child deaths globally. The vast majority of these were preventable, occurring due to preterm birth complications, pneumonia, or acute intrapartum related emergencies, such as birth asphyxia.11 See Figure 2 for the most common causes of neonatal and maternal mortality globally. Babies born in LMICs are at much greater risk of death than those born in HICs; children born in the WHO’s Africa Region are six times more likely to die in their first year than those born in the WHO’s European Region.12 Figure 2: Causes of Maternal and Neonatal Mortality Globally Source: Estimates from L.Say, et al. (2014)13 for Maternal Mortality, and Liu, et al. (2016)11 for Neonatal Mortality. Panel A: Causes of Maternal Mortality Panel B: Causes of Newborn Mortality 1 11 The SL@B funding partners engaged Duke University in 2018 to design and conduct an evaluation of the SL@B program to determine how well it was achieving its intended impact and to generate data-driven programmatic recommendations for future iterations of the program. The multi-disciplinary team of researchers and evaluators at the Duke Global Health Institute’s Evidence Lab and the Duke Global Health Innovation Center designed an evaluation with four key evaluation questions, informed by the original call for proposals and SL@B’s Theory of Change (see Appendix I for the SL@B TOC diagram):1 1. How does SL@B map onto the global landscape of MNH innovation? 2. Does SL@B fill a gap in MNH innovation funding? 3. What has been the impact of SL@B on sourcing, supporting, and scaling MNH innovations between 2011 and 2018? 4. What is the potential impact of SL@B-funded innovations on MNH mortality and in the MNH ecosystem? This final report utilizes a mixed-methods approach using quantitative and qualitative data collected through multiple sources: 1. Program data on the SL@B funding portfolio, provided by USAID and GCC and validated by the ESL@B team. 2. Extensive desk research and review of the MNH funding landscape, including a dataset that the ESL@B team created from publicly available information on a sample of 227 MNCH programs funded by 32 key funders between 2011-2018; as well as data on public and private investments in MNH in India and Kenya collected from the Pitchbook database, which aggregates data on venture capital, private equity, and various public sector investors. 3. Data collected through 80 semi-structured interviews, including with 33 SL@B innovators and implementing partners, of which 17 are based in LMICs, 19 RMNCAH funders, 28 global MNH experts, non-SL@B innovators, and SL@B funding partners. Innovator interviews largely (though not exclusively) focused on those operating (or planning to operate) in Kenya, Ethiopia, and India. 4. Data collected from SL@B innovators through an online survey to measure their progress through growth stages, scale, challenges, and successes accomplished before, during, and after their grant period. 5. Cost data collected from five SL@B transition to scale (TTS) innovators through an Excel-based costing tool to conduct cost-effectiveness analysis (CEA).2 The ESL@B team analyzed the data collected from the sources listed above and presented results in several previous reports to SL@B funding partners, specifically the SL@B Portfolio Review and Landscape Analysis report, the Acceleration to Impact thematic brief, the Summary of Key Findings from In-depth Interviews report, and the Interim Synthesis Report. The ESL@B team further synthesized these findings, in addition to incorporating new analyses, for this final report. The Duke University Campus Institutional Review Board (IRB) approved three separate protocols to cover all research related activities [IRB #2018-0370 (existing data), IRB #2018-0617 (interviews), and IRB #2019-0546 (quantitative survey)]. All key informants gave written informed consent to participate. PURPOSE OF THE EVALUATION DATA, METHODS, AND LIMITATIONS SL@B’s Focus on Innovation to Address Maternal and Newborn Mortality The SL@B program emphasizes the importance of developing, validating, and scaling promising social and busi￾ness innovations, with a primary focus on science and technology, service delivery, and demand creation to solve complex challenges. Since its launch in 2011, SL@B has funded 147 innovations,* of which 116 are unique, from 92 innovators solving for critical MNH needs in low-resource settings. These projects, including diagnostics, drugs, care delivery models, and other types of innovations, tackle the major causes of maternal and neonatal mortality and stillbirths. While the diverse portfolio includes innovations across growth stages and at various levels of scale, many are relatively early in development. Accordingly, the SL@B funding partners have designed the program to combine financial, technical, and strategic support for innovations on the path from idea to operating at scale. * At tthe time of writing this report, one award in Round 8 to Ona Kenya was still under negotiations, and so has been excluded from all analyses in this evaluation. 12 The structure of this final report is informed by key areas of interest for the SL@B partners, as described in the SL@B TOC and the SL@B Impact Areas Framework (please see Appendix I for the TOC diagram and Appendix II for the Impact Areas Framework). The report is organized as follows: • Section 2 describes the SL@B program in relation to the global MNCH landscape • Section 3 details SL@B’s role in building innovator capacity and accelerating innovations • Section 4 highlights partnerships created by SL@B innovators and the role SL@B played in facilitating partnerships for increased innovation impact • Section 5 outlines the work done to quantify SL@B’s impact • Section 6 concludes with a discussion presenting additional perspectives to inform program design and elements The ESL@B team provides recommendations for the SL@B program at the end of each section, informed by the findings in that section. We have also distilled these data-driven thematic recommendations into a priority list of potential program changes, presented in the Executive Summary. The discussion section encompasses a broader perspective of the ESL@B team members on the SL@B program and future directions beyond data-driven observations presented in the report. STRUCTURE OF THIS REPORT LIMITATIONS For transparency, we note several research limitations. The evaluation was largely retrospective in nature, with an emphasis on analysis of existing data collected over the course of the program and a review of publicly available data. Primary data collection included an online survey for innovators, which is subject to missed data from those innovators who did not respond as well as self-report bias; however, the survey received a response rate of 54%, which is considered high for an online survey. Likewise, primary data collection via key informant interviews could have been subject to self-report bias; however, the sample size was high enough to reach data saturation (no new themes emerging) which lent strength to our reported findings. Furthermore, quantifying SL@B’s impact on reducing mortality via cost-effective innovations at a portfolio level was not feasible due to the limitations of effectiveness data for early stage innovations and access to accurate cost data which was innovator-dependent. While portfolio-level CEA was not feasible, CEAs for four selected innovations is being conducted. The ESL@B team found that there is no clear or consistent definition of innovation as a concept, and that there is limited evidence regarding time to impact or scale for global health innovations, adding difficulty in establishing clear benchmarks. When possible, categorizations such as stage of growth were determined by the ESL@B team using available descriptions of projects based on established sources such as the International Development Innovation Alliance (IDIA) for growth stages.14 Finally, the SL@B program’s selection process and criteria are outside the scope of this report as the ESL@B team was asked not to review or analyze this data for the evaluation. 13 • Innovations funded by the SL@B program target 80% of the most common causes of newborn death and 60% of the most common causes of maternal death. • Compared to a study sample of 227 funded MNCH programs across 32 other funders, the SL@B program was more likely to fund product and technology innovations. In interviews, some MNH experts shared a concern that this focus on product and technology innovations creates a risk of developing a portfolio of standalone devices instead of integrated solutions. • The SL@B program appears to fund a greater proportion of early-stage innovations than most other MNCH funders, helping to fill a critical funding gap in the early-middle growth stages. Interviews with MNH experts indicate that SL@B’s focus on early growth stages has provided an opportunity to test ideas and produce viable solutions that otherwise would likely not exist. • Analysis of the private equity and venture capital investment landscapes for MNH in Kenya and India (two of the most common target countries for SL@B innovations) indicate this field has dramatically changed since the launch of SL@B in 2011, with increased attention to MNH and willingness to invest in these markets. • Interviews with MNH experts indicate that SL@B’s open-call approach has fostered interdisciplinary collaborations, setting it apart from other MNCH funders, including the individual agencies of the SL@B partnership. Data from the SL@B innovator survey indicate that the open call may also attract teams that include members outside of the health field to MNH innovation. • Despite the open-call approach, the SL@B review and selection process has tended to fund a higher proportion of innovations originating in high income countries (HIC) (83%) than low-and middle-income countries (LMIC) (17%). Of the HIC-based organizations, 5% have LMIC-based country offices as SL@B’s prime recipient. No known portfolio balance or target ratio for the SL@B portfolio of funded innovations was established. Interview data with MNH experts suggest that solutions designed by LMIC-based teams may be more likely to be viable, sustainable, and cost-effective. • The non-financial support provided by the SL@B program (e.g. Xcelerator workshops,* targeted connections made by grant managers) is highly valued by innovators, and distinct in the field of MNH innovation funding. KEY FINDINGS THE SL@B PROGRAM IN THE GLOBAL LANDSCAPE: Sourcing and Funding MNH Innovations 2 * The Xcelerator program is a key element of the non-financial support provided by the SL@B partnership through VentureWell s ince 2017 (and Duke University in 2018 and 2019) to provide training and capacity building to SL@B innovators to develop and scale their innovation. 14 It is important to consider the contributions and value of the SL@B program within the contexts of both the broader RMNCH ecosystem and the grand challenge approach. In examining SL@B’s position within the broader RMNCH landscape, we recognize that the SL@B program, totaling 77 million USD from 2011 to 2020, is a small program in the global RMNCH landscape. In 2011, funding from official development assistance and grants from BMGF (ODA+) for RMNCH was 12.2 billion USD, which increased to nearly 16 billion USD in 2017, the latest year for which funding information is available.3,4 Of this, funding for MNH in 2017 was 3.1 billion USD and approximately 2 billion USD in 2011.4 Despite such a large investment , an additional 33.3 billion USD per year is needed to achieve the targets set in the EWEC Global Strategy and SDG goals for RMNCH.15 As many funders do not differentiate targeted innovation funding within their portfolios, it is difficult to accurately say what percentage of global ODA+ funds have been spent on MNH-focused innovations. While the SL@B portion of funding for innovation is relatively small in the larger MNH funding landscape, it aims to achieve sizable impacts in terms of improving and saving lives, calling attention to neglected MNH needs in LMICs, and popularizing innovation to address those needs. SL@B’s sourcing and funding strategy in the global landscape of MNH funders To better understand trends in MNCH innovation funding, how the SL@B portfolio compares to the larger landscape in terms of health priorities and stages of growth, and the role played by the SL@B program within this field, the ESL@B team developed a funding landscape data matrix by collecting and analyzing publicly available information on 32 funders of MNCH programs and a sample of 227 projects supported by these funders between 2011 and 2018. The ESL@B team selected the sample of 32 MNCH funders for desk research with input from USAID and GCC. Selection criteria included funders who had funded at least one MNCH innovation between 2011 and 2018 and focused on three types of funders: philanthropic, investment (debt and equity), and development aid organizations that fund individual organizations. In addition, the team conducted 25 semi-structured interviews with funders and both SL@B and non-SL@B innovators. Taken together, analysis of the composition of the SL@B portfolio and the broader funding landscape provides insights on the role played by the SL@B program in meeting funding needs in the MNCH field, and determining whether there are remaining gaps in the funding landscape for MNCH innovation. Who and what does SL@B funding target? Type of organizations. Most global MNH funders invest in a variety of organization types, including non-profits, for-profits, and academic organizations. Some also fund governments, though this is not as common. Analysis of the MNH funding data matrix described earlier and key informant interviews indicate that philanthropic grant funders are more likely than equity or debt funders to source university-based projects. Similar to other philanthropic funders, a large portion of SL@B awards went to university-based innovators (n=59), followed closely by non-profit organizations (n=56). About a fifth of the portfolio (n=29) is for-profit organizations, and three innovations are from public international organizations (PIOs). Despite the majority of SL@B innovations being university-based, the SL@B program has funded different types of organizations across its eight funding rounds as described above. Other funders emphasized the need for diverse organizations as well; they noted that no single type of institution can drive change, which can only be accomplished by a combination of many. Having said that, most funders report that they value potential for impact and mission alignment as opposed to the type of organization when deciding which projects to fund. Type of innovation. For this analysis, innovations are categorized into four primary types: 1. Drugs and vaccines 2. Products (including devices and diagnostics) 3. mHealth (including digital health and eHealth) 4. Practice and approach (including care delivery and behavior change) CONTEXT FOR ASSESSING VALUE OF SL@B IN GLOBAL MNH LANDSCAPE 15 The innovation types were selected in consultation with USAID. While these categories are comprehensive, they are not mutually exclusive. For example, many of the mHealth innovations focus on care delivery and behavior change and could also be categorized as practice and approach models. For this analysis, the ESL@B team classed each innovation under one primary category for innovation type and growth stage, though many could very well fit into more than one at the same time. The SL@B program has maintained a strong focus on products, including devices and diagnostics. Across the eight rounds of SL@B, 60% of awards went to product innovations (n=89). Practice and approach innovations received 16% of awards (n=24). Innovations primarily classed as drugs/vaccines (n=18), and mHealth (n=16) are evenly represented, each making up about 12% of the awards made. Key informants interviewed for this evaluation expressed concerns that the majority of SL@B funding went to product-based innovations, given that the program was designed to source solutions of many types, not just technology. MNH experts in particular noted that there is a risk that new products and technologies will exist in the market as standalone devices, which will not be sufficient to solve MNH issues. These informants noted the importance of integrating innovations into more comprehensive solutions that augment the financial, policy, and care delivery systems of healthcare. It is the intention of the SL@B program to grow product-based innovations into integrated and bundled innovations through its staged funding approach. Notable examples of innovations that started off as a product in earlier rounds, but have grown into integrated solutions are the Bili-Kit (Little Sparrows), Pratt Pouch (EGPAF), and bubble CPAP (Rice University). Similarly, Chlorhexidine (JSI) and the Universal Anesthesia Machine (Gradian) are on their way to integration into the national health policy of their respective implementation countries. Similar efforts are underway for other innovations as the SL@B portfolio matures. The following quote from a key informant reiterates the need to strengthen the health system so that innovations could be easily integrated. Figure 3: Number of Innovations, by Innovation Type and Round Source: Duke University analysis of SL@B program information provided by USAID and GCC “But again, it depends on the strength of the systems; you could have the best innovation in the world, but if you don’t have a strong system to implement it, then, that’s just not going to work.” – HIC MNH expert 16 Figure 4: Funding Portfolios by Target Population Source: Global MNCH Funders’ Population Focus compiled by ESL@B team from public data available online; SL@B Population Focus received from SL@B program and validated by ESL@B team. * SL@B as a program is primarily focused on addressing MNH issues around the time of childbirth, although some innovations cover the broader spectrum including child and reproductive health i.e. MNCH or RMNCH. When putting SL@B in the context of the global ecosystem, it’s hard to parse out MNH since most global health data encompasses RMNCH. Therefore, in this report, we use MNH when it’s specific to SL@B and MNCH or RMNCH when referring to SL@B in relation to the broader ecosystem. ** Most funders of MNCH projects are not specifically focused on MNH but include it as a part of a larger portfolio of projects focusing on a variety of RMNCH and systems strengthening issues. Disease focus. The SL@B portfolio addresses the most common causes of maternal and neonatal deaths across the eight rounds, targeting 80% of the most common causes of neonatal death and 60% of the most common causes of maternal deaths. Despite the fact that the SL@B portfolio addresses the most common causes of maternal and neonatal mortality, a few key informants noted that there is still a gap in addressing stillbirth, a major contributor to mortality. This gap highlights the need for more innovations directly addressing stillbirth in the SL@B portfolio as well as in the larger MNH ecosystem. While a significant number of SL@B innovations may have the potential to contribute to reductions in stillbirths, that indicator is not routinely monitored or reported on by the innovators. Categorizing innovations by their potential impact on stillbirths was outside the scope of work for this evaluation. “One of the most fundamental questions [of the SL@B program], even before you get to the nitty-gritty of a life saved is – here are the three groups: 300,000 maternal deaths, quite a lot around the time of birth, 2.6 million stillbirths, half of which during 24 hours when the woman is in labor, and then neonatal death, 2.6 million of which about 40% of the first couple of days. So, of those, what’s the match between the inputs and whether they’re addressing the biggest things.” – HIC MNH expert Panel A: Global MNCH Funder Population Focus Panel B: SL@B Population Focus Population focus. Most funders of MNCH projects are not specifically focused on MNH but include it as a part of a larger portfolio of projects focusing on a variety of RMNCH and systems strengthening issues.* As indicated in Figure 4 below, over half (56%) of the projects reviewed** target maternal, newborn, infant, and child health broadly. The remaining projects are split between those that focus specifically on neonatal or maternal causes. The SL@B portfolio, on the other hand, is generally equally split with a primary focus on maternal health (37%), followed by maternal and newborn health (34%), and then newborn health (29%). 17 SPECIFIC FOCUS COUNT % OF PORTFOLIO Maternal 56 38.2% Postpartum hemorrhage 13 8.8% Pre-eclampsia 11 7.5% Intrapartum complications 7 4.8% Health service delivery 6 4.1% Reproductive health 6 4.1% Other (diagnostic tests, vaccine/drug delivery) 6 4.1% Micronutrient deficiency 5 3.4% Infection 1 0.7% Prematurity 1 0.7% Maternal & Newborn 49 33.3% Health service delivery 26 17.7% Intrapartum complications 8 5.4% HIV/PMTCT 5 3.4% Infection 4 2.7% Respiratory distress 3 2.0% Other (diagnostic tests) 2 1.4% Micronutrient deficiency 1 0.7% Newborn 42 28.5% Respiratory distress 16 10.9% Jaundice 8 5.4% Infection 5 3.4% Health service delivery 4 2.7% HIV/PMTCT 4 2.7% Prematurity 4 2.7% Other (drug delivery) 1 0.7% Grand Total 147 100% Source: SL@B portfolio targets of maternal and newborn mortality provided by USAID and GCC and validated by ESL@B team. Table 1: SL@B Portfolio by Causes of Maternal and Newborn Mortality For this evaluation, “For this evaluation, the ESL@B team used six stages of growth for innovations as defined by the International Development Innovation Alliance. 1. Ideation: Define problem, scan landscape, develop initial ideas 2. Research and Development: Develop specific innovative solution 3. Proof of Concept (POC): Field test innovation to assess potential 4. Transition to Scale (TTS): Refine model, develop partnerships to enter market 5. Scaling: Replicate or adapt to increase impact 6. Sustainable Scale: Wide-scale implementation, sustained by ecosystem Stage of growth and development Table 1 shows types of diseases and conditions targeted by SL@B. 18 For initial analyses, the ESL@B team defined four growth stages for innovations in the SL@B program based on the scaling stages developed by the SL@B program partners. In later analyses, in consultation with the SL@B funding partners, the ESL@B team transitioned from the SL@B growth stage categories to the IDIA framework for the analyses presented in this report.14 Figure 5 shows an approximate alignment between the different growth stage frameworks. Most MNCH funders included in this analysis are either agnostic to stage of growth of the innovation (from ideation to sustainable scale) or have a specific focus on the later growth stages. Across the sample of non-SL@B MNCH innovations, funding awards could be divided into thirds, with 36% going to the three early stages (ideation, R&D, and proof of concept), about 30% going to transition to scale stage projects, and about 34% going to scaling stage projects. However, growth stage data for the projects included in the funding landscape sample were limited in availability and difficult to verify. Therefore, any analysis of growth stages among other MNCH funders included in this research is only suggestive. Interviews with SL@B innovators revealed that a number of funders have provided early-stage support but many of these (including DFID, National Institutes of Health, National Sciences and Engineering Research Council, and the Wellcome Trust) are more likely to source innovations within large established institutions, such as universities, rather than start-ups. Further, early-stage support may be more likely to fund HIC-based teams, rather than LMIC￾based enterprises. Thus, LMIC-based social enterprises (for-profit and not-for-profit) may have fewer options for early-stage funding until they are able to show proof-of-concept or validation. In contrast to most funders in the landscape, the SL@B program funds a greater proportion of early-stage innovations. The ESL@B team used program data to determine the IDIA growth stage for each awardee at the time of award. For this analysis, we examined each innovation as distinct (n=147), including those that received multiple awards. Additionally, we class each innovation under one primary growth stage, but could fit into more than one at the same time. Table 2 shows the distribution of innovations in the SL@B portfolio by type and stage of innovation. Approximately 47% of innovations funded are in the ideation (n=3) and research and development (n=66) stages; but along with proof of concept (n=57), early-stage innovations make up 86% (n=126) of the SL@B portfolio from rounds one through eight. This concentration of early-stage innovations is reflected in product-based innovations as well: making up 61% of SL@B innovations (n=89); about 93% of product-based innovations are in the early stages of scaling: ideation, research and development, and proof of concept (n=83). Figure 5: IDIA and SL@B Innovation Growth Stage Categories Table 2: Number of Innovations, by Type and Stage TYPES/ STAGES IDEATION RESEARCH & DEVELOPMENT PROOF-OF￾CONCEPT TRANSITION￾TO- SCALE SCALING TOTAL Drugs & Vaccines 1 9 5 1 1 18 mHealth - 7 6 2 1 16 Practice/Approach 1 6 8 8 1 24 Products 1 44 38 7 - 89 Total 3 66 57 18 3 147 Source: SL@B program information provided by USAID and GCC, and validated by the ESL@B team. IDIA Scaling Stages 1. Ideation 2. Research & Development 3. Proof-of-Concept 4. Transition to Scale 5. Scaling 6. Sustainable Scale Define problem, scan landscape, develop initial ideas Develop specific innovative solution Field test innovation to assess potential Refine model, develop partnerships to enter market Replicate or adapt to increase impact Wide-scale implementation sustained by ecosystem Growth Stages Applied by SL@B 1. Development 2. Validation 3. Early Adotpion 4. Scaling Research and development of initial idea, create prototype Establish proof￾of-concept, field testing, clinical trials Entry into target market, demand generation Increase impact in original and/ or new markets 19 Source: SL@B program information provided by USAID and GCC, and analyzed by the ESL@B team Figure 6 shows that although the proportion of early-stage (ideation, research and development, and proof of concept) innovations was high in rounds one through four, accounting for over 80% of funded innovations in round one (n=21), in later rounds, SL@B funding trended towards later stages of development. This trend can be partly accounted for by SL@B’s maturing portfolio. Twenty-five innovations (17% of all SL@B innovations) that received early-stage funding in rounds one through five received follow-on later-stage funding from SL@B in rounds four through eight. Figure 6: Number of SL@B Innovations, by Stage of Growth and Round How does SL@B fund by stage of growth? Type of award. In the first four rounds, SL@B awarded two types of grants: seed and TTS. The proportion of seed grants in this period varied between 80% and 90%, with TTS grants making up the balance. However, several of the TTS awardees in these earlier rounds were still in early stages of growth (ideation, research and development or proof-of-concept, rather than an actual TTS stage), and were not yet poised for scale in one or more markets. Starting in round five, the SL@B program introduced validation grants to distinguish purely development grants from those that are past the idea stage and ready to test the prototype or validate their model. After the introduction of validation grants, the proportion of TTS grants increased significantly, from 13% in round four to 31% in round seven, while seed grants decreased from 87% to 25% in round seven. Only validation and TTS awards were offered in the request for proposal (RFP) for round eight, with all final awards ending up being TTS. Validation grants constituted two-thirds of all grants in round five, but this reduced to 44% in round seven. Of the number of grants awarded across the eight rounds, nearly two-thirds of awards were seed grants (n=93), about a fifth were validation grants (n=28), and 18% were TTS grants (n=26). Overall, these findings indicate that the SL@B program built a distinct portfolio as compared to the wider funding landscape by sourcing and funding more innovations in the earliest stages (ideation, research and development, and proof of concept). Most key informants expressed the importance of targeted funding at such an early stage and having the opportunity to test their ideas and produce viable solutions. “[SL@B has] managed to roll out funding for a variety of good ideas to people and groups who otherwise would not have had a chance of getting any funds… it did bring in interesting innovations which would otherwise have not seen the light of day.” – HIC MNH expert 20 Where is SL@B funding targeted? Among the global MNH funders reviewed, most target key countries with high levels of MNH need, including India, Nigeria, Mexico, Uganda, and South Africa. The SL@B portfolio targets many, but not all countries with the highest maternal and neonatal deaths, even when looking at the top 25 countries with the greatest burden in terms of absolute numbers. The ESL@B team also examined the overlap between the countries with a high proportion of proposed SL@B innovations and global maternal and neonatal mortality ratios. There is very little overlap between countries with the greatest burden of maternal and neonatal mortality ratios and those countries targeted for implementation by SL@B-funded innovations. Panels A and B in Table 3 show that SL@B is better represented when examining countries by absolute numbers of maternal and neonatal mortality. It is likely that there is a discrepancy between the countries that SL@B has tended to focus on (e.g. Kenya, Uganda) and countries with highest burden (especially in terms of rates and ratios) because of SL@B donors’ and HIC awardees’ established connections on the ground, better infrastructure for testing and validating innovations in countries like Kenya and Uganda relative to difficult-to-reach/poorly-resourced locations, and a reluctance to work in conflict/fragile states. Beyond the countries that SL@B targets, it is important to assess the subgroups that the innovators reach with their innovations. The ESL@B team attempted to use the EquityTool—a simplified version of the Wealth Index from the Demographic and Health Survey (DHS) which is widely regarded as the gold standard of population health and development in low income settings—to examine whether SL@B grantees were addressing equity within their beneficiary groups.18 Although this ended up not being feasible at the portfolio level, we did pilot its use with one product innovator, and results indicated that more than half of the beneficiaries (67%) are above the two lowest wealth quintiles. While this innovator worked in a lower-resource country, they did not necessarily primarily serve the most vulnerable. It is worth noting that this analysis was conducted based on midline data collected from an urban intervention site of the innovation. The reach of this innovation could change as it makes progress towards scale. For example, an innovation might start in an urban area for convenience and sustainability purposes and move to rural and/or underserved areas as it scales, thereby reaching a more vulnerable population. Nonetheless, addressing equity is a challenging but necessary gap that must be addressed to have a measurable impact on global mortality. While a larger portfolio-level analysis is needed for a comprehensive understanding of the SL@B’s program ability to address health inequalities within low-resource countries, our example indicates the need for innovations to collect data on their beneficiaries because working in countries with high maternal and neonatal mortality ratios might not suffice to address the need of the most vulnerable. Other aspects that need to be considered regarding achieving equity are sustainability and business models of an innovation. If an innovation is solely targeting the most vulnerable, it may never be sustainable and might require dependence on other entities. Therefore, the sustainability of the business model is often dependent on the type of beneficiaries an innovation intends to reach. 21 Table 3: Top 25 Countries with Highest Burden of Maternal Mortality Ratios, Newborn Mortality Rates, Maternal Deaths, and Newborn Deaths, Compared to Number of SL@B Innovations Targeting those Countries Source: Estimates from WHO, UNICEF, UNFPA, World Bank Group, and UN Population Division Panel A: Top 25 Countries by Maternal Mortality Ratios and Newborn Mortality Rates, Number of SL@B Grants16,17 MMR per 100,000; NMR per 1,000 live births Country MMR NMR Number of SL@B Grants South Sudan 1,150 83 0 Chad 1,140 45 0 Sierra Leone 1,120 43 1 Nigeria 917 46 4 Somalia 829 80 0 Central African Re￾public 829 70 0 Mauritania 766 65 0 Guinea-Bissau 667 56 0 Liberia 661 1 Afghanistan 638 46 0 Côte d'Ivoire 617 42 0 Gambia 597 51 0 Guinea 576 41 0 Mali 562 41 1 Burundi 548 0 Lesotho 544 50 0 Cameroon 529 1 Tanzania 524 9 Niger 509 0 Eritrea 480 0 Haiti 480 0 DR Congo 473 42 1 Zimbabwe 458 0 Eswatini 437 0 Ethiopia 401 4 Comoros 73 0 Angola 55 0 Pakistan 52 6 Djibouti 52 0 Equatorial Guinea 51 0 Botswana 50 0 Turkmenistan 49 0 Micronesia 41 0 Kiribati 41 0 Yemen 39 0 Panel B: Top 25 Countries by Number of Maternal and Neonatal Deaths, Number of SL@B innovations 7 Country Maternal Deaths Newborn Deaths Number of SL@B Grants Nigeria 67,000 336,372 4 India 35,000 631,754 17 DR Congo 16,000 144,291 1 Ethiopia 14,000 124,323 4 Tanzania 11,000 57,118 9 Indonesia 8,600 74,921 1 Pakistan 8,300 309,419 6 Afghanistan 7,700 54,890 0 Chad 7,300 0 Uganda 6,000 42,158 13 Côte d'Ivoire 5,400 37,360 0 Bangladesh 5,100 59,954 10 Niger 5,100 38,845 0 Somalia 5,100 48,764 0 Kenya 5,000 38,579 29 China 4,900 90,359 1 Cameroon 4,700 30,246 1 South Sudan 4,500 31,832 0 Mali 4,400 31,943 1 Sudan 3,900 50,820 0 Mozambique 3,100 43,458 0 Angola 3,000 67,375 0 Sierra Leone 2,900 1 Madagascar 2,800 0 Ghana 2,700 11 Egypt 40,898 1 Philippines 38,680 1 Yemen 34,083 0 Myanmar 30,569 0 22 Origin of innovation. Most global funders source innovations across multiple geographic regions, while a few formally target specific countries or regions. These funders often informally target organizations in geographies where they have stronger partnerships or believe there is untapped potential. Most global funders are open to funding teams headquartered in HICs as well as LMICs, provided the project targets LMIC populations. Global MNH funders identify and source innovations through different methods such as word-of-mouth and on-the-ground networks, in addition to putting out calls for proposals for specific focus areas at times. Relying on connections and networks serves a range of purposes for these funders, mainly sourcing local innovations and partners in LMICs and establishing long-term relationships on the ground. Despite this approach, some funders find it difficult to find institutions and innovators in LMICs who are a good fit for their funding program. SL@B has an open call approach that aims to source innovations from diverse geographic settings, especially from LMICs. However, the SL@B review and selection process has tended to yield and fund a higher proportion of innovations originating in HICs. Of the HIC- based innovations, 5% of them have their LMIC country offices as primary SL@B recipients. Between rounds one and eight, only 17% of funded innovations (n=25) were from teams whose headquarters are based in LMICs, who received 19% of the overall funding. The proportion of innovations from LMICs has fluctuated across rounds, highest in round eight (33%) followed by round one (25%), and lowest in round six (6%). The program seems to have made a significant shift in round seven in terms of the amount of funding awarded to LMICs: while innovations from LMICs made up 13% of round seven innovations, they received 36% of the funding amount disbursed; Figure 7 shows the distribution and investment of funding across rounds. Figure 7: Number of SL@B Innovations and Amount of SL@B Investment, by Awardee Location Source: SL@B program information provided by USAID and GCC, and analyzed by the ESL@B team The ESL@B team classified innovations as either one in a low and middle-income country (LMIC) or high-income country (HIC) based on the registered headquarters of the primary organization listed on the award. Even if some of the HIC-based organizations have operations led by a local country office, we classify such entities as HIC￾based as they have access to stronger networks for fundraising and may be more competitive in submitting grant applications than other LMIC-based organizations. However, this classification could potentially miss the fact that country offices are the primary recipients of the award, their leadership and staff have autonomy in decision-making, and that the innovation itself is embedded in LMIC-settings. The ESL@B team identified six out of the 122 (5%) HIC based innovations as having LMIC country offices as the primary recipients. Box 1. Classifying Innovations Based on Headquarters of Primary Award Recipient Each round has two vertical bars: within each round, the bar on the left shows the number of SL@B innovations funded in that round, and the bar on the right shows the total amount invested (in million USD). For both of these variables, the graph further shows the distribution between HICs (gray bar) and LMICs (purple bar). 23 Interview data also indicate challenges that LMIC innovators face, which may help to explain the lack of LMIC representation within the SL@B portfolio: • Limited access to information and awareness about funding opportunities through traditional funding announcement mechanisms such as SL@B in LMICs. • Inability of many LMIC innovators to regularly participate in international conferences and events such as DevelopmentXchange (DevX)* or other workshops (mostly due to visa, logistics, and financial issues) that connect them with the larger innovation community, broaden their understanding around MNH innovation, and link them with funding opportunities. • LMIC organizations that are not well-known or have strong track records might be judged less favorably than those (especially HIC-organizations) with better known capabilities by the review panels on measures such as feasibility and likelihood of success. • LMIC innovators find it challenging to compete with more experienced, better trained, and better resourced HIC innovators in terms of writing winning proposals and executing grants. • Grant/financial managers working with LMIC innovators may need to provide increased levels of management, capacity building, and time commitment. Among key informants who were asked about the composition of LMIC and HIC innovators in the SL@B portfolio, there was a clear consensus that this ratio was not ideal. Two-thirds of key informants underlined the importance of a more balanced portfolio in terms of innovator origin. Expert opinion from interview data suggests that people based in resource-limited settings are more likely to have a deeper understanding of the cultural and health ecosystem contexts, including constraints and barriers that innovations might face in moving to scale. Most key informants believed that innovations designed by LMIC innovators for LMIC contexts are more likely to be feasible, sustainable, and cost-effective. That said, the aggregate quantitative data to additionally support these expert opinions is still pending at the global level. Future longitudinal meta-analyses on the sustainability of innovations from multiple grand challenges and accelerators could further illuminate this issue. It is difficult within the current SL@B program data to measure the relative success of LMIC￾based and HIC-based teams in achieving key milestones on the path to feasibility and scale. This may be a consideration for partners when developing future iterations of the monitoring and evaluation strategy. As noted elsewhere in this report, a strong monitoring plan with growth stage-specific milestones can help the funding program identify factors that facilitate scale. Interview data suggest that “on-the-ground” expertise could influence the design phase of innovations and save time during the seed and validation (iteration) phases. “I think there should be a better balance for sure. You know, people from high-income countries are trying to develop innovations that they think would be appropriate for low-income countries and sometimes they’re successful and sometimes they’re not. People from low-income countries know exactly what the situation is, and they know what the possibilities are and constraints are; so, they might design with those constraints in mind to come up with solutions. So, I definitely think that there should be a better balance.” – LMIC MNH expert “We cannot develop an innovation just based out of our lab. Biomedical engineers will only actually have a clearer idea of what’s not working when they have conversations with target users as to how to make it work and then come back with a solution.” – HIC MNH expert “I think there are two or three barriers for LMIC. One is they’re openly competing with people like me [a HIC innovator at a big academic institution] who write grants for a living, companies for medical devices with 30 years even sometimes 40 years of experience designing medical devices, people like PATH, JHPIEGO, or FHI360. These are, you know, very experienced.” – SL@B innovator (drug) * The DevelopmentXChange (DevX) is an annual event organized by the SL@B partnership every year since 2011. It has been traditionally held at USAID headquarters in Washington, D.C. and attended by SL@B innovators, SL@B finalists, and potential scaling partners for SL@B innovations. 24 Box 2. Investment Funding for MNH The analyses above draw on a sample of funders primarily made up of philanthropic and development funders. To better understand the investment (rather than philanthropic) ecosystem for MNH and how it has changed since the launch of the SL@B program, the ESL@B team conducted an analysis of private and public investment (including debt and equity) targeting the maternal and neonatal health industry in Kenya and India. The team chose these countries because of the high proportion of SL@B-funded innovations targeting these markets. Using data from Pitchbook* , the team identified the “top ten” investors in organizations focused on MNH from 1990 to 2011 and compared these investment organizations with the top ten investors from 2011 to 2019 with the same MNH focus. For this analysis, investors consisted of venture capital, private equity, and public sector organizations who invested in organizations focusing on MNH. The analysis shows that the investment landscape for MNH in Kenya and India has dramatically changed since the launch of SL@B in early 2011, indicating increased attention on MNH issues and willingness to invest in these markets. Of note, while Kenya and India are among the top target countries for proposed SL@B innovations, both are middle-income countries. Lower-income countries may not necessarily see these burgeoning investment opportunities as readily until further development and infrastructure become available, as seen in India and Kenya in the last quarter of the twentieth century. According to Pitchbook data, prior to 2011, there were only 7 investors (of 106 investing in MNH) that made any deals in MNH enterprises within LMICs. This has increased nearly ten-fold, with over 70 investors out of 165 that made active deals as of 2019. From 2006-2010, the first time period in which any investment was made in MNH enterprises within India or Kenya, the ‘top ten’ list of investors were all privately-held venture capital firms or merchant banks that collectively invested $17 million USD in four distinct MNH-related companies, mainly hospitals, maternity services, or genomic-related companies. After SL@B’s debut, in early 2011 through 2014 and from 2015 to the end of 2019, the ‘top ten’ list of investors include venture capital funds, governments, accelerators/incubators, and not-for-profit venture capitalists who collectively invested more than $82 million in 22 MNH enterprises. The funded enterprises included social-impact organizations, innovators based in LMICs, and a greater inclusion of seed/early stage innovations. The investors themselves in 2019, compared to pre-2011, are not isolated to HICs but also represent domestic investment firms that are based in India and Kenya. In less than a decade, the amount invested has increased exponentially, indicating that this industry is still growing within LMICs such as India and Kenya. These investments, along with a growing atmosphere supporting innovation in India and Kenya, has led to a significant increase in the number of maternal and newborn services and innovations, and has played a role in the improved health outcomes over the past ten years. These findings indicate that SL@B-funded innovations in 2019 will exit the SL@B program into a vastly different funding ecosystem than those funded in earlier rounds. These changes may increase the number of potential follow-on funding partners who can take up successful SL@B innovations in markets like India and Kenya. *Pitchbook is a platform that gives access to comprehensive private & public capital market data. “There’s also a very important recognition that local organizations require more hand-holding, more intense management. Sometimes these organizations may not have the institutional and financial wherewithal. They may, but we have had experiences where it’s been more of a challenge.” – SL@B funding partner 25 SL@B differentiators Distinct portfolio by innovation type and stage of growth As noted earlier, the SL@B portfolio is distinct from the wider funding landscape in both type of innovation and stage of growth. Compared to the study sample of 227 funded MNCH programs across 32 other funders, the SL@B program is more likely to fund product innovations (such as diagnostics and devices) and far less likely to fund practice/approach or health system support innovations. In addition to a concentration of product innovations in the SL@B portfolio, the program appears to fund a greater proportion of early-stage innovations than other funders. This analysis implies that SL@B may help fill a critical need in the ecosystem for early-stage innovation support (one of the gaps in the MNCH funding landscape noted above). However, it is important to note that SL@B’s focus on early-stage, product-based innovations originating within universities could lead to longer time-frames for scale and sustainability relative to other funders’ focus on innovations in health practice and health systems support. Health priorities The desk review of funded projects in this space demonstrates that, like SL@B, several funders focus on key MNH issues, such as sepsis, birth asphyxia, preterm birth, low birth weight, and postpartum hemorrhage. However, a substantial portion (nearly 70%) of the MNCH projects funded by non-SL@B funders included in this review focus on strengthening health infrastructure and integrated service delivery approaches (such as improving integrated maternal and newborn care). Compared with the broader field of MNCH funding, SL@B has a relatively smaller focus on infrastructure projects: practice and approach innovations (16%) and mHealth innovations (10%). Grantee support The SL@B program stands out from the wider field with the integration of non-financial support. Interview data indicate that other funders do provide some level of non-financial support, typically consisting of connections to potential investors and collaborators, technical assistance, in-house corporate expertise, and consulting. SL@B’s accelerator program, however, stands out with its focus on MNH and the opportunity to provide early-stage scaling support. In interviews, grantees specifically mentioned the value of SL@B’s accelerator support and the focus on the scaling process. OPPORTUNITIES FOR PROGRAM IMPROVEMENT Even though the open call approach differentiates SL@B from other funders and promotes interdisciplinary collaborations, it may hamper implementation and scale of the innovations. By not having a pre-defined process or mechanism for involving LMIC-based stakeholders, such as national and sub-national governments in targeted agenda setting, SL@B faces challenges identifying what LMIC stakeholders perceive as the most viable innovations and those that respond to key MNH needs in contexts where innovations could have the most impact. This means that the portfolio is driven by supply-side rather than demand-side factors. Although key informants commend SL@B’s open call approach for facilitating a process that sources a range of ideas, over half stated that a more targeted approach (when it comes to defining the specific problem) would be a better sourcing strategy for SL@B. A targeted approach, such as having a call to address one specific problem or disease relevant to a particular setting, would enable SL@B to set priorities and fund innovations that could be more applicable on the ground in places with the greatest need. However, this approach would not necessarily target those innovations that are most likely to secure investment or sustainable funding long-term. “I think we should first have an assessment, then identify special critical areas, and then focus on those. Otherwise, even if we have innovations, we will end up without a significant impact after spending all that time and resources. There are areas which we really, really need immediate solutions for in newborn and maternal issues.” – LMIC MNH expert 26 RECOMMENDATIONS FOR SOURCING AND PORTFOLIO BUILDING 1. 2. SL@B could consider issuing targeted, demand-driven innovation calls, such as based on disease burdens and/or geographic priorities. To seed innovation that can tackle priority MNH needs in the areas of highest need, SL@B should explore the possibility of making its calls more targeted and demand-driven in the following ways: Acknowledging that SL@B’s open call intended to source innovations with no restrictions on geographic settings including LMICs, SL@B could consider a number of strategies to address the lack of LMIC-led innovations in its innovation portfolio. • SL@B could deeply engage key public and private sector LMIC stakeholders in priority countries to identify the greatest challenges faced on the ground, relevant national and sub-national priorities, and the contextual factors or constraints that are most critical to determining successful implementation and scaling. These inputs could help to inform targeted calls for innovations that better match the needs and priorities in the target markets, provide insights into the types of non-financial support that would be most meaningful for innovators, and also develop stronger communities of key stakeholders that could enable the development of more integrated, comprehensive solution sets with the highest potential to reach scale and sustainability. • SL@B could more proactively reach out to specific local universities, research institutions, and innovation intermediaries (e.g., incubators, accelerators) in target countries so that local researchers and innovators are made aware of the funding opportunity. • SL@B could engage with local bilateral donor missions to increase visibility of the program and attract local innovators and partners, taking more advantage of the capabilities and networks of all SL@B funding partners. • To involve more innovators from LMICs, SL@B could provide more targeted technical support during the application and selection phase. • SL@B could organize pre-proposal workshops targeting LMIC innovators where they can develop grant writing skills and get a better understanding of proposal development. • SL@B could hold regional convenings in target LMICs/regions similar to DevX (e.g. in East Africa) that are accessible to more LMIC innovators, increasing awareness of funding opportunities, technical support, and potential partnerships. Holding a platform like DevX in an LMIC context not only increases awareness, but also serves as a great networking platform that facilitates peer-to-peer learning, identifying mentors, and forging critical partnerships for growth. 27 SL@B’S ROLE IN BUILDING INNOVATOR CAPACITY AND ACCELERATING INNOVATION • Overall, the data from this evaluation indicate that SL@B has made significant achievements in both building the capacity of innovator teams and accelerating the trajectory of MNH innovations to market entry. • Diverse stakeholders value SL@B’s early-stage funding to establish proof-of-concept and the program’s intentional approach to progressively fund an innovation through growth stages in order to support a scaling pathway. • The SL@B program appears to accelerate progress through key milestones and growth stages: innovators on average moved through growth stages more quickly during their SL@B funding period than did innovators in the same growth stages either before or after their SL@B funding period. • Survey data show that innovators credit SL@B’s financial and grants management support (e.g. regular check-ins with SL@B program officers, capacity-building workshops, connections) with accelerating their trajectory. Specifically, innovators credit SL@B’s financial support with accelerating the achievement of several key milestones, including validation studies (90%), prototype/program development (89%), and usability studies (88%). • Interviews with innovators identified several non-financial supports that innovators credit with accelerating their growth, including the push to think about scaling plans early, having SL@B as a ‘champion’ to attract new funding/partners, introductions to commercialization and implementation partners via DevX, creating a peer network of innovators, and opportunities to participate in Xcelerator workshops and mentorship meetings. • The non-financial supports rated as most helpful by respondents to the SL@B innovator survey include the Xcelerator workshops, the DevX events, and pitch support. The supports rated as least sufficient include connections with the private sector and manuscript editing. KEY FINDINGS What did SL@B set out to achieve in this area? Building capacity of innovators and accelerating innovations to market is a goal of the SL@B program. The SL@B TOC defines acceleration of innovations as rapid scale up of effective ideas/interventions, leading to safe and healthy pregnancy and birth for every woman and her child, especially in hard-to-reach communities. The SL@B Impact Area Framework* [see Appendix II] further defines this as bringing innovations to scale faster (reducing the time to scale), further (broader reach or level of scale), and/or more equitably (including marginalized populations). Building the capacity of innovators to diagnose and navigate challenges along the scaling trajectory is an important part of the SL@B acceleration strategy. The SL@B Impact Area Framework defines successful innovator capacity building as the development of innovators’ capacity across core skill sets (e.g. business model, manufacturing, or supply chain knowledge) and how that is translated into broader learnings for the field (e.g. accelerators or shared knowledge products). As context, it is important to note that there is very little data available in the literature regarding scaling pathways, milestones, and timelines, and much of what does exist relevant to LMICs is based on qualitative case studies. What can be gleaned from the existing literature is that the average time from initial development of an idea to market entry and then scale differs by type of innovation and target market. Looking primarily at high-income markets, drug development takes an average of 24 years from initial discovery to demonstration of effectiveness.19 Clinical practice innovations take an average of 17 years from the first publication of evidence to wide adoption.20 Medical devices have the shortest timeline, with an average eight years from initial conception to regulatory approval.21 The literature also indicates that these timelines are longer for innovations entering LMICs, for those targeting rural populations, and for those trying to scale through public health systems. 3 * USAID leadership shared the SL@B Impact Framework listing Key Dimensions of Interest, in Powerpoint via email communication, with the ESL@B team in January 2019 (mid evaluation time) which was different from the SL@B TOC. 28 Primary challenges faced by innovators Analysis of data in awardee reports indicates that challenges differ by stage of growth and type of organization, though several are consistent across all innovators. Table 4 lists cross-cutting challenges, as well as issues specific to type of organization or type of award. There are no suitable comparison or benchmark data available in the literature for the SL@B innovation portfolio on timeline, scaling pathway, or milestones. Given the lack of robust data in this area, the ESL@B team has worked to answer questions related to this impact area with milestones targeted by the SL@B program and the self-attribution of innovators. Table 4: Innovator Challenges, by Type of Organization and Type of Award (n=55) Source: Analysis by the ESL@B team using data from 55 awardee reports submitted to USAID and GCC. CHALLENGE TYPE OF ORGANIZATION TYPE OF AWARD University Non-profit For-profit Seed Validation TTS Lack of funding to support innovation growth X X X X X X Product development and technical issues X X X X X Recruiting and retaining staff X X X X Feasibility and acceptability of innovation X X X X X Local implementation country issues Insufficient infrastructure or materials in target country X X X Country instability X X X Logistics related to trial or implementation X X X X Delays in regulatory approval X X X Lack of interest in take-up in target country X X Has SL@B accelerated the achievement of milestones or growth? Given the focus on early growth stages in the SL@B program, many SL@B innovators are not yet scaling but are still working towards market entry. Of respondents to the SL@B innovator survey conducted by Duke University in 2019, 28% are operating in at least one market and 15% are operating in at least two markets globally. To account for the relatively early nature of SL@B-funded innovations, the ESL@B team primarily relies on indicators of growth stage and achieve￾ment of milestones, rather than scale (e.g. number of products sold or patients reached), as measures of growth. Innovator trajectories before, during and after SL@B participation The quantitative survey administered in July 2019 was designed to measure progress through growth stages and scale, and assess challenges, and successes accomplished before, during, and after their grant period. Administered to 113 innovators, the survey yielded an overall 54% response rate (n=61). Based on the data, SL@B innovators on average spent 54 months in the development stage, 46 months in the validation stage, 53 months in early adoption stage, and 47 months in the scaling stage. Given reaching and remaining in the scaling stage is the final goal for many of the innovators, this latter number reflects how many months, on average, the teams have been working on activities in this stage up to July 2019, rather than number of months spent in this stage before moving on. There was wide variation across the portfolio within all four growth stages. Table 5: Number of Months in Each Growth Stage Source: SL@B innovator survey developed and administered by the ESL@B team *Responses in this category reflect how many months awardees have been in the scaling growth stage up to the time of the survey (Jul 2019), rather than how long it took to complete it. GROWTH STAGE MEAN, MONTHS [YEARS] MIN, MONTHS MAX, MONTHS [YEARS] Development (n=50) 54 [4.5 yrs] 1 116 [24 yrs] Validation (n=39) 46 [3.8 yrs] 1 72 [6 yrs] Early Adoption (n=18) 53 [4.4 yrs] 2 120 [10 yrs] Scaling (n=19)* 47 [3.9 yrs] 1 84 [7 yrs] 29 While there is no comparison group available for this analysis, the survey data allow comparison of innovators who worked on activities within each growth stage during their SL@B funding period with those who worked on those growth stage activities outside of their SL@B funding period. The findings presented in Figure 8 below compare innovators reporting at least 75% of activities within any given growth stage taking place during SL@B funding with those reporting less than 75% of the activities taking place during SL@B funding. Results indicate that SL@B funding helps to accelerate progress through growth stages overall. (This analysis does not include the “introduce and scale” stage, as the goal for most innovators is to reach that stage, rather than progress out of it.) The average time taken to complete activities and time spent within growth stages was shorter for innovators who reported that more than 75% of those activities took place during the funding period compared with those who reported that less than 75% of those activities were conducted during the SL@B funding period. Innovators conducting activities within the SL@B funding period completed them on average more quickly than those conducting activities outside the SL@B funding period. Source: SL@B program information provided by USAID and GCC Figure 8: Total Number of Months Spent in Each Stage Across Varying Levels of SL@B Support Source: SL@B innovator survey developed and administered by the ESL@B team SL@B’s Financial and Grants Management Support The SL@B program awards financial support in three categories, as detailed below in Table 6. Table 6: SL@B Funding Level, by Grant Type SL@B AWARD TYPE AMOUNT OF FINANCIAL SUPPORT # OF AWARDS (ROUNDS 1-8) Seed Up to $250k for up to 2 years 93 Validation Up to $250k for up to 2 years 28 Transition to Scale (TTS) Up to $2 million for up to 4 years 26 >75% of activities conducted during SL@B funding period <75% of activities conducted during SL@B funding period 30 Source: SL@B innovator survey developed and administered by the ESL@B team Note: Table includes only those activities for which the number of respondents was 15 or higher. Number of respondents for funding and grants management are provided in that order. Overall, respondents to the innovator survey credited SL@B’s financial support with accelerating their achievement of certain activities and milestones. In addition, they also found grants management support (e.g. regular check-ins with SL@B program officers, non-financial support) valuable to accelerate their trajectory. Table 7 presents the percent of innovators reporting that participation in SL@B accelerated their progress in achieving key milestones. Table 7: Stage of Innovators Reporting SL@B Accelerated their Achievement, by Activity ACTIVITY/MILESTONE PROGRESS ACCELERATED BY SL@B # OF AWARDS Funding Grants management Problem identification (n=30, 28 respectively) 73% 61% Prototype/program development (n=38, 36) 89% 61% Preclinical development (n=17, 16) 59% 50% In-country IRB approval (n=15, NA) 67% NA Usability study (n=17, 17) 88% 65% Feasibility study (n=20, 18) 80% 56% Validation study (n=18, 17) 94% 47% Dissemination of results at international meetings/ conferences (n=16, 16) 56% 38% Dissemination of results at meetings/conferences held in western countries (n=18, 18) 61% 39% All innovators interviewed for the Acceleration to Impact research brief who were asked about whether participation in SL@B accelerated their timeline to market entry and scale, stated that it did. Data from these qualitative interviews (n=11) indicate that the SL@B program accelerated their trajectory through several key features. • SL@B’s application process initiated early thinking about scaling plans and partners. • The Xcelerator workshops and mentorship increased their understanding of scaling pathways and preparation for the scaling process. • Many innovators found commercialization and implementation partners through direct introductions from SL@B funders and through DevX. Participating in DevX also led to important peer networks. • SL@B’s role in championing projects helps innovators attract additional attention and support, including from other funders. • The evidence that innovator teams were able to build through their SL@B grant allowed them to secure other funding and strategic partnerships. While several innovators in this interview sample noted that they probably would have gotten to market eventually, they believe that SL@B funding sped up their trajectory. Two innovators in the sample of eleven stated that their project would not exist without SL@B funding. The case study of BEMPU Health’s TempWatch (below) illustrates how SL@B’s multi-stage funding helped accelerate an innovation’s pathway to scale. 31 Bempu Health is a technology company based in Bangalore, India that develops life-saving products for newborns and children in LMICs.22 The company has two products that are funded by SL@B, Apneboot and TempWatch.This case study features the TempWatch, an electronic bracelet that continuously measures a newborns’ body temperature, and alerts a caregiver/ health care professional in case of hypothermia (body temperature below 36.5 °C). For this innovation, Bempu received both the SL@B validation grant as well as the TTS grant. Hypothermia is a major cause of newborn mortality, especially for underweight newborns in India and other LMICs.23 Early identification of hypothermia and subsequent treatment through Kangaroo Mother Care (KMC) (i.e. skin-to-skin contact between a parent and a newborn) is an evidence-based hypothermia management solution, particularly in LMICs.24 For KMC to be delivered in a timely manner, a newborn’s temperature requires frequent monitoring, especially in the first month of life.24 More often than not, health care providers in public and private health facilities in India are overworked and unable to regularly monitor a newborn’s body temperature.25 This gap in care could leave hypothermia cases unidentified, thus increasing the risk of newborn mortality due to lack of timely intervention.25 The Bempu TempWatch addresses this gap through an audiovisual alarm that continuously keeps track of a newborn’s temperature. It beeps and flashes an orange light in case of hypothermia to alert caregivers/ health care providers for a timely intervention, thus reducing risk of newborn mortality. During the SL@B validation grant (2016 – 2018), Bempu gathered pilot data to validate the accuracy and feasibility of the TempWatch device in LMICs. Bempu reached close to 2,000 newborns through the study related data collection and sold approximately 8,000 bracelets, also managing to secure a procurement request of an additional 3,500 bracelets by the Indian government.*,25 During the SL@B TTS grant (2018-2021), Bempu is scaling the TempWatch with an aim of reaching 40,000 babies in India by the end of the grant.* Some challenges Bempu experienced include working with partners such as the government and communicating the value of a new innovation to increase acceptability by health care providers and the community. SL@B’s support has been crucial to Bempu as the TempWatch progresses through the different innovation growth stages. It accelerated the validation process and helped facilitate identification of manufacturing partners and target market entry. † Bempu indicated that they received “highly helpful” support from SL@B, particularly in acceleration support including VentureWell and DevX, facilitation of connections to the public sector and potential non￾funding partners, highlighting the Bempu innovation in communications pieces (blogs, briefs, social media) as well as nominating and/or connecting Bempu for speaking / conference opportunities.† Having forums like DevX led Bempu to meet key partners at GCC, USAID, BMGF, UNICEF, and others who then opened doors for them that they would not have otherwise had.* Overall, SL@B’s support helped them increase access to the MNH community and navigate challenges through the customized accelerated support. The CEA conducted for Bempu by the ESL@B team determined that Bempu is a cost-effective innovation based on the WHO threshold for cost-effectiveness.‡ CASE STUDY 1: BEMPU Health, TempWatch * BEMPU SL@B Final Report submitted to funders, 2018, Bempu’s team communication with the E-SL@B team, 2020 † Evaluation of SL@B survey completed by BEMPU, 2019 ‡ Dixit, S., Fernholz, F., Ogbuoji, O., Shahid, M., Biru, B., Udayakumar, K., & Baumgartner, J. (2020). Cost Effectiveness Analysis: Bempu TempWatch. Durham: Evaluating Savings Lives at Birth (E-SL@B) Program; Duke Global Health Institute Evidence Lab & Global Health Innovation Center at Duke University SL@B’s staged-funding approach allowed Bempu to receive two rounds of grants that were pivotal in moving the innovation through validation and scale-up phases. Photo credit: https://www.BEMPU.com 32 In addition to the level of funding and multi-year funding duration of the SL@B program (with awards ranging 2-4 years), innovators noted the importance of funding for operations and organizational development. SL@B awards allowed organizations to develop infrastructure to support the core innovation, an important differentiator from some other types of grant funding that only allow specific types of program activities. Particularly for innovators without strong existing institutional support (e.g. start-ups), funding to cover operating costs and recruit a team is as critical as funding for direct project costs. For some innovators, SL@B was the first funding for their innovation and launched the project. Receipt of the SL@B award validated innovators’ projects internally (with their institutional leadership) and externally (with potential partners and other funders), allowing them to attract additional resources. As noted earlier, SL@B’s open call for funding also provides a “way in” for innovators who have not previously worked in the MNH or development space. Innovators interviewed for this evaluation also noted that SL@B funding allowed them to test their idea and build evidence that it worked. With the funding to test, iterate, and validate their innovation and develop stronger evidence, innovators were then able to make progress on scaling, such as forming distribution partnerships and securing additional funding. The below case study of Gradian Health Systems’ Universal Anaesthesia Machine (UAM) illustrates how one SL@B grantee was able to leverage SL@B’s funding to move their innovation along the path to scale, go to market with a much more integrated package that includes UAM training, and secure additional funding. “It was really definitely very catalytic early on, when we got our first SL@B funding at DevX. It really was our… first foray into this whole global health community.” – SL@B innovator (drug) “We would not have done this if it hadn’t been for SL@B... This [innovation] has become a foundational part of our strategy as we’re working with the public sector. We wouldn’t have gotten this kind of evidence under our belt.” – SL@B innovator (practice and approach) 33 Gradian Health Systems is a non-profit technology company that develops and distributes medical equipment as well as provides product and clinical training in resource-limited settings across the world. Gradian received a SL@B TTS grant to improve surgical and obstetric care in Zambia by scaling their CE￾certified Universal Anaesthesia Machine (UAM) and pairing it, for the first time, with a simulation-based course in 2017.26,27 During obstetric cases such as C-sections, hysterectomies, and myomectomies, reliable anesthesia machines and well-trained health care providers ensure safe deliveries. However, most women in LMICs lack this access.* Evidence shows that, in these settings, ~75% of the medical equipment are donated and a subsequent 40% of them are not used due to lack of local expertise in operating the machines, and/or access to oxygen supplies, electricity, and spare parts.28-31 The UAM, paired with Gradian’s capacity building model, addresses many of these surgical care limitations. The UAM has a built-in oxygen concentrator that generates its own medical-grade oxygen from room air (without external oxygen source) and comes with a three-year preventative and corrective maintenance warranty with local bio-medical technicians and spare parts, as well as on-site and online simulation-based training for anesthesia providers delivered by local trainers.32 Despite challenges experienced related to installation sites and infrastructure in Zambia, Gradian installed the UAM in 32 hospitals and trained more than 70 anesthesia providers during the SL@B funding period.* As a result, Gradian is beginning to see this model generate public and private sector demand for anesthesia care capacity, the UAM, and Gradian’s simulation-based training in alignment with Zambia’s first-ever National Surgical, Obstetric, and Anaesthesia Strategic Plan.* The achievement during the SL@B grant period validated the combined UAM and training package and helped to catalyze scale to other countries in East Africa.* Furthermore, in December 2019, Gradian won a World Bank Global Financing Facility (GFF) award (approximately $4 million USD) to reduce maternal and newborn mortality.33 The GFF award will support the UAM scale up in Tanzania and Sierra Leone, which is estimated to increase access to safe anesthesia deliveries and surgeries by 200,000 per year.33 SL@B gave Gradian and their local partners the opportunity to field test and refine a simulation￾based product training curriculum and methodology that has revolutionized Gradian’s model and increased their potential for impact.* The funding, networking opportunities, and the tailored support received through the program catalyzed opportunities for scaling this model within Zambia and across other countries, such as Tanzania and Uganda.* The following support from SL@B were highly helpful to Gradian to move the UAM innovation along: increasing understanding of the market Gradian is targeting or working in, providing reference letters, nominating and/or connecting Gradian for speaking / conference opportunities (e.g. Grand Challenges Meeting, Unite for Sight, and others), pitch support from professional coaches (e.g. preparation for the DevX pitch competition), highlighting UAM in communications pieces (blogs, briefs, social media) and promoting through media channels).* Gradian indicated that SL@B’s support could have been strengthened in facilitation of connections to potential partners for scale (non-funding) and technical advising provided by USAID mission(s).* CASE STUDY 2: Gradian Health Systems—the Universal Anaesthesia Machine (UAM) SL@B gave Gradian a model for simulation￾based anesthesia training and helped leverage follow-on funding that has catalyzed expansion of the UAM and training programs throughout East, Central, Southern, and West Africa* Photo credit: https://www.gradianhealth.org/ our-products/uam/ * Gradian Health Systems information derived from their SL@B Concept Note, 2017; SL@B Mid-term Report; E-SL@B Quantitative Survey, 2019; other dissemination documents submitted to funders between 2017-2019; communication with E-SL@B team, 2020 34 Non-Financial Support The level and types of non-financial support offered by the SL@B program sets it apart from other MNH funders. This was frequently mentioned by SL@B-funded innovators as a critical factor in their successes. The non-financial support includes accelerator support, targeted connections to potential partners, and technical assistance. Innovators interviewed for the Acceleration to Impact brief noted that the SL@B program helped them to focus on early preparation for scale and make connections with partners, funders, mentors, and peer networks. Respondents to the SL@B innovator survey were asked to rank the helpfulness of the support they received from SL@B within a number of technical assistance categories. Figure 9 shows those areas that received the highest proportion of “high” helpfulness rankings. More detailed analysis of survey data on how helpful and how sufficient innovators rated many aspects of the non￾financial support provided by SL@B can be found in Appendix III. Nearly all innovators interviewed for the Acceleration to Impact brief (n=11) mentioned that the DevX event was pivotal. Through capacity building sessions and networking, innovators had the opportunity to make important connections with mentors, experts, and funders, especially as they worked to scale. Seventy-seven percent of respondents to the innovator survey reported that they had participated in DevX at least once. Of those who have attended, 60% found the forum highly helpful, while 30% found it moderately helpful. Connections with peers (primarily gained through DevX) is frequently mentioned by innovator interviewees as one of the most important benefits of participating in the SL@B program. Innovators also mentioned other peer networks and accelerator programs as being helpful though, for most, this was secondary to the peer networks developed through SL@B. Source: SL@B innovator survey developed and administered by Duke University Note: Total number of survey respondents is 61. The number of innovators who reported receiving each type of support is noted in parentheses. Chart shows how many, of the innovators who report receiving each type of support, ranked its helpfulness as high. Figure 9: Non-Financial SL@B Support, by Percent of Innovators Reporting it as Highly Helpful 35 Analysis of innovator interviews indicates several activities and milestones for which SL@B’s non-financial support was particularly helpful. • Identifying target markets – the program’s focus on early preparation for scale and understanding the target market and end use helped innovators select markets and prepare for market entry. • Finding external champions – innovators developed relationships through the SL@B network that helped facilitate the growth of their innovation. • Identifying commercial, manufacturing, distribution, and implementation partners – many innovators found commercialization and implementation partners through DevX. Others met partners through direct introductions from their grant manager. • Iterations in model and target market(s) – mentorship from the grant managers and also the technical mentors associated with the SL@B program helped innovators to iterate throughout. Innovators noted that this support often extended beyond the grant period. OPPORTUNITIES FOR PROGRAM IMPROVEMENT The analysis presented in this report demonstrates that many aspects of the SL@B program have been beneficial in accelerating the trajectory of innovations, building innovator capacity, and helping teams overcome challenges. However, data from interviews, as well as the innovator survey, indicate that there are also possible program improvements that may increase SL@B’s impact in this area. Analysis of the SL@B innovator survey indicates that there are a number of areas of non-financial support that innovators found insufficient. Figure 10: Non-Financial SL@B Support, by Percent of Innovators Reporting it as Insufficient Source: SL@B innovator survey developed and administered by Duke University Note: Total number of survey respondents is 61. The number of innovators who reported receiving each type of support is noted in parentheses. Chart shows how many, of the innovators who report receiving each type of support, ranked it as insufficient. In particular, connections with the private sector emerged as an area where support could be strengthened. These data also indicate that support with manuscripts, and support from USAID interns and local USAID missions are areas where innovators have found the existing support to be insufficient. 36 RECOMMENDATIONS 1. Continue to recognize the importance of non-financial support. This support is critical to acceleration and highly valued by awardees. It also sets SL@B apart in the field as few funding mechanisms have integrated robust non-financial support in the way that the SL@B program has. 2. Maintain the intentional approach to multi-stage funding and recognition of the differing needs and timelines of innovations at each stage. Consider additional nuance for each stage and type of innovation. Setting specific success metrics for innovations at each stage (perhaps varied by type), offering supports unique to each stage, and moving successful innovations through to scale could increase program impact in this area. 3. Capture better data prospectively on each innovation over time so that there is an increased ability to benchmark and measure pathways as well as timelines for milestones. 4. Help innovators test the most critical assumptions and market viability as early as possible and iterate as needed, to improve efficiency of the support provided (fail fast mentality). 5. Provide innovators with a better understanding of the market entry process, manufacturer/distributer needs, commercialization, risk factors, and communication skills as early as possible. 6. Increase in-country support for target markets, to help innovators better understand the context, get real￾time insights on the ecosystem, and build local networks. This may include ecosystem building efforts and/or engagement of private- and public-sector players in LMICs throughout the program cycle. 7. Manage expectations among innovators, external partners, and internal stakeholders about the timelines, success rates, and potential for sustainability for innovations trying to scale in LMICs, particularly those embedded within existing health systems. For example, if the goal of the SL@B program is to save lives in the near term, there is a risk that the SL@B program can set itself up to look like it has not done enough because lives saved is not a realistic metric within three to five years for an early-stage innovation. Building success metrics and risk tolerance guidelines grounded in realistic understandings of timelines and failure rates may help create more alignment among all stakeholders. 8. Recognize the continued gap between exit from the SL@B program and actual scale. The SL@B program currently serves innovations up to a certain point but even successful TTS innovations will not be fully scaled at the end of their SL@B funding period. The SL@B program could address this in several ways, including developing an additional award category for post-TTS (i.e., scaling), building strong ecosystem partners in LMIC markets to take on innovations after exiting SL@B, and/or proactively facilitating hand-offs to other funders. Implementation programs supported by SL@B funding partners (including USAID, GCC, and BMGF) could also become natural handoff partners. (This recommendation is also reflected in the section on partnerships, below.) First, it is important to note that there are several ways in which SL@B support has been instrumental in accelerating the trajectory of MNH innovations and building the capacity of innovator teams. Based on analysis presented in this report, the ESL@B team recommends that the SL@B program maintains these programmatic elements. In addition, the findings presented here suggest several ways in which the SL@B program could improve its ability to build innovator capacity and accelerate innovations. These recommendations include: Innovator interview data indicate other areas where the SL@B program could better support capacity building and the acceleration of innovations. • Although innovators highly value the capacity building that SL@B provides, innovators interviewed for this analysis who received seed awards noted that earlier training in the commercialization process and pitfalls, business planning, and testing market viability would help them assess their needs and be better prepared for partnerships. • Increasing the program’s emphasis on support for market research and the importance of being prepared to shift locations if needed could save innovators critical time during the proof-of-concept phase. • Practice/approach innovations experience some important differences in the scaling trajectory but the acceleration support is largely tailored to product/device innovations. • Connections with key stakeholders in the target LMIC markets, including USAID missions, LMIC governments, and private sector, are underdeveloped, which hampers innovators’ ability to gain traction. Overall, the program would benefit from more and stronger in-country resources and networks. 37 SL@B-FACILITATED PARTNERSHIPS FOR INCREASED INNOVATION IMPACT 4 • Overall, innovators have benefited significantly from DevX via the convening of leading MNH ecosystem stakeholders, increasing their visibility, and fostering connections with key stakeholders, including other innovators. • Innovators highly value the capacity-building and technical assistance they receive at DevX to make pitch decks and business proposals, which are critical to engage with any potential partner. • Across the SL@B program, there is an opportunity for even more proactive support for potential partnership identification and development, including with private sector organizations, national and sub-national public sector health organizations, and organizations with local expertise. KEY FINDINGS What are the partnerships that SL@B grantees have formed to move their innovation along the path to scale and sustainability? Using data from milestone and final reports, the ESL@B team identified partnerships that SL@B innovators reported creating to make progress on their innovations. SL@B innovators across innovation types and growth stages have pursued a variety of partnerships with stakeholders in HICs and LMICs. Innovators’ responses to the online survey indicate that the scaling pathway of most innovations involves partners, whether through a multi￾stakeholder partnership (61%), inclusion as a policy recommendation (31%), or government procurement of the innovation (52%). Overall, 92% or 107 out of 116 unique SL@B innovations have established at least one partnership in the public, private, or academic/research sector. Of these, two-third, or 72 unique innovations (67%) have established one or more partnerships with public sector stakeholders to pilot, adopt, or integrate their innovation into the health system. Over half of the 72 innovations have established partnerships with the Ministry of Health (MoH) (n=58), which is significant, given that adoption or integration of an innovation within health systems is a key scaling strategy, with the potential to serve hard-to-reach populations as well as large numbers (see Figure 11). What did SL@B set out to achieve in these areas? In the current era of Sustainable Development Goals, the role of global partnerships to advance global health equity and address health challenges is well-known. Increasingly, it is acknowledged that no one stakeholder or sector can fully address any particular global health challenge. Across both the SL@B Impact Framework and the SL@B TOC is the recognition that an innovation’s pathway to scale and sustainability involves partnerships with the public and private sectors to create lasting impact. In SL@B’s call for proposals, the program emphasizes that “strong collaborations and commitments will enable and deploy more effective and sustainable solutions.”* A key pillar of the SL@B Impact Area Framework has been the “development of partnerships that bring together diverse competencies, including from private sector (with co-funding) and government partners” to harness scientific/technological, social, and business ingenuity, and to catalyze successful implementation at sustained scale.”† The SL@B TOC describes the facilitation of partnerships as the connections that are made both by the SL@B program, and by the innovators themselves. The TOC also highlights DevX as a platform for matchmaking, “both within the innovator community, and between innovators and potential collaborators and/or mentors.”1 To understand SL@B’s role in facilitating and fostering key connections for its innovators, the ESL@B team conducted a series of key informant interviews to gather data on the partnerships they have established to develop and scale their innovation. The team also analyzed innovators’ progress and milestone reports received from USAID and GCC. In addition, the team analyzed responses from innovators on the SL@B innovator survey (see data and methods section). * SL@B Call for Proposals, Round 6,7, and 8 † SL@B Impact Framework Listing Key Dimensions of Interest, in Powerpoint shared by USAID with ESL@B team in January 2019 38 SL@B innovations have also been actively engaged in partnerships with other key government stakeholders, including national regulatory authorities, sub-national health authorities, and hospitals and facilities. Many innovations hold partnerships with two or more of these stakeholder groups. For example, Massachusetts General Hospital (MGH), in their implementation of an evidence-based training package on the administration and monitoring of ketamine for essential surgeries when no anesthesiologist is available, established partnerships with county and national health authorities, professional societies, and public and private facilities in Kenya. MGH works closely with the in-country implementing partner and county health authorities to identify facilities that are best suited to support the implementation of the ketamine training package. Thus, partnerships have been required at multiple levels to achieve the end goal of program implementation in a sustainable manner. BEMPU, in scaling up distribution of their hypothermia alert bracelets engaged with multiple levels of state and central regulatory authorities as well as procurement bodies prior to procurement. They also had to work with numerous individuals and other key stakeholders to ensure that their product was approved and could be used within facilities in India. Nearly 60% (n=69) of SL@B-funded innovations have one or more partnerships with a private sector entity. Across the portfolio, SL@B innovations have been successful in forming private sector partnerships with stakeholders in both HICs and LMICs, and with for-profits and non-profits (see Figure 12). For-profit partnerships include pharmaceutical and device manufacturers, supply chain organizations and distributors, and organizations that provide technical and R&D and corporate support. Non-profit partnerships that SL@B innovations have established have a similar representation as for-profits, but also include a significant number of local implementing partners. For example, BEMPU sells their hypothermia alert bracelets directly, and through distribution partners in India and LMICs. INMED, a global humanitarian development organization, has partnered with Little Sparrows Technologies, a HIC social for-profit start￾up, Brigham and Women’s Hospital, and Bilimetrix, an Italian-based start-up spun off from an academic institution to deliver the Bili-kit in low-resource￾settings in Peru. Bili-kit is a suite of technologies to screen, diagnose, and treat neonatal jaundice. Another variation of for-profit and non-profit partnerships is that of Mbarara University, an LMIC academic research institution and their innovation for an augmented infant air resuscitator. Mbarara University partnered with Philips, a HIC multinational technology company to produce the device, but also partnered with local non-profits for thought leadership and facilitation of local networking. Figure 11. Number of SL@B Innovations with Public Sector Partnerships Source: Progress and final reports from SL@B innovators received from USAID and GCC. Figure 12. Number of SL@B Innovations with Private Sector Partnerships Source: Progress and final reports from SL@B innovators received from USAID and GCC. 39 HOW DID SL@B HELP INNOVATORS MAKE CONNECTIONS AND ESTABLISH TRANSFORMATIVE PARTNERSHIPS FOR ITS INNOVATORS? Beyond point-solutions to reduce maternal and neonatal mortality are global health partnerships that are critical to transforming health and healthcare in LMICs. Transformative partnerships are based on the principle of respect towards both individuals with the health need and the community in which they are embedded, elevating critical health issues to the core of national agendas through their shared decision-making with multi-sectoral partners, and seeking to create sustainable impact.34-36 In this section, we summarize responses from innovators on the online survey on the partnerships support received from SL@B, as well as the voices of innovators from key informant interviews on the role that SL@B has played in facilitating transformative partnerships for them. We rely on innovators’ reports of these partnerships, but the descriptive analyses presented in this report does not examine whether and how these partnerships were transformative. Findings from Key Informant Interviews Innovators credit two noteworthy aspects of SL@B’s support which enhanced their ability to establish key partnerships. Value of DevX for Partnerships Broadly, the value of DevX for participants related to partnerships included initiating new partnerships, preparing for partnerships, and having the platform to engage with future partners. All innovators in the interview sample (n=18) found value at DevX to both make partnerships and to build their capacity to engage with future partners. Innovators emphasized that although these connections emerged organically through interactions at DevX with industry stakeholders, they highly valued DevX as a platform that brings such important stakeholders together. Key informants noted that DevX is a great opportunity for innovators to come together, network, and meet potential collaborators. Innovators initiated important conversations at DevX, both with industry stakeholders that would be instrumental in moving their innovation forward, and with other innovators to identify areas of collaboration, including exploring new geographies or diversified focus areas. Of the grantees who credited DevX for facilitating their partnerships, one was directly facilitated by SL@B, and another one was established at another conference, but was strengthened at DevX. Two grantees mentioned that their future partner organizations sought them out after hearing about their innovation at DevX, but did not meet them there. One grantee also formed a partnership with a mentor whom they met through DevX, after which they identified mutual areas of interest and potential collaboration, and eventually established their partnerships as peers. 1. “The connection with a pharmaceutical company was a consequence of our first SL@B grant. I think it speaks to the utility of SL@B beyond simply the funds that are provided to grantees. We received the first grant in 2011; it was at the early stages of our program and it was the same year we were given the Peer Choice Award at the Development Exchange in 2011. It was subsequent to that we actually had 2-3 large pharmaceutical companies approach us.” -SL@B innovator (drug) “As we were going through the SL@B process at the Development Exchange, representatives from X organization were there, and after SL@B, they contacted me and asked if we would consider working with them, sort of combining efforts. And so, we got together and talked to join forces.” - SL@B innovator (drug) “We started out as novices, you know in-country, we did not know what a business plan was, but now, we have all that. So, if I look at it from the human capacity, we have been able to develop as an organization, it’s significant… I have learned how to engage, how to package information, and how to make a business case. And that’s invaluable.” - SL@B innovator (device) 40 Direct Matchmaking Innovators valued the utility of SL@B providing targeted matchmaking support to connect them with potential collaborators. Partnerships facilitated through direct matching by SL@B are also mentioned as an important assumption in the TOC for innovations to scale sustainably. The SL@B program has also facilitated connections to other partners for its grantees to address other needs that were relevant to their growth stages. As an example of these partnerships, program partners connected a SL@B grantee with a researcher from a HIC, who helped the grantee design a risk assessment tool that was a critical component of their mHealth platform for maternal health. The tool helped the grantee secure additional funding based on its current and future potential to triage and treat at-risk pregnant women. Additionally, grantees consider the SL@B peer-review process that their innovation goes through as an important mechanism for vetting by potential collaborators which opens doors for them to engage with future partners and funders. Transformative Partnerships Established by SL@B innovators In the key informant interviews, SL@B innovators highlighted the critical junctures at which the partnerships they established with SL@B support made a significant difference to help their innovation advance to the next stage of growth. Based on interview data with the sample of SL@B innovators, Table 8 highlights illustrative key partnerships that SL@B innovators formed, which subsequently became turning points for the trajectories of their innovations. The ESL@B team also examined the acceleration trajectory of five SL@B innovators who were key informants to the partnerships analysis based on their responses to the online survey. Of note, findings suggest that SL@B innovators with key partnerships established such as those described above spent fewer months in each IDIA growth stage relative to the average number of months spent by all innovators who responded to the survey. Innovators who did not form partnerships at DevX also highly valued their participation in DevX and noted that it helped them network and engage with mentors, funders, and technical experts. The exposure through DevX to business models, marketing, and pitch decks, combined with targeted capacity building efforts, helped innovators talk to potential collaborators and funders as well as think more clearly about the partnerships that would be needed to roll out their interventions for scale up. It also provided them the opportunity to refine their pitch, showcase their innovation, describe progress made, and seek out collaborators for specific needs at particular stages. 2. 3. “Getting an award from SL@B is a rigorous peer reviewed and competitive process, and the fact that by the time we met [device manufacturer], we had gotten a seed award from SL@B already that we successfully implemented, and we were auditioning for a validation grant which they saw us ‘Win’ at that meeting, was very powerful. I think it helped some of them believe in the technology more because of all these people and peer review process that the technology was going through.” - SL@B innovator (device) “What has come up was more getting to meet technical experts who’ve been able to help us find solutions to the whole process of rolling out the intervention. So, getting to know people who come to DevXs, there are people we know who mentored us through this process, to get us to where we are.” - SL@B innovator (device) “What I came to learn was that [device manufacturer] had reached out to [SL@B Funder] to look into their portfolio of innovations and see which ones may be ready for private investment and partners from that. So when [SL@B Funder] learned that we were in talks with [device manufacturer], that was positive, and also when [device manufacturer] knew that [SL@B Funder] was supporting us through SL@B, that was another thing that I think facilitated the growth of this relationship, because then we had [SL@B Funder] that trusted us, [SL@B Funder] that trusted [device manufacturer] and [device manufacturer], which cultivated this growing relationship.”- SL@B innovator (device) “When you look at the SL@B community, you’ve got this whole ecosystem of innovators, all of whom might well have very different technologies that they’re developing but a lot of going through exactly the same challenges and how to learn to implement a product in these situations. So, that’s been a great source of connection, discussion, and good information.” - SL@B innovator (drug) 41 Table 8. Illustrative Key Partnerships Formed by SL@B Innovators KEY PARTNERSHIPS PARTNER SECTOR SL@B INNOVATORS’ REASONS FOR PARTNERSHIP SL@B INNOVATORS’ PARTNERS’ REASONS FOR PARTNERSHIP SL@B SUPPORT TOWARDS PARTNERSHIP Development partnership Private (academic, for-profit) Developer provides cost￾effective solution to address critical needs in innovator’s region or focus area Leverage innovator’s local, on-the-ground networks and connections - Initiation of partnership dialogue at DevX - Endorsement of innovation by SL@B program Manufacturing partnership Private (for-profit, NGO) Manufacturer obtains technology (open or restricted license) from innovator to commercialize innovation at scale Leverage innovator’s focused research and clinical expertise - Initiation of partnership dialogue at DevX - Direct matchmaking between innovator and manufacturer - Endorsement of innovation by SL@B program Government partnership Public - MoH provides regulatory approval to roll out innovation in country - MoH identifies districts to roll out innovation - Trust in innovator’s expertise - Evidence of innovation’s effectiveness - Alignment with MoH priorities Endorsement of innovation by SL@B program Distribution partnership Private (NGO and for-profit) and Public Distributor brings supply chain efficiencies in country of implementation Innovator provides clinically validated solution to address MNH challenges in country Initiation of partnership dialogue at DevX between SL@B innovator and International NGO Source: Key informant interviews with sample of SL@B innovators conducted by the ESL@B team. Findings from Innovator Survey Highlights from the innovator survey validate the findings from the key informant interviews that, unlike most funders, a key programmatic element of SL@B that grantees value is the facilitation of connections for its grantees. Of the innovators who responded to the survey, 56% (n=34) reported that SL@B facilitated connections with potential funders, 28% (n=17) reported that SL@B facilitated connections with public sector partners, and 23% (n=14) reported that SL@B facilitated connections with private sector partners. The survey also asked respondents to indicate whether the partnerships support they received from SL@B (if applicable), was sufficient or insufficient among other elements of technical assistance. Data indicating whether innovators felt that SL@B support in facili￾tating connections with various stakeholders was sufficient or insufficient is shown in Figure 13. Figure 13: SL@B Partnerships Support, by Percent of Innovators Reporting it as Sufficient or Insufficient Source: SL@B innovator survey developed and administered by ESL@B team. Note: Total number of survey respondents is 61. The number of innovators who reported SL@B-facilitated connections is noted in parentheses. Bars show the percent of innovators who ranked the helpfulness of each type of SL@B-facilitated connection as sufficient or insufficient. 42 Figure 13 above demonstrates that the support towards forming partnerships provided by SL@B program staff has been valuable to innovators to make connections with different stakeholders including potential funders, public sector partners, and other partners. However, the figure also shows areas where the program could more directly address innovator needs, particularly in facilitating connections with private sector partners. OPPORTUNITIES TO FACILITATE STRONGER PARTNERSHIPS IN THE ECOSYSTEM Innovators consider the SL@B program as more than a funding partner. They expressed that the program has been an important point of call to connect them with other innovators as well as with key stakeholders to learn, discuss, and grow. While innovators appreciated the training sessions and the opportunity to make connections at events, they felt that SL@B and similar funders in the ecosystem can play an even greater role in the support they provide to help innovators make targeted connections. Interviews with SL@B innovators revealed that while they highly value SL@B’s efforts in facilitating connections for them, they note that these connections are not consistently available for all promising SL@B innovations. Findings from the innovator interviews are consistent with the responses to the online survey about the extent to which SL@B support in facilitating connections with potential funders, partners, public and private sectors was helpful and sufficient. Data from interviews and innovators’ milestone reports also shed more detail on the kinds of connections innovators are seeking and what they perceive would be valuable in moving their innovations forward. Private Sector and Funding Partners. Innovators expressed that they needed significant additional funding, particularly after they have proven that their innovation is effective and their SL@B funding has ended to support their adoption, integration, and scaling efforts. SL@B and other programs could be more intentional and proactive in addressing this funding gap after SL@B’s exit from an innovation and provide targeted connections to funding and implementing partners at later stages of development to help take the promising innovations to the next stage on their scaling pathway. Public Sector. Several innovators reported that they have a critical need for connections with the public sector in different countries to adopt and scale their innovation. In particular, innovators with little prior history of engaging with government stakeholders could significantly benefit from the additional boost from large funders to make the right connections with national health authorities as well as national and sub-national governments. These engagements are complex because of the different levels of public sector partnerships as well as the variability across countries. Additionally, in some countries such as India, there is considerable heterogeneity between states in terms of priorities and ease of operations. While it is crucial to get regulatory approval from the central government in India, state governments ultimately lead efforts to adopt and scale an innovation. In some cases, gaining a foothold in larger countries through the public sector can open doors in other countries for roll out and adoption. According to one KI, having regulatory approval from Ghana, for example, is often an access point to reach other countries in West Africa, while approval from Kenya opens doors to other countries in East Africa. The case study of JSI’s chlorhexidine project (below) illustrates how one SL@B grantee found collaboration with a local government crucial to scale their innovation. “We are still seeking other donor partners to allow us to expand and fill in some of the financial gaps. Part of that is that we would have liked to see a greater proportion go to [developing partner] so that they could continue to do more of the technology development. But we really needed the majority of the [SL@B] grant just for the implementation piece. So, there is a little bit of a financial gap there.” - SL@B innovator (diagnostic) “There is another category of partners that in my mind is critical and those partners are partners that will be able to adopt and deploy. This could be ministries of health or health multinationals/ organizations, and that kind of partnership is what is not very strong right now. [SL@B funder] can write to ministries expressing their enthusiasm in the technologies that show promise and ask ministries how they [ministries] can further help innovators; request innovators to co-apply with ministries if the innovation has reached some degree of maturity requiring implementation studies. This could bring greater ministry involvement but will require innovators to engage ministries more.” – SL@B innovator (device) 43 With a mission to improve the health of underserved people and communities, JSI Research & Training Institute, Inc. (JSI) is a non-profit organization that has implemented more than 2,500 projects in 107 countries across the world, since its establishment in 1978.37 One of the projects that JSI led was the SL@B funded Chlorhexidine Navi (Cord) Care Program (CNCP) to address newborn sepsis/infection in Nepal. Infection accounted for almost 70% of newborn deaths in Nepal, mainly due to unclean practices during home births, where the umbilical cord is cut with sharp non-disinfected household objects and then covered with turmeric, mustard oil paste, or other traditional practices.*,38 In 2006, research conducted by Johns Hopkins University in Nepal showed Chlorhexidine’s (CHX) efficacy in reducing infections caused by bacterial contamination of the umbilical cord stump.39 JSI conducted pilot studies and collaborated closely with the Nepali government and a local pharmaceutical company to create a standard antiseptic gel formulation of the product and a sustainable method of distribution, primarily through Female Community Health Volunteers.* Based on promising results from the pilot studies, the government standardized CHX use as part of the national essential newborn care guidelines.40 In 2011, JSI received the SL@B TTS award to support the government’s plan of scaling up CHX across Nepal. 41, 42 JSI, in partnership with the government, successfully achieved nationwide coverage in Nepal by the end of the SL@B funding period, in 2017.40 A total of 2.1 million newborns have received CHX umbilical care, resulting in an estimated 9600 newborn deaths averted during the SL@B funding period.41 JSI’s program activities to achieve scale included incorporating CHX into the Nepal government’s MNH policy, integrating CHX within existing government MNH programs, coordinating with the government and private sector to secure continuous product supply, and appending CHX guidelines in the Nepal medical professionals’ training curriculum.† Despite CHX’s wide coverage, the Nepal national compliance survey (household and homebirths) revealed a 59% usage rate which indicates continued room for improvement.40 SL@B’s funding and technical assistance were pivotal in JSI’s CHX scale up in Nepal. JSI indicated that they received “highly helpful” support from SL@B in a number of areas, particularly through DevX, facilitation of connections to potential partners for scale (non-funding), support for publication in open access journal, highlighting CHX in communications pieces (blogs, briefs, social media) and promoting through media channels), and nominating and/or connecting innovators for speaking/ conference opportunities (e.g. Grand Challenges meeting, Unite for Sight, and others).‡ In 2013, WHO added CHX to the Essential Medicine List for Children (EMLc), largely based on JSI’s research findings from Nepal.43 As a result, Nepal has served as a model for others countries interested in scaling up CHX and emulating JSI’s CHX implementation strategy. During the SL@B period, JSI provided technical assistance, coordinated with the Global Chlorhexidine Working Group, and led study tours and learning visits for more than 200 visitors from 20 countries; Bangladesh, Ethiopia, and Nigeria to name a few.† Furthermore, JSI made its program data, monitoring and evaluation, training and advocacy tools publicly available to support CHX-adopting countries. In recognition of its global advocacy efforts, the program received the USAID Global Science and Technology Pioneer Prize in 2013.44 CASE STUDY 3: JSI Research & Training Institute, Inc. (JSI) -- Chlorhexidine * JSI SL@B Work Plan, 2011 † JSI SL@B Final Report, 2017 ‡ Evaluating SL@B Quantitative Survey completed by JSI, 2019 JSI’s scale up success can primarily be attributed to its close partnership with the Nepali Ministry of Health, building on existing systems using Female Community Health Volunteers, and reliable in-country production of CHX ensured sustainability. Photo credit: https://www.jsi.com/project/chlorhexidine-na￾vi-cord-care-program/ 44 Local Expertise. A significant proportion of SL@B innovators originate from HICs, thus it can be difficult for them to make local connections and identify strong organizations with the right capabilities and expertise to adopt and implement the project in target LMIC markets. Innovators need on-the￾ground support with organizations who have the capabilities to provide implementation support, as well as deep networks among the public sector, health care providers, and community of users. This is particularly helpful for key milestones such as getting regulatory approval, testing user acceptability, creating advocacy platforms, providing training support, and distributing their innovation. International Organizations. For both LMIC and HIC innovators, introductions and endorsements from their funders to major global health organizations can foster collaboration to address shared goals. Innovators expressed strong interest in being connected to global bodies such as the WHO, UNAIDS, and UNICEF, but do not have the opportunity to form these connections. The goals for connections with global organizations are three-fold: 1. Regulatory approval and/or inclusion in standards or guidelines issued by such organizations is usually well￾recognized around the world. 2. The capacity of PIOs to serve as global conveners for procurement of innovations could ensure that needed innovations are produced at scale. 3. Global organizations are well positioned to navigate political challenges and instability, helping local actors, particularly in the public sector, take responsibility for implementing and scaling. “If you rely purely on government to implement something like this, the chance of success is very low. For them to take up a new idea and implement it, it is very, very hard. So, if you have somebody like [large local implementing partner] coming to the government and saying – look, we will help you introduce the product, we will provide backup support. So, the importance of having a nonprofit organization, the need for a non-governmental or a voluntary organization to work hand in hand with the government at the national level is also critical in my opinion, based on my experience.” -SL@B innovator (nutrient) “If you take for example Global Fund, Global Fund is deeply working with political authority in countries and have them set up what they call a country coordination mechanism, but at a political level, it is very high. The requirement from donor is very high for the government to be more involved. But you know how governments are in Africa, political instability and also, if it is coming from one partner, one to three NGOs, they take their time but if they have this type of support coming from the top level with big donors, I feel like this Saving Lives at Birth partnership, I think that could help us to accelerate the process.” -SL@B innovator (mHealth/practice) “If I go to any country and I speak to health ministries, and I propose something, their first question is – do you have WHO recommendations of this, and if WHO recommendations are there, they will accept it as a very good valid validation and they’re much more confident to move. Otherwise, if you come up with some idea for which there is no WHO opinion, they are less keen to take/ adopt it.” -SL@B innovator (nutrient) Multi-stakeholder partnerships (private, public, and PIO) are crucial for innovators as they go through a scaling pathway. While SL@B recognized the need to foster a supportive community for innovators, key stakeholders in the MNH innovation ecosystem would need to address this gap as well, in providing targeted support to innovators consistently (e.g. with funding partners, commercial manufacturers, national and sub￾national authorities, exit strategies, PIO endorsements). 45 KEY RECOMMENDATIONS FOR MNH INNOVATION AND FUNDING COMMUNITY 1. 3. 2. Implement tailored strategies for innovators to facilitate targeted partnership connections, including with funders. The funding community should develop and refine the strategy used by SL@B to make targeted and proactive connections for innovators more consistently. In particular, DevX emerged as a valuable platform to convene public and private sector stakeholders and for innovators to network and form connections. In that vein, other funders in this space could consider holding similar convenings in LMICs where it is easier to engage and form meaningful connections with key local stakeholders. The funding community also needs to proactively coordinate engagement between innovators and other innovation funders to make targeted connections at different growth stages based on the profile and need of innovators. There are a number of funders that provide stage-specific funding; early-stage funders such as SL@B could use its position to convene funders of later-stage innovations and develop a coordinated pipeline of funders to scale innovations. Engage with procurement platforms to support procurement for validated innovations. MNH innovation funders are aptly poised to facilitate introductions to global platforms for procurement of innovations across different countries (e.g., UNICEF), and inclusion of validated innovations in the WHO list. Given the fragmented nature of several of the key causes of mortality across geographies, even when a clinically validated solution is available and is backed by a commercial scale manufacturer, it is not often possible to manufacture at scale if the intended market is only within one country. A program like SL@B can ensure stable production by facilitating global procurement of innovations into countries where they are needed. Proactively engage LMIC national and sub-national stakeholders early, jointly determine priorities, and curate validated innovations that can be scaled in different countries based on need and demand. Engagement with local public and private sector stakeholders, particularly in priority LMICs should be intentional and strategic elements in funders’ program design. Suggested ways for funders to incorporate this include: a. Engage ministries and key local stakeholders in an advisory function prior to sourcing to ensure that innovations meet the needs and priorities of stakeholders in target countries. b. Invite LMIC public and private sector stakeholders to attend forums like DevX and the Grand Challenges Annual Meetings (GCAM), held in LMICs, to meet innovators, learn about innovations, and assess potential collaboration opportunities. c. Convene an integrated marketplace for innovations at the country or regional level with country stewardship and ownership of the platform. A national or regional marketplace would provide an opportunity for countries to select the most impactful innovations for their context, and to integrate scaling efforts with local health and financing systems. Similar efforts currently exist, such as the EWEC Innovation Marketplace, which closely collaborated with SL@B; but these efforts need to be strengthened and extended to the MNH funding community more broadly. We draw on interviews and survey data to highlight three data-driven recommendations applicable not only to the SL@B program, but also to the broader funding community and innovation ecosystem. 46 QUANTIFYING THE IMPACT OF SL@B (lives saved, cost, and equity) In addressing the fourth evaluation question, “What is the potential impact of SL@B-funded innovations on MNH mortality and in the MNH ecosystem?”, the ESL@B team determined that this question was really about the effectiveness of an innovation (lives saved), whether it is or would be affordable per person (cost), and whether the innovation was reaching those at greatest risk of poor health outcomes (equity). These three issues can largely be answered at the innovation level per a rigorous CEA. For reasons discussed below, we were able to conduct CEAs for a subset of SL@B innovations to help illustrate this important aspect of SL@B’s impact. Selection Process To conduct a CEA, we needed to identify and select innovations that were advanced in their pathway to scale. These innovations fell under the TTS category, and were in a position to provide the data needed to complete a CEA – historical cost data along with cost projections for scaling until 2030. This required cost data was provided by innovators via an Excel-based costing tool developed by the ESL@B team as part of the evaluation.2 The other data source needed to complete a CEA was the lives-saved estimate from individual impact models originally developed by one of the SL@B partners, GCC, and validated by the ESL@B team. The ESL@B team has also created a new innovation modeling checklist to assist with these validation efforts (see Appendix IV). The checklist includes standardized elements and assumptions that needed to be included or addressed in an impact model, similar to other methods’ standard checklists such as COREQ for qualitative data and CONSORT for randomized controlled trials.46, 47 To select the innovations for CEA, our team looked into 19 out of the total 26 TTS innovations funded, and selected four innovations to conduct a CEA. The rest of the 14 innovations were not pursued due to the following reasons: innovators were not interested (and we needed their participation for cost data), the innovations did not have an impact model yet, and/or the innovators had already conducted their own economic analyses including published CEAs. Significance of CEA The CEAs produced through this evaluation are not included in this report but are standalone documents on each of the four chosen innovations. They provide quantitative evidence on the associated cost of scaling a certain innovation as well as determining if an innovation is deemed cost-effective based on the WHO thresholds for measuring cost￾effectiveness. This information can be used by funders, program implementers, governments or other stakeholders to make informed decision regarding funding/scaling an innovation. However, it is important to note that CEA is only one type of economic analysis that could inform program/funding decisions, and it may not represent a holistic view of every economic element around the innovation. Increasingly around the world, CEAs and efficiency considerations 5 • SL@B has sourced and scaled some cost-effective innovations (e.g. Bempu TempWatch); however, impact estimates for most grantees are premature given their earlier stage of growth (and evidence level) in the scaling pathway.* Longer timeframes are needed to assess mortality impact for the lifecycle of the SL@B￾funded innovations. • SL@B TTS grantees are very interested in having CEA of their innovations in order to leverage funding for scale. We suggest either providing technical assistance within the SL@B program to conduct CEAs for all TTS grantees or link grantees with outside partners who can help them complete a CEA. • A number of SL@B innovations have modeled positive health impact and calculated attractive economic parameters such as cost-effectiveness, which shows the future promise of the portfolio.45 KEY FINDINGS 47 Key Takeaways from CEA Process During our collaborative CEA process, the ESL@B team learned that most innovators had not planned and/or thought much about the expenses that would be incurred during the expansion of their project, which is vital for conducting a CEA and assessing the feasibility of scaling. However, gathering and documenting historical costs was feasible, albeit time-consuming, for most innovators. We recommend that donors encourage innovators to document their structure of costs in relation to the number of beneficiaries, think about future expenses, and consider doing CEAs when applicable. We also learned that CEAs could be a necessary next step for many innovators as they seek more funding or progress through their pathway to scale. CEA is an important analysis and a building block for evidence￾based decision making that innovators could leverage as they interact with government, funders, and other decision-making stakeholders in different settings and countries. Having said that, CEA is not the only way to determine whether an innovation is having an impact or not. Other tools such as the publically available EquityTool could be used to assess the reach and impact of an innovation.18 Even if an innovation is not yet deemed cost-effective, proving that it is reaching the most vulnerable could be another way of quantifying impact. During our evaluation process, we were able to do an EquityTool analysis for one innovation which the innovator found helpful to understand the wealth quintiles of their current beneficiaries. By way of doing that, we also learned that many innovations do not have a data-based system of monitoring economic status of their beneficiaries, a proxy for those most vulnerable to poor health outcomes. We recommend that donors encourage innovators to document their reach by known wealth quintiles (or other socio-economic status indicators) to ensure they are reaching beneficiaries who could most benefit from their innovation. * Dixit S, Fernholz F, Ogbuoji O, et al. Cost Effectiveness Analysis: Bempu TempWatch. Durham: Evaluating Saving Lives at Births (E-SL@B) Program; 2020. are combined with social, political and administrative parameters, making them one of the main pillars for resource allocation and decision making. Therefore, it is vital to understand how the cost of implementing an innovation in a county or a region fits in overall public health expenditure of that area. This might impact the feasibility of scaling up an innovation in a region even if the innovation is cost-effective according to the WHO thresholds. Furthermore, CEAs are a snapshot in time, and, therefore are subject to change if conditions for the innovator change down the line (e.g. if the innovator receives more funding or is not showing the expected impact). To this end, CEAs are best positioned to provide evidence on a particular innovation at a certain point in time, and not much insight on a portfolio level. The CEAs for the four SL@B innovations that are being conducted capture the impact of an innovation based on the current availability of data. If a certain innovator scales more than the current assumptions, and achieves higher impact, then the CEAs results will improve further. To model some of the future changes and uncertainties in innovators’ expansion plans, the CEAs include sensitivity analysis of cost and impact variables using methods such as Monte Carlo simulation (a method that was used to determine the range of variability in an innovation’s outcome for all variables/assumptions involved.) 48 CONCLUSION Results from this evaluation of SL@B from 2011 to 2020 show that as a multi-donor partnership seeking to grow and scale innovations, the program has helped to address critical gaps in the MNH landscape by funding early￾stage innovations that aim to address common causes of maternal and neonatal mortality, sourcing from a variety of fields to address key MNH needs, and building the capacity of innovators through a range of non￾financial support activities. For a recap of key findings and recommendations, please see the Executive Summary. DISCUSSION Throughout this report, we present findings and proposed recommendations linked directly to the data, research methods, and analyses undertaken in this evaluation program. In this section, we further expand upon our recommendations by leveraging the team’s broader expertise and experience in MNH, global health innovation, health economics, and evaluation methods. Thus, the recommendations in this discussion section represent only the views of the authors for further consideration by the SL@B program partners. They generally build from the key recommendations in each section and provide more specific guidance for future priorities and action. As a pioneering model for co-investment and explicit open innovation approaches to address critical challenges in MNH, the SL@B program should be commended for its forward-looking and inclusive design. The learnings and partnerships formed through SL@B have also supported and been incorporated into additional Grand Challenges for Development, including for Zika and Ebola. A vibrant Grand Challenges community now exists that includes and is made stronger by the SL@B community of innovators, funders, private sector partners, and other supporting organizations. The community that SL@B has brought together is critical to the future of MNH and is a valuable asset that has been cultivated and nurtured over eight years. Regardless of the future of SL@B, there should be a commitment to keeping the MNH innovation community actively engaged and growing over time. To maximize its impact and build on its differentiated strengths and capabilities in the future within the rapidly evolving global health innovation landscape, SL@B could be re-designed with a mandate to serve as a comprehensive MNH innovation platform, not primarily or predominantly as a more narrow grant funding mechanism with some add-on non-financial support. Such an innovation platform would include many of the components and approaches described below. Deep focus on “problem sourcing,” applying the principles of human-centered design to effectively describe, analyze, contextualize, and prioritize MNH challenges for innovations and innovators to target; this should be done in close partnership with and co-led by LMIC based organizations such as Grand Challenges Africa, African Academy of Sciences, regional bodies such as West African Health Organization (WAHO), and national policy-shaping entities such as Ministries of Health and Ministries of Science and Technology, NITI-Aayog (India), and Biotechnology Industry Research Assistance Council (BIRAC, Grand Challenges India). This curation and prioritization of MNH challenges suited for innovation, with strong local voices and context, would itself be an important service from SL@B for the broader MNH community, helping innovators and other key stakeholders make better decisions and more effectively target their potential solutions. We also see a significant opportunity to more effectively link SL@B-supported innovations, and SL@B as an innovator platform, into the broader efforts in MNH funding and implementation of SL@B funding partners. This could include operational models such as incorporating SL@B innovators directly into MNH implementation programs, creating specific funding priorities within implementation programs to integrate SL@B supported innovations, leveraging implementation partners to highlight and bring more visibility to SL@B-supported innovations, and holding local and regional LMIC-based innovation marketplaces (leveraging local and regional offices and missions of SL@B funding partners) linked to implementation projects that highlight responsive and promising SL@B innovations. 1. 2. 6 49 In addition to LMIC public sector partners, SL@B could deploy a network-of-networks organizing model to better engage the growing RMNCH community, and more precisely and effectively describe its primary role and value-add in the context of other groups, including the Partnership for Maternal, Newborn and Child Health (PMNCH), the Global Financing Facility (GFF), and Every Woman Every Child (EWEC). SL@B can especially ensure that the voices and perspectives of innovators and the MNH innovation community are fully represented in other networks, and make coordination and handoffs more effective over time. Furthermore, SL@B could strongly consider and target the inclusion of additional private sector partners, and donor country offices for future iterations. A stronger and more inclusive partnership would enable more successful identification, support, and scaling of innovations in MNH. It will enable SL@B to more effectively focus on integration and adaptation of successful point innovations to support more comprehensive solutions for complex health challenges. As the MNH field matures and the investment into innovation grows, SL@B has a significant opportunity to serve as the central coordinating platform for MNH innovation, bringing together public and private sector partners to more effectively communicate, collaborate, and accelerate impact. There are clear tensions internal to SL@B funding partners, and development organizations in general, about the right levels of investment between innovation and implementation of current best practices. Stronger and more proactive internal governance and communications efforts would potentially alleviate those tensions and highlight the potential synergies across the innovation to implementation spectrum. Explicit portfolio management approach for SL@B with multiple tracks. Building upon the key recommendations set out in this report for more clear and transparent targets for risk tolerance and expected success factors at different stages and tracks of innovation, we also propose that SL@B consider additional flexible opportunities for add-on funding, more integrated access to private sector funding, and explicit expected success rates for each tier. Key considerations for such a portfolio management approach include: a. Distinct tracks within the SL@B program (separate to the current breakdown by stages of development) would require different evaluation criteria and support mechanisms, and may include: i. innovations addressing the most intractable problems in the hardest places to reach and/or among the most vulnerable populations, where innovation is certainly needed but it is unlikely to lead to market-based, sustainable solutions; ii. curation and scaling of promising market-based solutions in target LMICs where conditions could support financial sustainability for such private-sector solutions; and iii. later-stage scaling of proven innovations, where there is a gap within SL@B but also across other MNH funding sources; larger scale funding and different types of non-financial support are needed for this stage of scaling, and the new Innovation-to-Scale initiative supported by the GFF in partnership with Laerdal Global Health is one example that could be built upon. b. Another aspect of portfolio management that is important to consider is to publicly acknowledge and quantify the expectation that not all investments will or should be successful. A traditional grant-making and global development aid approach attempts to support all grantees equally and expects incremental progress from each, while a more explicit venture philanthropy management approach across the portfolio might set targets for success and failure at various stages of financing and growth, and see the accelerated shut-down of unsuccessful ideas as a key marker of success rather than failure. While there is a lack of existing benchmarks for expected success rates for global health innovations, one can infer appropriate principles from other literature, including the venture capital and pharmaceutical industries. A common approach to examining startup failure is to examine 1) failure to raise the next round of capital (which for SL@B innovators could include philanthropy, donor aid, and/or private investment), or 2) failure to exit (which for SL@B innovators could include acquisition, licensing, and/or handoff of the innovation to an implementing partner or government).48 Data on startups in the US tracked from 2008 to 2010 show that nearly two-thirds of startups stagnate in the venture capital funnel as they fail to exit or raise the next round of funding.49 While the rate of failure to exit decreases as a startup progresses through financing stages, the rate of failure to raise the next round of financing is highest at the earliest (seed to Series A, 79.4% failure rate) and the final stages (Series G to Series H, 82.6% failure rate).48 At each round, there are fewer startups making it to 3. 4. 5. 50 the subsequent round for more capital.49 Similarly, the pharmaceutical industry has strong data on success rates that firms utilize to manage risk. In a study of clinical drug development success rates from 2006-2015, overall success from Phase I to approval was approximately 10%. Significant variability exists between the phases, with success rates varying from 63.2%, 30.7%. 58.1% and 85.3% from transitions between Phase I to II, Phase II to III, Phase III to New Drug Application/Biologic License Application (NDA/BLA) and NDA/BLA to approval respectively.50 Building from these and other appropriate industry data, the SL@B program can set initial benchmarks for expected success rates by stage and track, and measure performance over time, through which it can adapt targets for its portfolio. This approach also holds strong promise to be deployed across the Grand Challenges and global health innovation communities. c. To strengthen its operating approach as a venture philanthropy program, SL@B could consider adding more investment professionals, especially those with LMIC venture capital experience, to the SL@B design and implementation team including the formal investment committee. Finally, we note the lack of comprehensive and high-quality data and evidence to inform the development and scale-up of global health innovations, including in MNH. While SL@B has undertaken program-level external reviews, it has the opportunity to lead the global health innovation field by significantly strengthening its approach to data, evaluation, and evidence.This will require setting clear indicators and metrics of success at project, portfolio and program levels, and for the investment; and commitment to prospective evaluation, evidence generation, and transparent sharing of results. We note that while SL@B currently has a retroactively developed TOC, in our formal and informal engagement with teams across the SL@B partnership, we found that there is little reference to this TOC in decision-making and priority-setting, and a lack of clarity in how the TOC guides internal project and portfolio evaluations.1 SL@B has an opportunity to demonstrate world-leading commitment to evaluation and evidence and incorporation of continuous improvement principles and systems of a learning organization within and across the program. Much as it led the way as an early Grand Challenge for Development, SL@B can again lead the way for evidence generation and use in driving innovation, maximizing and accelerating impact while ensuring strong stewardship of scarce resources. 6. “For any good program to be successful, you need to have the courage to pause and course correct as you go along. This is very important for an ambitious and wonderful program such as SL@B. One key question to ask in this pause mode is the question “Why are we doing this?” or “Who are we doing this for?” Is it for the innovators, for the industry or is it for the mums and babies around the neediest places in the world? If the answer to this question is mums and babies, then SL@B must pause and redirect their efforts to ensure that there is an innovative business model to scale the innovations to the mothers and children that need it most. This is not business as usual and to assume that just because there is seed funding for an innovation (not underestimating the importance of that) there will be an automatic deployment of the technology.” – HIC MNH industry investor 51 1. Lalli M, Ruysen H, Blencowe H, et al. Saving Lives at Birth; development of a retrospective theory of change, impact framework and prioritised metrics. Global Health. 2018;14:13. doi:10.1186/s12992-018-0327-z 2. Dixit, S, F Fernholz, O Ogbuoji, A Finnegan, B Biru, K Udayakumar, and J Baumgartner. “Costing Tool for Health Innovators.” Durham: Evaluating Saving Lives at Births (E-SL@B) Program: Duke Global Health Institute Evidence Lab & Global Health Innovation Center at Duke University, 2019. 3. Arregoces L, Daly F, Pitt C, et al. Countdown to 2015: changes in official development assistance to reproductive, maternal, newborn, and child health, and assessment of progress between 2003 and 2012. Lancet Glob Heal. 2015;3(7):e410-e421. doi:10.1016/S2214-109X(15)00057-1 4. Dingle A, Schäferhoff M, Borghi J, et al. Estimates of aid for reproductive, maternal, newborn, and child health: findings from application of the Muskoka2 method, 2002–17. Lancet Glob Heal. 2020;8(3):e374-e386. doi:https:// doi.org/10.1016/S2214-109X(20)30005-X 5. Grand Challenges Canada. Saving Lives at Birth. https://www.grandchallenges.ca/programs/saving-lives-at-birth/. Published 2019. 6. Canada GCCD. The Grand Challenges Approach. 2011. 7. WHO, UNICEF, UNFPA, Group WB, Division UNP. Trends in maternal mortality: 1990 to 2015: estimates by WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division. 2015. 8. World Health Organization. Maternal Mortality Factsheet. 9. WHO. Newborns: reducing mortality. https://www.who.int/news-room/fact-sheets/detail/newborns-reducing￾mortality. Published 2019. Accessed June 8, 2020. 10. World Health Organization. Stillbirths. https://www.who.int/reproductivehealth/topics/maternal_perinatal/ stillbirth/en/. Published 2016. 11. Liu L, Oza S, Hogan D, et al. Global, regional, and national causes of under-5 mortality in 2000-15: an updated systematic analysis with implications for the Sustainable Development Goals. Lancet (London, England). 2016;388:3027-3035. doi:10.1016/S0140-6736(16)31593-8 12. World Health Organization. Global Health Observatory (GHO) Data: Infant Mortality Situation and Trends. https:// www.who.int/gho/child_health/mortality/neonatal_infant_text/en/. Published 2019. 13. Say L, Chou D, Gemmill A, et al. Global causes of maternal death: a WHO systematic analysis. Lancet Glob Heal. 2014;2:e323-e333. doi:https://doi.org/10.1016/S2214-109X(14)70227-X 14. The International Development Innovation Alliance (IDIA). Insights on Scaling Innovation. 2017. https://www. idiainnovation.org/idia-insights. 15. Fernandes G, Sridhar D. World Bank and the Global Financing Facility. BMJ. 2017;358:j3395. doi:10.1136/bmj.j3395 16. UNICEF. Maternal Mortality. https://data.unicef.org/topic/maternal-health/maternal-mortality/. Published 2019. 17. UNICEF. Neonatal Mortality. https://data.unicef.org/topic/child-survival/neonatal-mortality/. Published 2019. 18. Metrics for Management. The Equity Tool. https://www.equitytool.org/. Published 2018. 19. Desiraju R, Nair H, Chintagunta P. Diffusion of new pharmaceutical drugs in developing and developed nations. Int J Res Mark. 2004;21:341-357. doi:10.1016/j.ijresmar.2004.05.001 20. Balas EA, Boren SA. Managing Clinical Knowledge for Health Care Improvement. Yearb Med Inform. 2000:65-70. 21. Kaplan A V, Baim DS, Smith JJ, et al. Medical device development: from prototype to regulatory approval. Circulation. 2004;109(25):3068-3072. https://wwww.unboundmedicine.com/medline/citation/15226221/Medical_ device_development:_from_prototype_to_regulatory_approval_. 22. BEMPU Health. TempWatch: Baby Temperature Monitor. https://www.bempu.com/tempwatch. 23. Lunze K, Bloom DE, Jamison DT, Hamer DH. The global burden of neonatal hypothermia: systematic review of a major challenge for newborn survival. BMC Med. 2013;11(1):24. doi:10.1186/1741-7015-11-24 24. Department of Reproductive Health and Research World Health Organization. Kangaroo Mother Care, A Practical Guideline.; 2003. https://apps.who.int/iris/bitstream/handle/10665/42587/9241590351. pdf;jsessionid=4EAC5B98EB8C0D33794CB5DE721915B4?sequence=1. 25. National Health Mission Rajasthan, WISH Foundation, BEMPU Health. Pilot of the BEMPU Device for Reduction in Hypothermia and Infection Related Neonatal Mortality and Morbidity in Rajasthan.; 2017. 26. Gradian Health Systems. Our Model. https://www.gradianhealth.org/how-we-work/. BIBLIOGRAPHY 52 27. Gradian Health Systems. Gradian Health Systems Wins Saving Lives at Birth Award to Improve Maternal & Newborn Healthcare in Zambia. https://www.gradianhealth.org/press-release-gradian-health-systems-wins-saving￾lives-birth-award-improve-maternal-newborn-healthcare-zambia/. 28. World Health Organization. Guidelines for Healthcare Equipment Donations. Heimann P, Issakov A, Kwankam SY, eds. 2000. 29. Perry L, Malkin R. Effectiveness of medical equipment donations to improve health systems: how much medical equipment is broken in the developing world? Med Biol Eng Comput. 2011;49(7):719-722. doi:10.1007/s11517-011- 0786-3 30. Adair-Rohani H, Zukor K, Bonjour S, et al. Limited electricity access in health facilities of sub-Saharan Africa: a systematic review of data on electricity access, sources, and reliability. Glob Heal Sci Pract. 2013;1(2):249-261. doi:10.9745/GHSP-D-13-00037 31. Belle J, Cohen H, Shindo N, et al. Influenza preparedness in low-resource settings: A look at oxygen delivery in 12 african countries. J Infect Dev Ctries. 2010;4:419-424. doi:10.3855/jidc.859 32. Gradian Health Systems. Anaesthesia Workstation. https://www.gradianhealth.org/our-products/uam/. 33. Global Financing Facility. Global Financing Facility and Partners Announce Winners of Innovation Awards to Save the Lives of Women, Children and Adolescents. https://www.globalfinancingfacility.org/global-financing-facility￾and-partners-announce-winners-innovation-awards-save-lives-women-children. Published 2019. 34. Powell DL, Gilliss CL, Hewitt HH, Flint EP. Application of a Partnership Model for Transformative and Sustainable International Development. Public Health Nurs. 2010;27(1):54-70. doi:10.1111/j.1525-1446.2009.00827.x 35. Buse K, Tanaka S. Global Public-Private Health Partnerships: lessons learned from ten years of experience and evaluation. Int Dent J. 2011;61(s2):2-10. doi:10.1111/j.1875-595X.2011.00034.x 36. Saldinger A. What Makes a Global Health Partnership Succeed. Devex. https://www.devex.com/news/what-makes￾a-global-health-partnership-succeed-84858. Published 2014. 37. JSI. International - JSI International Public Health Management. https://www.jsi.com/international. 38. USAID. Nepal’s Navel Glazers. USAID Stories. https://stories.usaid.gov/nepals-navel-glazers/#page-9. Published 2019. 39. Mullany LC, Darmstadt GL, Khatry SK, et al. Topical applications of chlorhexidine to the umbilical cord for prevention of omphalitis and neonatal mortality in southern Nepal: a community-based, cluster-randomised trial. Lancet (London, England). 2006;367(9514):910-918. doi:10.1016/S0140-6736(06)68381-5 40. Lamichhane B, Khanal L, Shrestha P, Dawson P, Singh S. Nepal Scale up Chlorhexidine as Part of Essential Newborn Care: Country Experience. J Nepal Health Res Counc. 2018;16:359. doi:10.3126/jnhrc.v16i3.21443 41. JSI. Journey to Scale: Bringing a Lifesaving Commodity to Newborns Across Nepal. https://www.jsi.com/journey-to￾scale-bringing-a-lifesaving-commodity-to-newborns-across-nepal/. Published 2013. 42. JSI. Introducing Chlorhexidine in Nepal. https://www.healthynewbornnetwork.org/hnn-content/uploads/Nepal￾Introducing-Chlorhexidine_-Case-Study.pdf. Published 2013. Accessed July 31, 2020. 43. World Health Organization. World Health Organization Model List of Essential Medicines, 21st List. Geneva; 2019. 44. JSI. Nepal as Global Leader in the Use and Scale Up of Chlorhexidine. https://www.healthynewbornnetwork.org/ hnn-content/uploads/Nepal-Global-Leadership-Chlorhexidine.pdf. Published 2017. 45. Ali A, Nudel J, Heberle CR, Santorino D, Olson KR, Hur C. Cost effectiveness of a novel device for improving resuscitation of apneic newborns. BMC Pediatr. 2020;20(1):46. doi:10.1186/s12887-020-1925-5 46. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Heal Care. 2007;19(6):349-357. doi:10.1093/intqhc/mzm042 47. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. doi:10.1136/bmj.c332 48. Quintero S. Dissecting startup failure rates by stage. 2017. https://medium.com/journal-of-empirical￾entrepreneurship/dissecting-startup-failure-by-stage-34bb70354a36. 49. CB Insights. Venture capital funnel shows odds of becoming a unicorn are about 1%. 2018. https://www.cbinsights. com/research/venture-capital-funnel-2/. 50. BIO, Biomedtracker, Amplion. Clinical Development Success Rates 2006-2015.; 2016. https://www.bio.org/sites/ default/files/legacy/bioorg/docs/Clinical Development Success Rates 2006-2015 - BIO, Biomedtracker, Amplion 2016.pdf. 53 AUTHORS’ ROLES AND CONTRIBUTIONS Note on Authorship: As joint principal investigators of the Evaluating Saving Lives at Birth program, Joy Noel Baumgartner and Krishna Udayakumar share joint senior authorship by providing strategic and detailed guidance to inform the evaluation. Blen Biru, Sowmya Rajan, and Andrea Taylor are joint first authors, contributing to multiple aspects of study design, data collection, analysis and writing of the report. *Disclosures: Krishna Udayakumar is the Co-PI of Evaluating SL@B (ESL@B) as well as PI of Accelerating SL@B (ASL@B), a program that has provided accelerator support and technical assistance for select SL@B innovations offered through another team at Duke Global Health Innovation Center from January 2018 to May 2020. ASL@B has also managed the DevelopmentXChange (Devx) in 2018 and 2019. To offset potential bias, Joy Noel Baumgartner led all efforts on data collection, analysis, interpretation, and writing involving the non-financial support provided by SL@B (including the Xcelerator and DevX), while both PIs (Krishna Udayakumar and Joy Noel Baumgartner) co-led all other aspects of the evaluation. The following organizations have provided research and operational support (funding and/or in-kind) through Duke Global Health Innovation Center or Duke-affiliated non-profit, Duke Global Health Institute Evidence Lab, and Innovations in Healthcare, in the past three years (2018-2020): Amgen, Asia Pacific Medical Technology Association, AstraZeneca, Bayer, Bill and Melinda Gates Foundation, Cardinal Health, Catholic Relief Services, Colorado Hospital Association, Commonwealth Fund, Duke Corporate Education, FHI 360, Grand Challenges Canada, GlaxoSmithKline, LifeNet International, McKinsey & Company, Medtronic, National Institutes of Health, Novartis Foundation, Pfizer, Pfizer Foundation, Takeda, United States Agency for International Development, Vynamic, and World Economic Forum. AUTHOR CONTRIBUTION Blen Biru, MSc Duke Global Health Institute Evidence Lab Overall study design, data collection, analysis, writing, editing Sowmya Rajan, PhD Duke Global Health Innovation Center Overall study design, data collection, analysis, writing, editing Andrea Taylor, MSW Duke Global Health Innovation Center Overall study design, data collection, analysis, writing, editing Kate Crissman, MPH Duke Global Health Innovation Center Overall analysis and writing and editing Siddharth Dixit, MIDP Duke Global Health Institute Evidence Lab CEA-related data collection, analysis, writing Pratik Doshi, MS Duke Global Health Innovation Center Analysis and writing Minahil Shahid, MPP Duke Global Health Institute Evidence Lab CEA-related data collection, analysis, writing, and analysis of online survey Joy Noel Baumgartner, MSSW, PhD Duke Global Health Institute Evidence Lab Principal Investigator, overseeing all aspects of research design, data collection, analysis, writing, and interpretation of findings Krishna Udayakumar, MD, MBA,* Duke Global Health Innovation Center Principal Investigator, overseeing all aspects of research design, data collection, analysis, writing, and interpretation of findings 54 ABOUT THE CENTERS Duke Global Health Innovation Center The Duke Global Health Innovation Center’s (GHIC) mission is to study and support the scaling and adaptation of innovations, and related policy reforms, to address critical health challenges worldwide. The GHIC strives to have an impact on healthcare and health through scaling of health innovations, promoting policy and regulatory changes, and implementation projects in health systems. The GHIC links global health, health policy, and health innovation efforts across Duke University. Duke Global Health Institute Evidence Lab The Duke Global Health Institute (DGHI) Evidence Lab blends theory and practice and draws upon the research and policy expertise across Duke University to inform our evaluations and to disseminate new evidence to policymakers, donors and diverse stakeholders to inform decision-making. With deep, on-the-ground knowledge and experience with a wide range of global health projects, our team offers research and practice-based understandings of regional health challenges. A core principle of the DGHI Evidence Lab is to strengthen the evaluation capacity of local project counterparts on collaborative projects. Duke Center for International Development The Duke Center for International Development (DCID), a unit within Duke University’s Sanford School of Public Policy, advances international development policy and practice through interdisciplinary approaches to post￾graduate education, mid-career training, international advising and research. DCID faculty and staff continuously strive to create programs that meet the specific needs of each client and student. Additional Contributors AUTHOR CONTRIBUTION Fernando Fernholz, MSCE, MPA, PhD Duke Center for International Development Co-lead on CEA, data collection, analysis, and writing Osondu Ogbuoji, MBBS, MPH, ScD Duke Global Health Institute, Center for Policy Impact in Global Health Co-lead on CEA, data collection, analysis, and writing Amy Finnegan, PhD Duke Global Health Institute Evidence Lab Overall study design, data collection, analysis, and writing of first draft Stacey Dixson, Ed.M Duke Global Health Institute Evidence Lab First draft of writing 55 APPENDIX I SL@B’S THEORY OF CHANGE Source: Lalli M, Ruysen H, Blencowe H, et al. Saving Lives at Birth; development of a retrospective theory of change, impact framework and prioritized metrics. Glob Health. 2018;14(1). doi:10.1186/s12992-018-0327-z 56 Key outputs sought by SL@B funding partners organized into thematic impact areas. APPENDIX II SL@B’S IMPACT AREA FRAMEWORK Source: SL@B Impact Framework listing Key Dimensions of Interest, shared by USAID via email communication in Powerpoint, with ESL@B team in January 2019. IMPACT AREA DESCRIPTION Lives Saved / Improved Current number of lives saved or DALY’s or QALY’s improved by fielded innovations Acceleration to Market Future impact of bringing innovations to scale faster (reducing the time to scale), further (broader reach or level of scale), or more equitably (including marginalized populations) Pipeline development and new partnerships Bringing different expertise and unconventional partners (e.g., car mechanics) to tackle MCH challenges and the resulting innovations (e.g., Odon device) Market Shaping Improving efficiency of relevant markets for these innovations to increase LMIC access by: 1) Reducing transaction costs 2) Increasing market information 3) Balancing supplier and buyer risks* Capacity Development Development of innovators’ capacity across core skill sets (i.e., business model, manufacturing or supply chain knowledge) and how that is translated into broader learnings for the field (i.e., accelerators or shared knowledge products, etc.) Increased awareness of MNH issues: 1. Media Attention 2. Funding Leveraged 3. Enabling Environments Awareness defines as: 1. Mainstream media attention, such as newspapers/blogs 2. Funding leveraged, including incremental funds to current donors’ MNH budgets and attracting new funders 3. Capacity building amongst enablers of innovation, including intended policymakers/regulators (i.e., LMIC MoH’s), universities, innovation accelerators, etc. Platforms that drive acceleration Establishing practices that help MNH donors efficiently support a new way of doing business around driving innovation, supporting innovators, and bringing successful innovations to scale 57 APPENDIX III SL@B’S NON-FINANCIAL SUPPORT The SL@B innovator survey, conducted by Duke University in 2019, asked innovators about a variety of non￾financial support areas (this list was determined in consultation with SL@B funding partners). The survey asked respondents how helpful (low, medium, and high) and how sufficient (sufficient or insufficient) each type of support from SL@B had been. Figures 1 through 10 below present a summary of how helpful and sufficient the following areas of SL@B’s non-financial support was for innovators: grants management, market entry, public relations, accelerator, and networking. The figures also note the number of respondents who reported receiving each type of support. Figure 1.Grants management support provided by SL@B SL@B Support Found Helpful SL@B Support Found Sufficient Figure 2: Grants management support provided by SL@B Number of respondents Number of respondents 25 20 15 10 5 0 40 35 30 25 20 15 10 5 0 Low High Low High 4 11% 3 8% 17 46% 32 86% 6 46% 5 38% 5 39% 6 46% Grants Management (n=37) Grants Management (n=37) USAID Intern (n=13) USAID Intern (n=13) 58 Figure 3: Market entry support provided by SL@B Figure 4: Market entry support provided by SL@B Number of respondents Number of respondents 25 20 15 10 5 0 35 30 25 20 15 10 5 0 Low High Low High 6 19% 4 12% 2 10% 14 44% 14 44% 8 30% 6 22% 11 55% 15 75% 11 44% 16 64% 10 53% 7 37% 12 44% 20 74% 5 25% 5 20% 5 20% 3 - 16% 9 47% Market Understanding (n=32) Market Understanding (n=32) Development Impact Frameworks (n=27) Development Impact Frameworks (n=27) Market Identification (n=25) Market Identification (n=25) SL@B Proposal Support (n=20) SL@B Proposal Support (n=20) TA by USAID Missions (n=19) TA by USAID Missions (n=19) SL@B Support Found Helpful SL@B Support Found Sufficient 59 Figure 5: Public relations support provided by SL@B Figure 6: Public relations support provided by SL@B Number of respondents Number of respondents 30 25 20 15 10 5 0 35 30 25 20 15 10 5 0 Low High Low High 9 27% 10 30% 17 52% 21 64% 6 26% 8 33% 6 35% 11 65% 6 35% 6 35% 5 50% 4 - 40% 7 29& 16 70% 7 70% 11 48% 15 62% 9 38% 6 26% Communications Support (n=33) Communications Support (n=33) Speaking Opportunities (n=23) Speaking Opportunities (n=23) Media Promotion (n=24) Media Promotion (n=24) Journal Publication Support (n=17) Journal Publication Support (n=17) Manuscript Editing (n=10) Manuscript Editing (n=10) SL@B Support Found Helpful SL@B Support Found Sufficient 60 Figure 7: Accelerator support provided by SL@B Figure 8: Accelerator support provided by SL@B Number of respondents Number of respondents 30 25 20 15 10 5 0 35 30 25 20 15 10 5 0 Low High Low High 7 22% 3 - 9 % 19 59% 29 85% 5 15% 5 16% 9 43% 16 76% 5 24% 8 50% 4 25% 3 - 19% 10 62% 12 50% 17 71% 21 62% 25 78% 4 17% 5 21% Pitch Support (n=32) Pitch Support (n=32) VentureWell Accelerator (n=34) VentureWell Accelerator (n=34) Other Accelerator Support (n=24) Other Accelerator Support (n=24) Business Plan Development (n=21) Business Plan Development (n=21) High-touch Consulting (n=16) High-touch Consulting (n=16) 3 - 14% SL@B Support Found Helpful SL@B Support Found Sufficient 61 Figure 9: Networking support provided by SL@B Figure 10: Networking support provided by SL@B Number of respondents Number of respondents 35 30 25 20 15 10 5 0 50 45 40 35 30 25 20 15 10 5 0 Low High Low High 5 11% 4 - 9% 28 60% 41 87% 12 35% 11 32% 8 42% 12 71% 4 24% 7 50% 11 79% 9 53% 6 43% 2 - 14% 10 59% 9 35% 18 69% 7 41% 12 63% 10 29% 21 62% 8 31% 6 23% 8 31% 5 26% 5 26% DevX (n=47) DevX (n=47) Facilitate Connections to Funders (n=34) Facilitate Connections to Funders (n=34) Facilitate Connections to Non-financial Partners to Scale (n=26) Facilitate Connections to Non-financial Partners to Scale (n=26) Facilitate Connections to Public Sector for Scale (n=17) Facilitate Connections to Public Sector for Scale (n=17) Support Developing Proposals for Follow￾on funding (n=14) Support Developing Proposals for Follow￾on funding (n=14) Letters of Support (n=17) Letters of Support (n=17) Facilitate Connections to Private Sector for Scale (n=14) Facilitate Connections to Private Sector for Scale (n=14) 6 32% 4 - 24% SL@B Support Found Helpful SL@B Support Found Sufficient 62 APPENDIX IV IMPACT MODELING CHECKLIST, WORKING DRAFT Section/Topic Checklist item Parameters Included Sensitivity Assessed Y/N/NA Describe Y/N/NA Describe Model description Theory of Change Describe the innovation's Theory of Change (TOC) or logic model. Study perspective Describe the perspective of the study and relate this to the impacts being evaluated. Options include: innovator perspective, societal perspective, health/epidemiological perspective. Time horizon State the time horizon(s) over which impact is being evaluated and describe why appropriate. Target population Describe the target population. Assumptions Demographic factors Disease burden Does the model include assumptions for how the at-risk population will change over time without the intervention? Describe whether the assumption is based on a level or trend change or both. For example, fertility trends, neonatal and maternal mortality trends etc. Describe the source of information used. Efficacy & Fidelity Efficacy Does the model account for the efficacy of the innovation with typical use and perfect use compared to some counterfactual? Describe the source of information used. State the rigor of the evidence base for efficacy measurements. Options include: pilot, RCT, observational study (case￾control), anecdotal evidence. Describe the sample size, p-value, effect size, level/tier of health system included in evidence, etc. Describe the source of information used. Fidelity to treatment protocol (intermediary) Does the model include a parameter for quality of service delivery by intermediaries (e.g. health workers, manufacturing)? Describe the source of information used. Fidelity to treatment protocol (beneficiary) Does the model include a parameter for fidelity to treatment protocol by the beneficiary (e.g. improper use, sub-optimal use)? Describe the source of information used. Health system factors Equitable access Does the model include a parameter for equitable access to services? For example, does access to service vary across rural vs. urban populations, across vulnerable populations etc.? Describe the source of information used. Stock outs Does the model account for stock outs? Describe how the model takes this into consideration and if not, why not. Describe the source of information used. Referrals Does the model account for referral of severe cases? For example, what proportion of cases will be referred and once referred what proportion will actually reach care. Describe how the model takes this into consideration and if not, why not. Describe the source of information used. Attrition of intermediaries Does the model include attrition and replacement of CHWs, nurses, etc.? Describe how the model takes this into consideration and if not, why not. Describe the source of information used. Scenarios Innovator scale-up Does the model include innovator reported scale up as a scenario? Describe the scenario. Universal health coverage Does the model include universal health coverage as a scenario? Describe the scenario. Note. Developed by Evidence Lab at the Duke Global Health Institute (DGHI). CONTACT Krishna Udayakumar, MD, MBA Co-Principal Investigator Director, Duke Global Health Innovation Center, Duke Global Health Institute Associate Professor of Global Health and Medicine ku@duke.edu Joy Noel Baumgartner, PhD, MSSW Co-Principal Investigator Director, DGHI Evidence Lab, Duke Global Health Institute Associate Research Professor of Global Health joy.baumgartner@duke.edu This draft report was produced for review by Grand Challenges Canada and the United States Agency for International Development. It was prepared by the Duke Global Health Innovation Center and the Duke Global Health Institute Evidence Lab at Duke University, and was authored by: Blen Biru, MSc, Duke Global Health Institute Evidence Lab Sowmya Rajan, PhD, Duke Global Health Innovation Center Andrea Taylor, MSW, Duke Global Health Innovation Center Kate Crissman, MPH, Duke Global Health Innovation Center Siddharth Dixit, MIDP, Duke Global Health Institute Evidence Lab Pratik Doshi, MS, Duke Global Health Innovation Center Minahil Shahid, MPP, Duke Global Health Institute Evidence Lab Joy Noel Baumgartner, MSSW, PhD, Duke Global Health Institute Evidence Lab Krishna Udayakumar, MD, MBA, Duke Global Health Innovation Center With significant contributions from: Fernando Fernholz, MSCE, MPA, PhD, Duke Center for International Development Osondu Ogbuoji, MBBS, MPH, ScD, Duke Global Health Institute, Center for Policy Impact in Global Health Amy Finnegan, PhD, Duke Global Health Institute Evidence Lab Stacey Dixson, EdM, Duke Global Health Institute Evidence Lab DISCLAIMER: This product is made possible through the generous support of the Saving Lives at Birth partners: the United States Agency for International Development (USAID), Grand Challenges Canada (GCC) (with funding from the Government of Canada), Norwegian Agency for Development Cooperation (NORAD), the Bill & Melinda Gates Foundation (BMGF), the UK Department for International Development (DFID), and the Korea International Cooperation Agency (KOICA). The contents of this report are the sole responsibility of the Duke Global Health Innovation Center and the Duke Global Health Institute Evidence Lab, and do not necessarily reflect the views of Saving Lives at Birth partners.