Nala* (20) holds her son Ahmed* (1) while he receives therapeutic milk in Somalia. March 24, 2021. Image credit: Sacha Myers / Save the Children READY I: Final Evaluation September 2018-March 2022* Evaluation of READY Initiative—Global Readiness for Major Outbreak Response READY / March 2022 1 *This report is an evaluation of the first phase of READY (READY I). READY has now been granted funding for a second phase (READY II) from April 2022 through March 2024. Acknowledgements This evaluation was conducted by Veronique de Clerck and Don Johnston, independent consultants, on behalf of the READY initiative. The consultants would like to acknowledge all members of the READY consortium, as well as participants from other agencies, who provided invaluable contributions and insight for this evaluation. The evaluation has benefited greatly from the openness and support of all those involved in the process. Particular recognition goes to Laura Cardinal, READY’s Chief of Party, and Eilidh Higgins, READY’s Senior MEAL Advisor, for their support, transparency, and guidance. This evaluation is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents of the evaluation are the responsibility of the READY Initiative and do not necessarily reflect the views of USAID or the United States Government.  Led by Save the Children, READY is implemented in partnership with the Johns Hopkins Center for Humanitarian Health, the Johns Hopkins Center for Communication Programs, UK-Med, EcoHealth Alliance, and MERCY Malaysia. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response READY / March 2022 LIST OF FIGURES I LIST OF TABLES I ACRONYMS & ABBREVIATIONS II 1 INTRODUCTION AND BACKGROUND 1 2 EVALUATION PURPOSE, SCOPE, APPROACH, AND METHODOLOGY 3 3 LIMITATIONS 6 4 FINDINGS 7 A. DESIGN 7 B. IMPLEMENTATION, CAPACITY, PREPAREDNESS 10 C. CROSS-CUTTING THEMES 14 D. LONG-TERM RESULTS AND SUSTAINABILITY 15 5 CONCLUSIONS AND RECOMMENDATIONS 16 REFERENCES 22 ANNEXES 25 ANNEX 1: TERMS OF REFERENCE 25 ANNEX 2: EVALUATION MATRIX 29 ANNEX 3: EVALUATION QUESTION GUIDE 32 ANNEX 4: DOCUMENT REVIEW 38 ANNEX 5: LEVEL OF ENGAGEMENT 39 ANNEX 6: SURVEY QUESTIONS 41 ANNEX 7: DEDOOSE ANALYTICAL CODEBOOK 47 ANNEX 8: READY PRODUCED AND SUPPORTED PRODUCTS 48 List of Figures FIGURE 1. READY INITIATIVE THEORY OF CHANGE 3 List of Tables TABLE 1. EVALUTATION QUESTIONS AND OECD-DAC CRITERIA 4 TABLE 2. DATA COLLECTION METHODS 5 Evaluation of READY Initiative—Global Readiness for Major Outbreak Response READY / March 2022 ii Acronyms & Abbreviations BHA Bureau for Humanitarian Assistance CDC Centers for Disease Control and Prevention COVID-19 Coronavirus Disease 2019 DEI Diversity, Equity, and Inclusion DIS Data Implementation Science EVD Ebola Virus Disease EQ Evaluation Question GBV Gender Based Violence GHC Global Health Cluster GOARN Global Outbreak Alert and Response Network IASC Inter-agency Standing Committee IFRC International Federation of the Red Cross INGO International Non-Governmental Organization IYCF Infant and Young Child Feeding KII Key Informant Interview NGO Non-Governmental Organization OECD-DAC Organization for Economic Co-operation and Development’s Development Assistance Committee PHEIC Public Health Agenda of International Concern RCCE Risk Communication and Community Engagement SAG Strategic Advisory Group UNICEF United Nations Children’s Fund USAID United States Agency for International Development UN United Nations WASH Water, Sanitation, and Hygiene WHO World Health Organization Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 1 1 Introduction and Background The 2014 West Africa Ebola Virus Disease (EVD) was pivotal in generating institutional reflection around Public Health Emergencies of International Concern (PHEIC). Since then, significant structural recommendations and reforms have been made to strengthen the World Health Organization (WHO), United Nations (UN), and broader humanitarian response systems. These include the World Health Assembly’s recommendations for addressing the global health emergency workforce, contingency funding, research and development, and health system strengthening (WHO, 2015; Moon et al., 2017, p.8). It is estimated that the yearly probability of occurrence of extreme epidemics could increase up to threefold in the coming decades (Marani et al., 2021, p.4). Climate change ecosystem imbalances, mass population displacements, urbanization, globalization, and increasing trends of antibiotic resistance have increased the risk of infectious disease outbreaks (Baker, 2022). Despite considerable progress in sanitation and access to health care worldwide, infectious disease prevention and control remains a challenge. Rapid containment of new and emerging pathogens has not always proven possible (Webster, 2004; Chen et al., 2021). Multiple outbreaks within a short time span over an increasing geographic range lead to further depletion of the scarce resources that are available for containment efforts (Mills et al., 2006). Outbreaks occurring in regions with weak public health infrastructure and underfunded outbreak preparedness can also lead to containment failure (WHO 2020). Additionally, rumors, misinformation, and public distrust of government, public health systems and health measures make containment efforts complex. For example, during both the 2014 West Africa EVD epidemic and the EVD epidemics that occurred in the Democratic Republic of Congo from 2018–2020, communities reached for culturally familiar explanations of disease transmission and rejected disease control practices that clashed with their traditional healing and burial practices (Roca, 2015; WHO, 2018; Crawford, 2021). Rumors circulated that the disease had been created by the government (Schwerdtle et al., 2017); this type of community feedback provides an underutilized but powerful insight into new forms of evidence and tools in outbreak responses (McKay et al., 2022, p.11). In humanitarian terms, outbreaks were identified as simultaneously “increasing population vulnerability and reducing the system’s response capacity” (Altare, 2019). These issues add layers of complexity to existing deficits in the humanitarian sector’s ability to meet the needs of affected communities (ALNAP, 2015; ALNAP, 2018). The 2018 State of the Humanitarian System Report (ALNAP, 2018) found that in 2017 the total combined field personnel in the humanitarian sector numbered approximately 570,000 with Non-Governmental Organization (NGO) workers estimated to contribute 331,000 field personnel. The humanitarian NGO sector is highly disparate in its workforce size and scale, with six International Non-Governmental Organizations (INGO) accounting for roughly a quarter of the humanitarian spending and very few national NGOs having budgets over $2 million per annum (ALNAP, 2018). This imbalance in INGO– national NGO representation reflects a broad challenge experienced by the humanitarian system as a whole—how to balance the promotion of a localization agenda that encourages Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 2 increased resilience on the one hand, while at the same time continuing to build global surge capacity, so that in tandem they can respond to outbreaks against known and novel pathogens (Sullivan, 2022; Start Network, 2017), with the result that fewer humanitarian crises become overwhelming (Robillard, 2021). Considering these post-West Africa EVD assessments of the global health and humanitarian sectors’ performance the Bureau for Humanitarian Assistance (BHA), under the United States Agency for International Development (USAID), awarded a three-year, $8 million grant to Save the Children to lead the READY initiative. In 2018 the READY initiative, a global consortium that augments NGO capacity for infectious disease outbreak response in humanitarian settings, was formed. The READY initiative was developed to meet a deficit in readiness and response capacity by improving operational and technical capacity across sectors, using an integrated technical approach, and supporting coordination with global and regional stakeholders. The READY initiative proposal presented the consortium as a body that aims to “ensure that the right people are in the right place at the right time, with the right skills, equipment, funds, [and] coordinated, integrated ways of working”—a level of coordination that has been found to be lacking in recent humanitarian emergencies (Castellarnau 2018). The READY initiative brought together capacity from Save the Children; the Johns Hopkins University Center for Humanitarian Health; the Johns Hopkins University Center for Communication Programs; UK-Med, MERCY Malaysia; and the EcoHealth Alliance. These organizations have well-recognized capacities to deliver on READY’s ambition to fill critical gaps in NGO outbreak response capacity by leveraging expertise across operational, academic, clinical, and communications domains to develop a capacitated humanitarian NGO sector for future infectious disease outbreaks. The main objectives of the READY initiative were initially threefold: 1. Improve coordination, both operational and technical, between global and regional disease response platforms and initiatives. 2. Strengthen the operational capacity of organizations to launch an infectious disease outbreak response more efficiently. 3. Improve technical preparedness and readiness across relevant sectors for infectious disease response in humanitarian settings. BHA acknowledged that READY needed flexibility in its approach to enable it to fine-tune its activities. In the early stages of the award, there was a pivot to expand the focus of READY. For example, in assessing operational needs, it was found that to strengthen boots-on-the-ground emergency outbreak response and real-time learning in the interests of the "greater good," the initiative would need to broaden its ambitions beyond building the capacity of consortium partners only, to include broader NGO capacity￾building. This pivot made READY more fit-for-purpose. A second major pivot occurred when, during the award period, a novel infectious disease - Coronavirus Disease 2019 (COVID-19) was detected in China that was declared a WHO PHEIC in January 2020. By March 2020 the outbreak was declared a pandemic and the whole global health and humanitarian sector—including READY—pivoted towards COVID-19. The ambitions and working methods of READY changed in response to COVID-19, and its Year 3+ work plan reflected these changes. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 3 Figure 1. READY theory of change 2 Evaluation Purpose, Scope, Approach, and Methodology Purpose The purpose of this evaluation has been to identify to what extent engagement in READY activities and events successfully augmented NGO readiness and capacity for outbreak response. It also seeks to identify which activities it did well, which areas it can improve on, and where future similar initiatives can add value. Objectives The objectives of the evaluation were: 1) To assess the relevance of the READY initiative’s design and activities in building response readiness capacity. 2) To determine which operational and technical activities were perceived as the most effective and impactful for improving organizational-level NGO readiness to respond to disease outbreaks. 3) To assess READY’s engagement and coordination in the global outbreak and humanitarian coordination structures and how this contributed to achieving results. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 4 4) To assess cross-cutting themes including integration, localization, and diversity and inclusion. Scope The evaluation concentrated primarily on READY outputs such as products (i.e., technical and operational tools and guidance), trainings, and webinars but also on READY’s consortium partners and their engagement with global, and to a lesser extent regional and country level outbreak response coordination platforms and partnerships. The evaluation period covered the first phase of READY from September 2018 through the beginning of March 2022. The geographical scope was global. Approach The evaluation was formative in that it focused on what had been learned that could help to shape the future of the initiative and improve performance. An inception phase determined the scope of the evaluation, improved the evaluation team’s understanding of READY, and identified key areas of focus for evaluation questions. A mixed-methods approach was applied which allowed for triangulation and synthesis across primary, secondary, qualitative, and quantitative data sources. Using the Terms of Reference (see ToR in Annex 1) and interviews carried out during the inception process, four priority areas of investigation were identified. These informed the framing of four high-level evaluation questions (EQs) and a series of sub-questions. The four areas are: (A) Design (model and strategy); (B) Implementation (delivery and partnerships); (C) Cross-cutting themes (localization, integration, diversity, and inclusion); (D) Long-term results (outcomes) and sustainability. Each area of investigation was mapped against evaluation criteria from the Organization for Economic Co-operation and Development’s Development Assistance Committee (OECD-DAC) (see Table 1). Table 1. Evaluation questions and OECD-DAC criteria A. Design à relevance EQ 1: To what extent is READY relevant in strengthening disease outbreak coordination, operational capacity, and technical preparedness? B. Implementation à effectiveness, efficiency EQ 2: To what extent and how have READY activities built organizational and individual capacity to respond effectively and efficiently to major disease outbreaks? C. Cross-cutting themes à integration, localization, diversity, and inclusion EQ 3: To what extent were cross-cutting issues part of the design and were they operationalized effectively? D. Results à performance, outcomes, sustainability EQ 4: How did READY outputs contribute to sustainable long-term outcomes improving readiness and response capacity? Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 5 Methods During the inception phase, the evaluation matrix (see Annex 2) set out an overall approach and methods for data collection and analysis of EQ responses. The matrix also mapped sources of evidence and criteria for evaluation. Methods of data collection and analysis included a review of documents and key READY deliverables; semi-structured key informant interviews (KIIs) (see Annex 3); an online feedback survey from the target audience; and a workshop with READY consortium members to share and discuss evaluation findings (see Table 2). A document library was established, and qualitative and quantitative documents were reviewed by the evaluation team. These included key documents on the initiative’s design, landscaping, and consultations; work plans, progress reports and biannual reviews; monitoring data; technical tools and guidance documents; activity trackers; training materials and training results; and products and webinar materials and webinar results (see Annex 4). Key informant interviews were the primary data collection tool for this evaluation and generated rich insights from informants across five different categories of stakeholders at global, regional, and country level: (1) consortium partners; (2) NGOs; (3) UN agencies; (4) donors; and (5) the target audiences (see Annex 5). As part of the inception phase, eleven informal interviews were conducted to help scope the evaluation. During the data collection phase, thirty-two semi-structured KIIs were conducted. An online survey was developed in collaboration with the READY team which followed a non-probability sample, targeting READY audience members who utilized or participated in global, regional, and country-level READY products, webinars, and trainings. Participants represented different geographical areas, sectors, and organizations, including local stakeholders. The survey was developed in KoBo and participants were invited via email addresses they provided. The survey used mostly closed questions, with some open-ended questions; this mix allowed for a quantitative analysis complemented by qualitative interpretation (see Annex 3). A participatory workshop was conducted on March 24, 2022 and included eleven individuals from across the consortium members. During the workshop, preliminary findings, conclusions, and recommendations were presented and discussed, and feedback was received. Data collection methods Document review 35 internal documents were reviewed. Semi structured KIIs 11 inception interviews were conducted with all consortium partners. 32 semi-structured interviews with representatives from consortium partners (15), donors (6), United Nations (UN) agency (1), independent consultants (2), and NGOs (8). Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 6 Online survey An online survey was sent out to 3,374 people using the READY training and webinar participant lists; 217 people responded to the survey. Participatory Workshop A workshop was conducted on March 24, 2022, attended by 11 people from across the consortium. Table 2. Data collection methods Analysis For qualitative analysis a code book was developed (EQs, sub-EQs, sectors), and the data was entered in Dedoose® analytical software (see Annex 7); the analysis explored recurring themes and subthemes. Quantitative data was analyzed using basic Excel functions. All data from the different sources and methods were then triangulated and synthesized to inform the findings of this report. 3 Limitations The following limitations were identified: § The endline evaluation was conducted before phase I of the READY initiative ended: Several key deliverables from the workplan had not yet been completed or were not disseminated before the end of the evaluation period and are therefore not captured fully in the analysis.1 § Time was limited: The evaluation timeline was short for an initiative such as READY, which works at global, regional, and country levels, is quite complex in nature, and underwent two pivots and changes to its program design and reporting indicators during the life of the initiative. More time would have made it possible to carry out additional KIIs and to capture additional and more diverse perspectives. The team was, however, able to increase the number of KIIs from 25 to 32, mapping stakeholders against engagement categories to ensure a representative selection. § Data collection was mostly conducted in English: As a result, the survey may not have captured insights from respondents for whom English is a second language. The team conducted a small number of interviews with French-speaking stakeholders (e.g., Cameroon). § There was a low response rate to the online survey: Low response rates were anticipated early in the evaluation process and mitigation actions included a clear introduction outlining the benefits of engagement; a reminder email sent from the READY Chief of Party; a limited set of mostly closed questions; a time limit of five minutes; and simple and clear questions. The 6.4% response rate was nevertheless below what was anticipated. In addition, as reported by consortium members, lack of focus on READY branding and identity, especially early in the initiative, may have resulted in some people engaging with READY products and events but not 1 Some of these include: the Outbreak READY! digital simulation; WASH in Epidemics eLearning; Making Connections—Stories of Integration in Outbreak Response; Nutrition eLearning; Infant and Young Child (IYCF) Feeding, Remote Counseling: How to Support Caregivers During Infectious Disease Outbreaks and Other Settings; RCCE Readiness Kit—Communicating with Communities in Epidemics and Pandemics; Risk Communication and Community Engagement (RCCE); Outbreak Coordination—A Quick Guide from NGOs. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 7 recognizing them as such. A few survey respondents reported being unaware that they had engaged with the READY initiative. § Pivots and changes in READY’s activities and monitoring: These include: (1) a stronger focus on the greater good, resulting in broader NGO capacity strengthening across the sector as opposed to focusing solely on strengthening consortium member readiness; and (2) the pivot to a COVID-19 pandemic response, which was valuable in many ways but disrupted continuity and made the evaluation challenging, particularly in terms of tracking outcomes and impacts, and comprehensively capturing all of the initiative’s activities. § Program monitoring: The team was reliant on monitoring data that has been put in place by READY, mostly output level based, and due to the pivots mentioned earlier, monitoring indicators changed over time. This resulted in a lack of continuity and might have resulted in a lower strength of evidence. To complement monitoring data, as mentioned above an online survey and an analysis of training data and READY activities was conducted. Additional reviews were undertaken to mitigate these data limitations: The evaluation team were in communication with senior READY initiative personnel in the report drafting stage, to help overcome data gaps and inconsistencies in knowledge, and facilitate preparation of the final report. 4 Findings A. Design This section describes to what extent the READY design has been found to be relevant and coherent in strengthening disease outbreak coordination, operational capacity, and technical preparedness. It unpacks the original READY design along with amendments made to it over time, and shows how needs analysis, donor flexibility and external factors inform READY’s strategic ambitions and activities. The initial aim of READY was broad and ambitious, and deliberately incorporated flexibility into its design and implementation. This supported READY in effectively defining its overall strategic vision and target audience and allowed it to assess priority sectoral gaps and develop appropriate activities over time; but it also resulted in delays in READY’s ability to better define its strategic direction. The complexities of outbreak response were acknowledged in the KIIs in the conceptualization and award stage, and initial READY objectives were found to be highly relevant, as evidenced by their alignment with post-Ebola reviews of the global outbreak system and the humanitarian NGO sector (ALNAP, 2018; Moon et al., 2017; WHO, 2015). The ambitious scope of the original design was ultimately considered valuable, but it did add some ambiguity to the operational process. Initially several key external stakeholders found READY’s ambitions to be too broad and somewhat fragmented. External stakeholders found that if global and regional consultations with READY had been held sooner, this would have resulted in an earlier understanding of existing outbreak response sector roles, reforms, and gaps in provision. High turnover in leadership and key staff also contributed to delayed progress and some periods of uneven strategic direction. The desk review and the interviews demonstrated that there were uncertainties about where READY could add most value in its first year Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 8 and that it needed time to align and position itself alongside UN and state-based response systems. It took time for READY to define where it could best add value and it was not until around January 2020 that READY had narrowed its scope to areas of specific relevance. After the first year of the initiative, a comprehensive needs analysis and global and regional consultations were instrumental in effectively identifying relevant and specific gaps. Four large landscaping analyses were carried out: (1) Available Technical Guidelines Assessment Report; (2) Landscape Analysis of Global Public Health Emergency Response Systems and Guidance; (3) Review of Outbreak Preparedness Training and Gaps Analysis; and (4) READY Landscape Analysis: Snapshot of Public Health Emergency and Response Initiatives. These produced a set of twelve high-level recommendations that were an accurate proxy of desired outcomes, but which were also realistic in the context of what one three-year initiative could achieve. The Landscape Analysis provided insights into key platforms and initiatives in the outbreak response sector and identified existing externally supported activities that met READY’s three original objectives. For example, the Landscape Analysis findings recommended that READY not pursue generating new guidance material but rather position itself to contribute to ongoing work to fill technical gaps and develop operational “how-to” documents, including on cross￾cutting themes such as one-health, ethics, gender, social behavior, and child protection. It also suggested that READY could engage directly with major global surge mechanisms such as the Global Outbreak Alert and Response Network (GOARN), Emergency Medical Teams, etc., to help define and guide the development of “outbreak specialist” cells. In relation to capacity-building, there was a recommendation that READY could break out of the silos of sector-specific or disease-specific trainings and implement an integrated approach to its outbreak trainings as well as focus on non-technical (soft) skills such as leadership and management, negotiation and conflict resolution, planning and resource management. The Landscape Analysis recommendations were further aligned to lessons learned from the 2014 Ebola disease outbreak such as the need for emphasis on risk communication and community engagement (RCCE), coordinated responses, and integrated responses.2,3,4 In addition to the landscape analysis, primary stakeholders such as UN agencies and INGOs were included via global and regional consultations. This resulted in identifying relevant activities capable of addressing key sector gaps and ensuring that these did not duplicate existing activities in the outbreak sector. Consultations involving WHO, GOARN, sectoral clusters, and working groups allowed READY to support a combination of activities that amplified its contribution to outbreak response readiness and permitted it to fine tune its mix of local and global capacity strengthening. READY’s initial broad focus and flexible programming also allowed for two relevant programmatic pivots and a return to its initial ambitions. The first pivot occurred 2 WHO. (2020, April 10). Ebola then and now, Lesson 7. WHO’s emergency response structure has changed. 3 WHO. (2020, April 10). Ebola then and now. Lesson 6. Social science and community engagement integrated into the response. https://www.who.int/news-room/feature-stories/detail/ebola-then-and-now 4 WHO. (2020, April 10). Ebola then and now: Eight lessons from West Africa that were applied in the Democratic Republic of the Congo. https://www.who.int/news-room/feature-stories/detail/ebola-then-and￾now Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 9 early in the award, when the initial focus on building response capacity of consortium partners shifted to a focus on strengthening the capacity of the broader humanitarian NGO community. This shift relates to the fact that only a few organizations have appropriate technical and global surge capacity in responding to major disease outbreaks, and there is untapped capacity within the humanitarian sector (Barbelet 2021). The second pivot was towards COVID-19 which dominated the middle period of the award. In addition, the COVID-19 experience showed that national NGOs were often left to do the work as main frontline responders, and that despite this and the fact that they are often in the best position to navigate interactions between local communities and governments (WHO, 2022), not enough has been invested in NGOs. Following the second pivot, the program developed a Year 3+ workplan which was able to refocus on the broader original aims of the initiative after two years of COVID-19. It did this by building on prior learning, activities, and novel needs seen with the humanitarian NGO sector and directed attention to other infectious disease outbreaks, and system resilience. While the impact of COVID-19 on the initial READY strategy and activities was significant, pivoting to support COVID-19 readiness and response was predominantly successful. READY was seen as highly agile in supporting partner initiatives, and in providing added value aligned with the sector’s emerging needs and priorities. READY effectively pivoted from its broader aim of augmenting infectious disease response readiness to focus specifically on COVID-19 readiness and response capacity. The READY COVID-19 Coordination Report: Consultation and Findings, a companion piece to the existing Landscape Analysis, provided insights into COVID-19 public health and humanitarian response coordination and leadership platforms that assisted READY to amend its strategic focus.5 Engagement with the Global Health Cluster (GHC) COVID-19 Task Team, the Information Management Task Team and GOARN was highly relevant to gaining understanding of key operational challenges that partners faced in the field. As a result, READY was able to provide support to these task teams including through development of interim guidance notes on ethical decision-making during the COVID-19 response, development of essential health services prioritization guidance notes, guidance on COVID-19 vaccination, and a desk review that identified barriers to access to services responding to gender-based violence (GBV) in humanitarian settings. READY co￾led the RCCE sub-group on community engagement in low resource settings, and in 2021 focused on COVID-19 vaccine hesitancy, targeting, and access. Two consecutive COVID-19 ninety-day interim work plans were developed and implemented during this pivot. In October 2020, READY launched its Year 3+ workplan, which included translating lessons learned from COVID-19 back into original activities and developing new activities responding to evolving outbreak readiness needs. It brought back the original broader focus covering general outbreak readiness and specific pathogens beyond COVID-19; this has meant that the initiative maintained its three initial objectives while effectively responding to the pandemic. This work plan continues to 5 READY COVID-19 Coordination Report. (2020). https://www.ready-initiative.org/ready-covid-19- coordination-report Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 10 augment the existing capacity of NGOs to respond to major disease outbreaks in humanitarian settings within the COVID-19 context and beyond. COVID-19 gave time￾specific relevance to READY, enabled a focusing of its initial broad ambitions, and further established its “greater good” focus. B. Implementation, Capacity, Preparedness This section describes the extent of READY activities and how they have built organizational and individual capacity to respond effectively and efficiently to major disease outbreaks. It assesses READY’s delivery of outputs under each of its three objectives and their contribution to outcomes. As tracked by its monitoring framework, READY has delivered on its outputs under each ofits three objectives. READY’s monitoring was mostly focused on measuring activity outputs. Indicators were aligned with the pre￾pivot and post-pivot work plans, and some were retired or added when the work plan was updated which may have decreased continuity and consistency in the available data. In addition, the target group comprises a mix of individuals and institutions; trainings, products and webinars are varied, and specific goals and objectives add to the challenge of measuring sustained long￾term outcomes or impacts on individual and organizational capacity. The last semi-annual report of September 2021 (which does not include the final six months of implementation) found that 15 out of the 19 indicators (78%) met or exceeded their cumulative targets. Some activities were still ongoing at the time of the evaluation and so full initiative data is not included in this analysis. READY encountered barriers that impacted performance and achievement of results in relation to enhancing organizational and individual capacity to respond effectively to major disease outbreaks. Across the majority of KIIs and the desk review, the evaluation identified several challenges. The initial broad strategic direction, along with turnover of leadership and staff, resulted in delays in READY positioning and aligning itself within disease response sector mechanisms. The initial broadly defined purpose and mandate of READY was the main issue, and many consortium partners stated that an earlier narrowing of focus would have benefitted READY’s efficiency. In addition, staff turnover created a cascade of delays to key meetings, and double hatting of READY personnel who may not have had the skillset or time to carry out their responsibilities. Cross-agency collaboration was impacted by competing mandates, different work cultures, and time zone difficulties, with some participants working across a 16-hour time difference. The impact of COVID-19 on READY resulted in new working methods, work activities, and new ways of collaboration and engagement; overall, COVID-19 was a challenge that propelled READY into valued activities and approaches. With reference to Objective 1, READY has continuously participated in key global operational and technical coordination platforms which has allowed READY to Objective 1: Improve coordination, both operational and technical, between global and regional disease response platforms and initiatives. Objective 2: Strengthen operational capacity of organizations to launch an infectious disease outbreak response more efficiently Objective 3: Improve technical preparedness and readiness across relevant sectors for infectious disease response in humanitarian settings Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 11 understand and work proactively to fill identified gaps as well as advocate for NGOs to be included in the conversation. The results of the evaluation team’s desk review and the interviews it conducted demonstrate that READY’s level of engagement expanded over time, with the result that it was represented on 42 different coordination taskforces, working groups, and global clusters. 6 READY engaged with humanitarian and outbreak response coordination platforms, and COVID-19 task teams as well as with specific technical sector clusters and working groups. This increasing engagement helped to address gaps in cross-cutting areas such gender, RCCE, and mental health and psychosocial support. READY also co-led an RCCE Collective Service sub-group which has a focus on community engagement in low resource settings. Further, it effectively led GHC COVID-19 Task Team initiatives on ethical decision-making, development of priority health services packages, and identifying access barriers to GBV services during the COVID-19 response. Interviews showed that global engagement improved over time, resulting in a better understanding of the role READY can play to support organizations and coordination bodies. Leadership used this engagement to promote buy-in and entry points for feedback and dissemination of READY activities. Likewise, READY delivered products on more granular cross-cutting and transversal issues such as child protection, water, sanitation, and hygiene (WASH), and maternal and new-born health and infant feeding. Further, under Objective 1, READY completed a COVID-19 Landscape Analysis study, a follow-up COVID-19 Coordination Report and developed Outbreak Coordination Guidance for NGOs which identified more ways in which NGOs might address gaps and enhance their role within the public health and humanitarian architecture.7 Together with the deliberate inclusion of Outbreak Coordination Guidance into Operational Readiness Trainings, READY’s actions had the aim of demystifying outbreak coordination for NGOs but it is unclear to what extent this objective was achieved. With reference to Objective 2, READY efficiently adapted activities through the Year 3+ work plan which better reflected emerging needs and priorities resulting from COVID-19. Collective trainings, tools, and webinars, as well as the increased surge capacity of consortium partners, indicate a strengthened operational capacity. Analysis of interviews, training data and its online survey shows that READY enhanced the operational capacity of NGOs including but not limited to their responses to COVID-19. Key activities were: • Global Outbreak Preparedness Planning and COVID-19 Readiness Workshops • Epidemiology and Modelling of Infectious Diseases in the Humanitarian Setting training • Communities in Humanitarian Settings: COVID-19 Micro-trainings 6 as of Sept 2021 7 Ongoing at the time of writing. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 12 • Operational Readiness Training Program for Major Disease Outbreak Response • Outbreak READY!: A Digital Readiness and Response Simulation • Training and support for surge rosters. Trainings From the online survey, 96%–100% of respondents (n=100) who participated in one of the five major trainings expressed agreement that READY trainings are relevant in building outbreak response capacity.8 In these five major trainings the range of participants who agreed that it increased their knowledge and that they have or will apply learnings from the trainings in their work was between 84%–100%. The most relevant training was Epidemiology and Modeling of Infectious Diseases. While all trainings scored high, the trainings that contributed most to enhancing knowledge were the Communities in Humanitarian Settings COVID-19 Micro-Trainings and the COVID-19 Readiness workshops. Some training data indicated a low level of knowledge increase and retention. Participants in the Epidemiology and Modeling training showed a 7.9% improvement in scores with a high pre-test score averaging 80.8% and post-test scores averaging 88.7%. Micro-Training participants showed an average improvement of 5.4% with a pre-test score averaging 62.4% and post-test scores averaging 67.8%. Despite this, 96% provided positive feedback regarding the quality and content of the training package and average satisfaction with training was 4.5 out of 5 stars. From the Micro-Trainings follow-up survey, 71% of respondents reported feeling completely confident in their capacity to strengthen their COVID-19 response workplan, capacity to implement community health, WASH, and RCCE interventions for a COVID-19 response, and capacity to design linkages across community health, WASH, and RCCE for a COVID-19 response. READY’s support in improving surge roster capacity resulted in deployments, mostly related to COVID-19 responses, across all consortium partners. 9 This was perceived as relevant and effective in enhancing the operational response capacity of the consortium partners. Save the Children effectively supported online roster￾8 The five major trainings were: COVID-19 Readiness Workshop; Epidemiology and Modeling of Infectious Diseases in Humanitarian Settings; Global or Country-Specific Outbreak Preparedness Planning Workshops; Operational Readiness Training Program for Major Disease Outbreak Response; Communities in Humanitarian Settings COVID-19 Micro-Trainings. 9 JHU CHH and JHU CCP did not have any deployments during the award period. "The key thing I learned was risk assessment, meaning that you identified the risks, the probability of occurrence and the consequences so that you can focus on a few areas of importance, rather than try to respond to everything." Key stakeholder: Cameroon "The training was really designed around the needs and the structure of the organization: this was very useful. They really took the time to get to know us—to understand our capacity, our vision, what we want to do and what kind of role we want to play." Key stakeholder: Cameroon Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 13 trainings hosted through Kaya Connect and UK-Med consolidated Standard Operating Procedures (SOP) for communicable diseases within all WHO Emergency Medical Team health services. All SOPs were validated by WHO Emergency Medical Team and this effectively mandated UK-Med to respond to infectious diseases in humanitarian settings. Across the consortium there are 528 people on the roster as of September 2021, and 99 deployments. Webinars Webinars provided the best opportunity for applying knowledge.10 Between 75%–100% of READY webinar respondents (n=100) stated that the webinars were relevant in building operational or technical capacity. Between 72%–87% of respondents stated the webinars enhanced individual knowledge. A total of 31% of respondents strongly agreed and 46% agreed they have or will apply learnings from the webinars in their work. For the first COVID-19 webinar series the attendance analytics recorded a total of 3,824 attendees across 14 webinars (1,497 unique participants). These 14 webinars had an average attendance of 273 individuals who came from a combined total of 260 unique organizations, institutions, and government agencies, including 77 NGOs. The second series had slightly greater average reach with an average of 294 individuals per webinar for a total 1,177 attendees across four webinars. In this second series participants represented 271 unique organizations, institutions, and government agencies. This included 66 unique NGOs and participation from over 70 countries. The increase in the second series was most likely due to a greater focus on COVID-19’s implications in humanitarian settings. Website, knowledge hub and humanitarian sector training platforms Assessing the effectiveness of the READY website, despite the small sample, 90% (n=79) of those who engaged expressed agreement that it is a relevant source of information to enhance capacity-building in major disease outbreaks; 79% stated that it increased their knowledge; and 82% stated that they have or will apply information from the READY website in their work. Website page views of key knowledge products like the COVID-19 RCCE Toolkit for Humanitarian Actors (RCCE Toolkit), the COVID-19 Micro-Training pages, and the Integrated Framework reached a cumulative total of 18,674 by September 2021. In addition, interviews with external stakeholders indicated that READY’s knowledge hub, launched in December 2019, provided discussion forums and tools to help line managers 10 Webinars included: COVID-19 & Humanitarian Settings: Exploring the Controversial Issues Webinar Series (October 2020–January 2021), single or multiple session; COVID-19 & Humanitarian Settings: Knowledge & Experience Sharing Webinar Series (April–July 2020), Barriers to Gender-Based Violence Health Services in Humanitarian Settings During the COVID-19 Response (February 2022); COVID-19 Vaccines for Marginalized Populations: Risk Communication and Community Engagement (October 2021); Strengthening Frontline Public Health Services During COVID-19: Introducing Innovative IYCF Tools for Health and Nutrition Workers (May 2021); Integrating Technical Sectors in a COVID-19 Response: A Framework and Expert Panel Discussion (May 2021); Operationalizing One Health to Support Humanitarian Sectors Outbreak Response (April 2021); Introducing Guidance for Alternative Care Provision during COVID-19 (Jan-Feb 2021); Maternal, New-born, and Reproductive Health in Emergencies (MNRHiE) and COVID-19: Successes, Challenges, and Next Steps (December 2020); RCCE: Perceptions, Misinformation, and Concerns in African Countries in the Time of COVID-19 (April 2020). Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 14 integrate WASH, RCCE, and Community Health.11 Active discussion forums came from two RCCE communities of practice: the East and Southern Africa Regional RCCE Working Group and the Middle East, North Africa, and Eastern Mediterranean Inter-Agency RCCE Working Group. In addition, and to ensure effective training dissemination, some READY deliverables were embedded into other humanitarian sector training platforms such as Kaya Connect and Save the Children’s COMPASS. 12,13 READY trainings, webinars, and website effectively reached target audiences across global, regional, national, and local areas due to engagement and consultation activities but also through READY’s open access resources. Survey results demonstrate that the decentralization of capacity-building was useful in regions where outbreaks occur. A total of 50% of survey respondents (n=212) reported working in Africa and 20% worked in the Asia Pacific region; 52% worked for an international NGO and 11% worked for a national NGO. With reference to Objective 3, READY has developed effective technical guidance notes and tools to enhance preparedness and readiness across relevant sectors. READY’s engagement and leadership with relevant Inter-Agency Standing Committee (IASC) clusters and working groups as well as the newly formed COVID-19 sub-working groups and task teams resulted in the delivery of relevant technical products that were led or supported by READY. Most informants stated that the collaborative mapping of specific gaps in the different sector groups promoted dissemination and is likely to have increased technical preparedness across the different key domains. READY produced or supported the production of a long list of technical guidelines, guidance notes and SOPs (see Annex 8) with key products being the COVID-19 Integrated Framework for Isolation and Quarantine as Non-Pharmaceutical Interventions against COVID-19, and technical guidance notes and tools on health, maternal and newborn health, nutrition, WASH, and child protection. The COVID-19 Integrated Framework aimed for a holistic, multisectoral, and integrated COVID-19 response but the available data was not adequate to show the extent to which it met its long-term objective of minimizing COVID-19 transmission in vulnerable communities. Despite this, most KII respondents stated that the technical guidance and tools were highly relevant in filling specific sectoral gaps—for example, the work on GBV; the Standard Operating Procedure for RCCE in Outbreak Response in Humanitarian Settings in the RCCE Readiness Kit; the RCCE Toolkit, the Child Protection Guidance Notes to the Child Protection Minimum Standards; the In-Country Child Protection and Health Integration Workshops in the Democratic Republic of Congo and Bangladesh. C. Cross-Cutting Themes Findings around Diversity, Equity, and Inclusion (DEI) show that READY did engage and create procedural space for these issues and that DEI was at the forefront of KII participants’ perceptions. While DEI was highlighted in the engagement mechanism, the key decision-makers and staff represented the Global North. The membership (as of 11 READY community discussion forums. https://community.ready-initiative.org/ 12 Online Training-Kaya Connect E-Learning. https://kayaconnect.org/ 13 Save the Children COMPASS for Health, Nutrition and WASH. https://compass.savethechildren.org.uk/ Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 15 October 2021) in the Core and Technical Advisory team was as follows: seventeen were based in the United States, twelve in Europe, two in Malaysia, and one each in Kenya, Ethiopia, and Jordan. COVID-19 impacted consortium members differently—MERCY Malaysia became heavily committed to a local deployment in response to the COVID-19 outbreak, and while this response met READY’s Objective 2 ambition towards deployment, it meant that MERCY Malaysia was less able to be deployed elsewhere. The formation of regional READY Advisory Groups was an effort to redress the relative lack of representation from the South; however, its decision-making capacity was only advisory, and a few KIIs suggested that these groups did not significantly influence strategic choices.14 As reported in the KIIs, READY did build in the principles of diversity and inclusion. The initial consultation phase that led to the Landscape Analysis deliberately included consideration of DEI in its methodology. As mentioned in a KII “They did very well. Well designed in that perspective, particularly ensuring the diversity, inclusion, and gender and integration.” Mentoring was carried out in READY’s surge roster program and diversity and inclusion were very evident in the process of selecting training participants. D. Long-Term Results and Sustainability This section explores READY’s contribution to long-term results and prospects for sustainability. The fourth evaluation objective unpacks the role of localization in providing effective and sustained local readiness capacity for outbreak response. READY contributed to strengthening the humanitarian system’s ability to provide valued results and it contributed to making the infectious disease response more resilient and adaptive in the face of changing circumstances. Given the size and complexity of the humanitarian system and its diverse stakeholders, as well as the short READY award period, it is difficult to measure overall long-term outcomes and impact so far. According to key informants, READY was perceived as delivering on its initial long￾term objectives and, while recognizing the abovementioned continuity issue in relation to reporting, READY met its agreed key indicators. Addressing technical gaps in partnership and working in close consultation with existing coordination platforms, as well as rolling out trainings and guidance documents through the broader stakeholder community facilitates achievement of READY’s longer-term ambitions. The Operational Readiness Trainings involving local responders was perceived as particularly relevant to assuring a local sustained capacity. In relation to increased surge capacity and trainings within the consortium, it is too early to suggest that this is a sustainable mechanism, although the technical expertise developed is considered highly relevant in the initial and acute phase of an outbreak response. While aiming for sustainability has not been at the 14 The READY Advisory Groups had the following functions: 1) Provide input and direction on key deliverables in READY’s Year 3+ workplan intended for rollout at the country level, including but not limited to the integrated framework, the operational readiness checklists and workshops and simulations; 2) Advise on engagement and coordination opportunities within priority countries, and socialize READY products, activities, and events within these structures and their own organizations; 3) Assist READY in identifying gaps, challenges, and trends in national-level NGO infectious disease responses and strategies; 4) Review READY work for cultural and national relevancy, ensuring the initiative remains fit-for-purpose across multiple different response landscapes and that it meets the needs of both international and local response NGOs. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 16 forefront of this phase of the READY initiative, in early January 2021 READY “began to think about READY’s legacy through initial sustainability planning.”15 READY developed and operationalized most of its commitments to localization, albeit late in the award cycle, and stakeholders found that there was opportunity for more ambition in this regard. READY adopted specific localization commitments into its design in January 2021. These commitments were perceived as a good start but could have been more ambitious. Future commitments should be better aligned to current global localization commitments (WHO, 2022). Among the KIIs there was a lack of agreement about READY’s initial delivery in relation to building management capacity, administrative capacity, and supporting funding opportunities for national NGOs, although these were later included as key parts of the Outbreak Preparedness Planning Workshops. Some KIIs indicated that the BHA award deliberately emphasized the readiness of INGOs which meant that engaging and sourcing feedback from field-level outbreak responders was limited. A few stakeholders stated that the essence of localization is not fully understood within the sector and that READY is well placed to increase localization ambitions, including by advocating for strengthening local capacity in global platforms and to donors. Establishing READY Advisory Groups in the Africa and Asia regions, the translation of READY products, and especially the pilot implementation of the Operational Readiness Training Program for Major Disease Outbreak Response for smaller, local NGOs in Turkey, Sudan and Cameroon were highly valued in expanding reach and strengthening response capacity of local responders in the long term. The high NGO demand for this training program indicated a large appetite for this type of training and together with localization commitments, READY saw an opportunity to engage and strategize with local players. 5 Conclusions and Recommendations Conclusion 1 The relevance and coherence of READY was dependent on its success in positioning itself within the disease outbreak response sector and on READY’s understanding of how to engage with the global and local outbreak response architecture. There is evidence that READY is aligned with international priorities, particularly where there were strong partnerships, for example with WHO, GHC, and GOARN. While the initial goal was broad and ambitious, this came with some uncertainties around defining specific activities. The landscape analysis, and the high level of continuous partner engagement with clusters and other global actors were instrumental in identifying key sectoral gaps, developing appropriate activities, and disseminating products that were complementary to what already existed in the outbreak response sector. Recommendations in relation to conclusion 1: 1.1 Define the long-term objective of READY for the next 5 years. Consider the most value-adding role within the outbreak response ecosystem that is achievable. This evaluation finds that the most pertinent objective would be a balance of localization￾directed skill development in high-risk locations, global actor surge capacity maintenance, 15 READY Semi-Annual Program Report (October 2020 – March 2021). BHA and READY discussed sustainability prospects in a three-day meeting in January 2021. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 17 and integration of training. This work should be done in close collaboration with the cluster system, WHO and GOARN. Position READY to build up the competencies of local NGOs, support the buildup of the surge capacity of sustainably resourced INGOs, and build up the knowledge bank around PHEIC while addressing specific sectoral gaps. Assure the capacity to adapt to possible new epidemic/pandemic response architectures and maintain high levels of funding and operational flexibility. PHEIC and pandemic response architecture is now being restructured, based on lessons learned from COVID￾19. READY now has considerable knowledge to draw upon to contribute to this process. 1.2 Diversify funding. Assure buy-in from a broader range of funding partners to promote sustainability. The BHA and USAID have shown long-standing commitments to addressing emerging pandemic threats and PHEIC. However, the global community is at an inflection point where there is a growing consensus about the need for financing of preparedness and response (G20 HLIP 2021; IPPPR 2021). This commitment to preparedness will help READY to build on pandemic-linked elements that are aligned with READY’s original objectives and draw support from other agencies in extending its reach. Conclusion 2 The deliberately broad but flexible design and implementation phase was favorable in allowing READY to adapt appropriately to the changing environment (i.e., COVID-19), to better define the target audience (NGOs), and to produce products that added the most value. This flexibility allowed for relevant programmatic pivots including the strengthening of NGO capacity which contributed to a better nuancing of READY’s strategic direction. The impact of the COVID-19 pandemic was significant and as a result, READY shifted its emphasis toward COVID-19 readiness. This pivot was seen as highly relevant and COVID-19 products, webinars, and trainings added significant value. In addition, they were provided at the onset of the pandemic and enhanced the readiness of the target audience. Both pivots also contributed to a greater nuancing of READY’s strategic ambitions. In addition to its COVID-19 products, trainings and webinars were perceived as relevant and valuable in the early stages of the pandemic. While the two pivots that occurred (the first being during the launch of READY and the second being the shift to COVID-19) were predominantly positive, they also resulted in delays, activity changes, and reduced activity continuity. Recommendations in relation to conclusion 2: 2.1 Keep the flexibility to react to novel outbreaks. Retain READY’s operational and financial (non-earmarked) flexibility for both system-building work and field-based response work. The READY consortium’s mix of implementation, academic, policy, and training skills permits the transfer of experience-based learning in a context of highly credible consortium partners. Conclusion 3 Evidence from interviews demonstrated that READY products, trainings, and webinars were seen as tailored to specific gaps and that this was where READY added most value (COVID-19, RCCE, Child Protection). The overall rating provided by training participants was very high. Over the course of the award period READY achieved its planned outputs across its three objectives, addressed gaps at global and local levels across different Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 18 sectors, and worked in partnership at all levels with a multitude of relevant stakeholders. Earlier engagement with higher level partners and ensuring a greater continuity at the level of human resources could have enhanced READY’s efficiency in earlier defining its target audience and where it would add most value. Due to the limitations of outcome reporting, it was not possible through the desk review to establish how successful READY was in enhancing operational and technical capacity of the target audience. Recommendations in relation to conclusion 3: 3.1 Assess the relevance of and develop competency-based frameworks for training and to sustain knowledge for individuals. Investigate developing competency-based frameworks with cluster leads; consider scaling up the training of trainers using novel regional mechanisms like ASEAN Coordinating Centre for Humanitarian Assistance on disaster management, Africa CDC, and the WHO Public Health Emergency Operations Centre Network; mentoring and follow up; build in routine catch-up and refresher sessions; in case of an outbreak, link trained local partners into response mechanisms and funding arrangements based on their achievement of core competencies.16,17 These actions will help increase the transfer of knowledge into practice for many NGOs, which have smaller administrative structures, and it will permit greater coordination of actors within outbreak settings where performance is required. 3.2 Ensure that capacity is maintained at individual and institutional levels. Target INGOs to conduct annual refresher simulations that will maintain operational readiness. READY should link to training modules around non-technical (soft) skills such as coordination, leadership, management, negotiation, conflict resolution, planning, resource management, and partnership and relationship maintenance, as there are many existing technical trainings. This is a recommendation in the Review of Outbreak Preparedness Training and Gaps Analysis. Conclusion 4 Desk review evidence shows that localization became an ambition mid-period and READY integrated high-quality capacity-building programs for a selected group of local responders. While implemented late in the initiative’s timeline, localization efforts have been considered especially relevant and value-adding—potentially the way forward for a future scale-up. READY’s open access resources effectively reached target audiences not only at global and regional, but also national and local levels. Establishing READY Advisory Groups in the Africa and Asia regions, and especially the pilot implementation of the Operational Readiness Training Program for Major Disease Outbreak Response for smaller and local NGOs in Turkey, Sudan and Cameroon were highly valued in expanding reach and strengthening response capacity of local responders in the long term. Recommendations in relation to conclusion 4: 16 Africa CDC. https://africacdc.org/download/africas-emergency-response-workforce-rapid-response-team￾directory/ 17 AHA center is the ASEAN Coordinating Centre for Humanitarian Assistance on disaster management, Jakarta, Intergovernmental organization. Core functions are coordination, disaster information management, knowledge and outreach, resource management. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 19 4.1 Expand existing localization commitments in ambition and coverage. In addition to READY’s localization commitments and in relation to the Grand Bargain include the following action points: (1) support responders in fragile contexts and armed conflicts; (2) support localization by having an aggregated project target of at least 25% of READY’s funding directed to local and national responders capacity-building and resourcing; (3) emphasize to READY’s donor(s) the benefit funding READY-trained local responders in the event of an outbreak; (4) beyond the regional READY Advisory Groups, ensure that local responders are represented in READY strategic planning and implementation; (5) use regional capacity to implement training of trainers and refresher trainings to reduce unnecessary airmiles by trainers; (6) ensure that local and national responders are competent to access international funding for outbreak response, such as the UN-led country-based pooled funds, the Central Emergency Response Fund, the IFRC Disaster Relief Emergency Fund and NGO-led and other pooled funds; (7) promote awareness of key resources relating to novel approaches that can increase NGO engagement, such as the Joint Intersectoral Analysis Framework and the WHO Public Health Emergency Operations Centre Network.18,19 4.2 Balance localization with global response capacity. Continue supporting localization but undertake a mapping of actual capacity to help clusters ensure that they have a workable surge capacity to call upon. In tandem with the WHO Health Cluster and Emergency Teams, produce a resourcing projection so that the commitment to surge capacity is announced, resource needs are measurable, and resources are available for deployment. Commit to formal partnership agreements with GHC, GOARN, WHO, IASC etc.—be a provider of last resort to them, filling key critical gaps at global and regional levels. Conclusion 5 There are uncertainties about how READY’s outputs translate into mid-to-long-term outcomes or impacts due to the complexity of the design, the pivots that occurred, and to a smaller extent changes in work plans and indicators. These factors, along with READY’s relative youth as an initiative, make it impossible to draw conclusions about its sustained results. There are concerns about how improved capacity can be maintained in the long term and how READY’s outputs can be cascaded or effectively disseminated to achieve the biggest possible reach in the long term. While knowledge management in the global humanitarian sector is complex, READY joined up with numerous other knowledge hubs with the aim of increasing reach and retaining knowledge within the NGO and global infectious disease sector. Considerable thought went into READY’s dissemination and implementation strategy; it achieved dissemination though existing communities of practice and maintained a reference website as a knowledge library of the initiative. Recommendations in relation to conclusion 5: 5.1 Extend high-level engagement with regional bodies and non-NGO partners to increase READY’s reach. This should include engaging with the European Health 18 Charter for Change: Localisation of Humanitarian Aid. https://charter4change.org/ 19 The Grand Bargain calls for 25% of funding to go directly to local responders. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 20 Emergency Preparedness and Response Authority, Africa CDC, ASEAN Coordinating Centre for Humanitarian Assistance on disaster management, Red Cross and Red Crescent regional offices, and IFRC expertise to minimize duplication and increase training harmonization and the development of consensus-based definitions. 20 Review READY partners’ ambitions regarding outbreak surveillance and laboratory expertise/capacity as this field has increased in scale significantly since COVID-19 (African Development Bank, 2021). Conclusion 6 By 2022, READY has put itself in a strong position to positively influence international coordination platforms: it has increased its range of partnerships and the operational capacities of NGOs. READY has increased the human resource pools in several technical domains, particularly around RCCE. The production of technical guidance has contributed to the sector’s outbreak response capacity; however, the new COVID-19 world is different from the pre-READY landscape and localization has now also become a major cross￾cutting theme. The READY has built up core partners’ capacity to contribute and to achieve the initial aims of the initiative to improve coordination, and to strengthen organizations’ operational capacity and technical preparedness. Recommendations in relation to conclusion 6: 6.1 Continue to expand engagement and build reputation. Remain aligned and complementary to the global cluster mechanism and get involved in the International Treaty on Pandemic Prevention, Preparedness and Response—a historic and global accord. The READY consortium already has members who have established academic and implementation competencies, who could engage through READY with the intergovernmental negotiating body established by WHO for this purpose. 21 READY’s engagement with many local partners offers opportunities to facilitate a broader consultation process to ensure that local NGO actors contribute. Outside of this treaty proposal it will be necessary to follow up and engage in ongoing discussions with WHO on building and increasing capacity of human resources in emergency response as this is likely to be a shared ambition of READY and WHO in relation to human resource competency. 22 Cross-cutting the conclusions Make use of dissemination and implementation science (DIS).23 To ensure that READY products and future ambitions achieve the greatest possible impact, invest in DIS (Shelton et al., 2020). To increase the impact of innovation implementation, adopt pilot 20 HERA European Health Emergency preparedness and Response Authority—next step in completing the European Health Union. This body is set up to strengthen Europe’s ability to prevent, detect, and rapidly respond to cross-border health emergencies by ensuring the development, manufacturing, procurement, and equitable distribution of key medical countermeasures. IFRC CF—there are links with the IFRC’s CEA/RCCE department, but it is unclear to what extent RCCE’s Readiness kit is linked up with the groundbreaking CF mechanisms rolled out by the IFRC in the latest DRC EVD epidemics (300,000 individual verbatim records of feedback received by over 800 RC volunteers doing field work in 29 health zones). 21 WHO. (2021, December 1). https://www.who.int/news/item/01-12-2021-world-health-assembly-agrees-to￾launch-process-to-develop-historic-global-accord-on-pandemic-prevention-preparedness-and-response 22 There is no formal announcement as yet on this. 23 Center for Disease Control. https://www.cdc.gov/pcd/issues/2018/18_0525.htm Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 21 and scale-up strategies,24 so that the READY consortium can benefit more people and foster policy and program development on a lasting basis. To increase the impact of dissemination, develop a rigorous dissemination strategy/campaign and prioritize reputable existing channels using free and unfettered access to READY products. This will include use of major massive open online course platforms (e.g., EdX, Coursera, FutureLearn), UN agency and cluster specific domains such as OpenWHO, INGO large actor internal platforms such as Médecins Sans Frontières Tembo, the IFRC Learning platform, and humanitarian and development course aggregators such as Disaster￾Ready.25 Consider investigating novel initiatives that complement existing competencies for outbreak response such as the London School of Hygiene & Tropical Medicine Epiverse initiative, or the ambitions of regional humanitarian organizations such as the ASEAN Coordinating Centre for Humanitarian Assistance on disaster management, and Africa CDC, who have shown a greater focus on outbreak response. 24 ExpandNet. https://expandnet.net/ 25 OpenWHO. https://openwho.org/ Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 22 References African Development Bank. (2021, June 21). Covid-19: African countries win the race against time with testing laboratories. African Development Bank Group. https://www.afdb.org/en/news-and-events/covid-19-african-countries-win-race￾against-time-testing-laboratories-44228 Active Learning Network for Accountability and Performance in Humanitarian Action. (2015). The state of the humanitarian system 2015. London: ALNAP/ODI. Active Learning Network for Accountability and Performance in Humanitarian Action. (2018). The state of the humanitarian system 2018. London: ALNAP/ODI. Altare, C.K.V. (2019, September). Infectious disease epidemics in refugee camps: a retrospective analysis of UNHCR data (2009–2017). Journal of Global Health Reports, 3. Baker, R.E., Mahmud, A.S., Miller, I.F., et al. (2022). Infectious disease in an era of global change. Nature Reviews Microbiology, 20, 193–205. https://doi.org/10.1038/s41579-021-00639-z Barbelet, V.D. (2021). Interrogating the evidence base on humanitarian localisation: a literature study. London: ODI. Caballero-Anthony, Mely, Cook, A.D.B., & Chen, Christopher. (2021). Knowledge management and humanitarian organisations in the Asia-Pacific: practices, challenges, and future pathways. International Journal of Disaster Risk Reduction, 53, 102007. https://doi.org/10.1016/j.ijdrr.2020.102007. Castellarnau, M.S. (2018). Bridging the emergency gap. Barcelona: MSF OCBA. Chen, Qiulan, et al. (2021, December 1). Rapid and sustained containment of Covid-19 is achievable and worthwhile: implications for pandemic response. British Medical Journal, (Clinical research ed.), 375 e066169. doi:10.1136/BMJ-2021-066169 Crawford, N.H.K. (2021). The Democratic Republic of Congo’s 10th Ebola response: lessons on international leadership and coordination. London: Humanitarian Policy Group, ODI. Cruz, Margarita, and Adam Schmitt. (2019). Strategy and knowledge management in humanitarian organizations. In S.U. Villa (Ed.), Decision-making in humanitarian operations. Cham: Palgrave Macmillan. G20 High Level Independent Panel (HLIP) on Financing the Global Commons for Pandemic Preparedness and Response. (2021). A Global deal for our pandemic age: report of the G20 HLIP. Geneva: G20. Independent Panel for Pandemic Preparedness & Response (IPPPR). (2021). COVID-19: make it the last pandemic. Geneva: IPPPR. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 23 Kapiriri, Lydia, et al. (2022, March). Priority setting and equity in COVID-19 pandemic plans: a comparative analysis of 18 African countries. Health policy and planning, 37 (3), 297–309. https://doi.org/10.1093/heapol/czab113. McKay G., Baggio, O., Camara, C.A., et al. (2022).’The response is like a big ship’: community feedback as a case study of evidence uptake and use in the 2018– 2020 Ebola epidemic in the Democratic Republic of the Congo. BMJ Global Health, 7: e005971. doi:10.1136/ bmjgh-2021-00597 Marani, M.K., Katul, G.G., Pan, W.K., & Parolari, A.J. (2021). Intensity and frequency of extreme novel epidemics. Proceedings of the National Academy of Sciences of the United States of America, 118. Mills, C.E., Robins, J.M., Bergstrom, C.T., & Lipsitch, M. (2006). Pandemic influenza: risk of multiple introductions and the need to prepare for them. PLoS Med, 6 (3):e135. Moon, S., et al. (2017). Post-Ebola reforms: ample analysis, inadequate action. British Medical Journal, 356(380). doi: https://doi.org/10.1136/bmj.j280 Raftery, P., Hossain, M., & Palmer, J. (2021). An innovative and integrated model for global outbreak response and research—a case study of the UK Public Health Rapid Support Team (UK-PHRST). BMC public health, 21(1), 1378. https://doi.org/10.1186/s12889-021-11433-0 (2021). 17. Robillard, S.A. (2021, March 27). Localization: a “landscape” report. Boston: Feinstein International Center, Tufts University. Grand Bargain Localisation Workstream. https://reliefweb.int/report/world/localization-landscape-report Roca, I. (2015, Apr 16). The global threat of antimicrobial resistance: science for intervention. New Microbes New Infectious, 6, 22–9. doi: 10.1016/j.nmni.2015.02.007. Rosen, J. (2021, 2 August). Pandemic upheaval offers a huge natural experiment. Nature. https://www.nature.com/articles/d41586-021-02092-7 Schwerdtle, P., De Clerck, V. & Plummer, V. (2017). Survivors’ perceptions of public health messages during an Ebola crisis in Liberia and Sierra Leone: an exploratory study. Nurs Health Sci., 19(4), 492–97. doi: 10.1111/nhs.12372. Shelton, R., et al. (2020). What Is dissemination and implementation science?: an introduction and opportunities to advance behavioral medicine and public health globally. Int.J. Behav. Med, 27, 3–20. https://doi.org/10.1007/s12529-020- 09848-x Smith, K., et al. (2014). Global rise in human infectious disease outbreaks. Journal of the Royal Society, 11(101), 20140950. https://doi.org/10.1098/rsif.2014.0950. Start Network. (2017). The future of humanitarian surge. London: The Start Network. OCHA: https://www.unocha.org/our-work/coordination/surge-capacity Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 24 Sullivan, M. (2022, January 13). Opinion: why a ‘grand bargain’ on localization keeps falling short. Devex Inside Development Humanitarian. https://www.devex.com/news/opinion-why-a-grand-bargain-on-localization￾keeps-falling-short-102389 Webster, R.G. (2004). Wet markets—a continuing source of severe acute respiratory syndrome and influenza? Lancet, 363(9404), 234–36. doi:10.1016/S0140- 6736(03)15329-9. World Health Organization (WHO). (2015). Sixty-eighth World Health Assembly: Resolutions and decisions annexes. Geneva: WHO. https://apps.who.int/gb/ebwha/pdf_files/WHA68-REC1/A68_R1_REC1- en.pdf#page=37 WHO. (2018). Risk communication and community engagement preparedness and readiness framework: Ebola response in the Democratic Republic of Congo in North Kivu. WHO IRIS. https://apps.who.int/iris/bitstream/handle/10665/275389/9789241514828- eng.pdf?sequence=1&isAllowed=y WHO. (2020). Pandemic Influenza Preparedness Framework: Progress Report, 1 January – 30 June 2020. Geneva: WHO. WHO. (2021). From worlds apart to a world prepared: Global Preparedness Monitoring Board report. Geneva: World Health Organization. WHO. (2022). Why does localization matter in health emergency response? Geneva: WHO. WHO. (2022, February 9). Vaccine manufacturing workshop for SEAR & WPR. WHO Local Production and Assistance Unit (LPA). https://www.who.int/news/item/09-02- 2022-vaccine-manufacturing-workshop-for-sear-wpr WHO. (2022, February 14). GPMB calls for a renewed global commitment to ‘six solutions for a safer world’ in 2022. Global Preparedness Monitoring Board. https://www.gpmb.org/news/news/item/14-02-2022-gpmb-calls-for-a-renewed￾global-commitment-to-six-solutions-for-a-safer-world-in-2022 Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 25 Annexes Annex 1: Terms of Reference Consultant Scope of Work: READY Evaluation Timeframe: December 2021–March 2022 Level of Effort: Up to 35 working days Location: Remote—available to work across various time zones in the Americas, Middle East, Africa I. Background In 2018, the United States Agency for International Development’s Bureau for Humanitarian Assistance funded Save the Children to lead a three-year initiative to augment global capacity to respond to major disease outbreaks. Through a consortium of partners, the READY initiative supplements existing efforts to strengthen coordination between global humanitarian outbreak structures and operational organizations responding to outbreaks. READY is building and retaining capacity among operational consortium members, non-governmental organizations (NGOs), and other stakeholders to respond to major outbreaks more quickly and effectively, including COVID-19, with an integrated approach that addresses holistic needs of affected communities—with community engagement and communications at the center. For more information on READY’s work, please visit https://www.ready-initiative.org/. Over the life of the initiative, READY has carried out routine project monitoring and activity specific evaluations for key activities, such as the Outbreak Preparedness Planning/COVID-19 Readiness Workshops, the COVID-19 Micro-Trainings, and the Epidemiology and Modeling of Infectious Diseases in the Humanitarian Setting training series. Additionally, lessons learned from internal, activity-specific after-action reviews and external participant feedback have been used to adapt and strengthen READY approaches and activities/events. As READY approaches the end of the initiative in March 2022, we are seeking a consultant to lead on an evaluation of READY’s work as a whole. II. Evaluation Purpose The purpose of this evaluation is to identify to what extent engagement in READY activities and events successfully augmented NGO readiness and capacity for outbreak response. READY wants to identify what we did well, what we can improve on, and where we can add the most value for future projects. III. Evaluation Type READY plans to carry out an external, mixed methods performance evaluation to review how well the initiative achieved its objectives, identify challenges faced, and highlight lessons learned and best practices for future programs of a similar nature. Though the methodology will be finalized with the support of the selected Consultant, it is anticipated that the evaluation will largely rely on qualitative methodology with a focus on interviews and discussion with key stakeholders. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 26 IV. Evaluation Questions READY is interested in understanding the relative effectiveness of its various activities in strengthening NGOs’ ability to respond to outbreaks, within each of the project’s three objectives: 1) To improve I/NGOs’ operational and technical coordination with critical disease coordination platforms, 2) To strengthen the operational capacity 2 of organizations to more efficiently launch an integrated humanitarian response to infectious disease outbreaks, and 3) To develop effective technical preparedness and readiness across relevant sectors for infectious disease response in humanitarian settings. To evaluate the current activities under READY, the evaluation will explore the effectiveness of the overall project in achieving its stated goal. The key question this evaluation seeks to answer is “Did organizations or individuals who engaged in READY activities and events (trainings, webinars, e-learning courses, etc.) improve their readiness to respond to major disease outbreaks?” Key themes to explore: • Were the chosen operational and technical activities the right ones to build organizational and individual capacity to respond to major disease outbreaks, and to what extent were they successful at doing so? • Which operational and technical activities were perceived as the most useful and relevant for improving organizational level NGO readiness to respond to disease outbreaks? • Did READY engage in the right global outbreak and humanitarian coordination structures at the right time? How did engagement in these mechanisms help/or not READY to achieve impact? READY also seeks to document lessons learned from support activities that can be used by READY or similar initiatives for future readiness support. Focus areas include: • Localization commitments • Structure and engagement of the READY Advisory Groups (RAG) and Strategic Advisory Group (SAG) • Engagement with and functionality of the READY Knowledge Management Hub V. Evaluation Methods The proposed methodology includes: ▪ Document review including review of donor reporting, after action reviews from READY activities, participant feedback, pre-and post- test scores, activity learning objectives, etc. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 27 ▪ Online survey targeting READY email subscribers (a global network of over 2,000 individuals who have subscribed to READY updates and/or engaged in READY trainings, events, and activities), stakeholders, and partners o Note: Given the nature of the survey and READY’s target populations, there is limited capacity to develop a scientific sample. This will be a non-probability sample using self-selection sampling. ▪ Qualitative interviews (key informant interviews and/or focus group discussions) with READY staff, consortium partners, advisory group members (RAG and SAG), external stakeholders who have engaged with READY’s work (webinars, capacity-building trainings, technical guidance development and operationalization, etc.), and coordination bodies (Health Cluster, RCCE Collective Service, etc.) The Consultant will work with the READY M&E Advisor and the READY Core Team to develop an evaluation plan and methodology at the start of the consultancy. READY can provide support and access to online survey platforms such as Survey Monkey, Google Forms or Kobo Toolbox. VI. Evaluation Findings Dissemination Plan READY will share findings from the evaluation with key stakeholders, and relevant partners. It will also upload the final report to the Development Experience Clearinghouse and READY’s website. A full dissemination plan will be 3 developed with the READY team and BHA and will include a final report and presentation of findings to key stakeholders. VII. Expected Deliverables & Timeline Recruiting and onboarding the Consultant will take place in final quarter of 2021. Data collection for the evaluation will take place in the final quarter of the program (Jan-March 2022). READY will aim to have the report finalized and disseminated by the end of March 2022. The Consultant’s deliverables and tentative timeline (subject to the commencement date of the evaluation) are outlined below. Key deliverables are in bold. The Consultant and the READY M&E Advisor will agree on final milestones and deadlines at the inception phase. Deliverables and Tentative Timeline Milestones Timeline The Consultant is contracted and commences work Dec 2021 The Consultant will facilitate a workshop with the relevant stakeholders at the commencement of the project to develop the inception report. Jan 2022 Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 28 The Consultant will submit an inception report including: ▪ evaluation objectives, scope, and key evaluation questions ▪ description of the methodology, including design, data collection methods, sampling strategy, data sources, and evaluation matrix against the key evaluation questions ▪ data analysis and reporting plan ▪ caveats and limitations of evaluation ▪ risks and mitigation plan ▪ ethical considerations including details on consent ▪ stakeholder communication and engagement plan ▪ key deliverables, responsibilities, and timelines ▪ resource requirements ▪ data collection tools (for feedback from stakeholders including the SAG and RAG) Once the report is finalized and accepted, the Consultant must submit a request for any change in strategy or approach to the M&E Advisor and Chief of Party. Jan 2022 The Consultant will finalize the data collection tools (including survey instrument and interview guide) and finalize the analysis and data management plan. Jan 2022 The Consultant will lead on data collection and analysis Feb 2022 The Consultant will share the Draft Report including the following elements: • Executive summary • Background description of the program and context relevant to the evaluation • Scope and focus of the evaluation • Overview of the evaluation methodology and data collection methods • Findings aligned to each of the key evaluation questions • Specific caveats or methodological limitations of the evaluation • Conclusions outlining implications of the findings or learnings • Recommendations Feb/Mar 2022 Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 29 Annex 2: Evaluation Matrix Evaluation Questions Evidence sources Analysis methods I DESIGN EQ 1: TO WHAT EXTENT IS THE READY PROGRAM RELEVANT IN STRENGTHENING DISEASE OUTBREAK COORDINATION, OPERATIONAL CAPACITY AND TECHNICAL PREPAREDNESS? Relevance Coherence Connectedness 1.1. How well-tailored was the READY program design to global disease outbreak response needs and gaps? Explain how? 1.2. To what extent did the design of the READY program involve program beneficiaries? (Note: link to EQ3) 1.3. To what extent was the READY engagement (technical & operational) aligned to global outbreak response strategies, policy, and initiatives and how? 1.4. To what extent are program objectives and activities aligned to donor policy and strategy? 1.5. How did the design embed principles of long-term outcomes, sustainability, and diversity & inclusions? (Note: link to EQ4) § Global strategy and policy on outbreak response, including COVID-19 § BHA strategy and policy on outbreak response, including COVID-19 § ToC § Online survey § KII’s and workshops § Triangulation and synthesis across the various sources of data II IMPLEMENTATION EQ 2: TO WHAT EXTENT AND HOW HAVE READY ACTIVITIES BUILT ORGANIZATIONAL AND INDIVIDUAL CAPACITY TO RESPOND EFFECTIVELY AND EFFICIENTLY TO MAJOR DISEASE OUTBREAKS? Effectiveness Efficiency Coordination Engagement 2.1. What were output/activities that were perceived as adding value? 2.2. What were enablers and barriers to effective and efficient implementation of these outputs/activities? 2.3. How adaptive was READY to the changing events, including the impact of Covid-19? 2.4. To what extent were the different partnerships and READY’s engagement in outbreak and humanitarian structures appropriate and effective (consortium, RAGs, SAGs, TAG’s, IASC clusters, GOARN)? § MEL framework § Indicator reporting § Program reporting § Action review reports § Communication exchanges § READY website § Budget case § Online survey § KII’s and workshops § Review of READY deliverables § Budget analysis § Triangulation and synthesis across the various sources of data Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 30 2.5. To what extent has READY made effective use of available resources to achieve the outputs (economy, flexibility) III LOCALIZATION EQ 3: TO WHAT EXTENT HAS THE READY INITIATIVE EFFECTIVELY DELIVERED ON LOCALIZATION? Cross-cutting Coherence 3.1. What is READY’s understanding of localization and how does this compare to global and donor localization commitments and strategies? 3.2. In what ways has READY operationalized localization and what were areas that involved internal change or advocacy? § Literature review; WHS 2016, Grand Bargain Localization Workstream commitments, NEAR Localization performance measurement framework, USAID Principles for Engaging Local Systems26 § KII’s and workshops § Document review § Triangulation and synthesis across the various sources of data IV RESULTS EQ 4: HOW DID READY OUTPUTS CONTRIBUTE TO SUSTAINABLE LONG-TERM OUTCOMES IMPROVING READINESS AND RESPONSE CAPACITY? Outcomes Coverage 4.1. What were the results of the READY program? (Contribution to outcomes) 4.2. To what extent were appropriate monitoring and evaluation mechanisms in place to measure results? 4.3. What are READY I strategies, activities, partnerships, and engagement events (positive or negative) that can inform the READY II phase? § MEL framework § Document review § Online survey § KIIs and workshops § Analyze contribution § Triangulation and synthesis across the various sources of data 26 USAID. https://www.usaid.gov/sites/default/files/documents/1870/LocalSystemsFramework.pdf. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 31 Sustainability 4.4. To what extent does the program provide prospects for sustainability27 (operational, financial, environmental)? 4.5. How has READY operationalized principles of diversity, inclusion, and equity? § USAID/BHA principles for sustainability28 § Online survey § KIIs and workshops § Triangulation V LESSONS LEARNED EQ5: WHAT WERE THE LESSONS AND BEST PRACITICES FROM READY PHASE I THAT CAN INFORM EADY PHASE II? 5.1. What are the main challenges and lessons learned from the response to the COVID-19 pandemic? § Online survey, KIIs, FGDs § Document review § Categorize lessons learned § Triangulation 27 USAID, Definition on Sustainability: sustainability refers to the ability of a local system to produce desired outcomes over time. Discrete projects contribute to sustainability when they strengthen the system’s ability to produce valued results and its ability to be both resilient and adaptive in the face of changing circumstances. 28 USAID/Bureau for Humanitarian Assistance emergency application guidelines. Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 32 Annex 3: Evaluation Question Guide QUESTION GUIDE—KEY INFORMANT INTERVIEWS READY Initiative—Global response to major disease outbreaks KEY INFORMANT INTERVIEW GUIDE: Interviewee code: __________________________________ Org/role: ________________ Interviewer: ___________________________________ Date: ___________________ § Thank you – Thank you for the interviewee and for your time today § Your name – My name is ___________, and I work as a consultant for Save the Children US. We are conducting an external endline evaluation for the READY Initiative. This was requested from UASAID/BHA and both for the purpose of accountability and learning. Are you aware of this initiative? § The main objectives of the READY initiative are threefold, namely to: (1) Improve coordination, operational and technical, between global and regional disease response platforms and initiatives; (2) Strengthen the operational capacity of organizations to more efficiently launch an integrated humanitarian response to infectious disease outbreaks; (3) Improve technical preparedness and readiness across relevant sectors for infectious disease response in humanitarian settings. § Purpose – I would like to do an interview with you. I have a semi-structured questionnaire and the main purpose of the interview is to hear your views on this program and the lessons learned that can be used for future READY programming. The evaluation will explore 3 areas of enquiry: 1/the relevance of the program design, 2/how effectively/efficient the implementation has been, and 3/ the results and sustainability prospects. § Confidentiality / consent to record – Would it be okay if I record this interview, just as a back-up to my notes? I will delete the recordings after I have finished typing and reviewing our notes. This interview is entirely confidential, and your name will be Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 33 removed from all interview transcripts and recordings, and quotations will not be attributed to any individual or organization. If you prefer for us not to use quotations from your interview at all, please let us know. § Duration – I expect the interview to take approximately 60 minutes – is that okay? If you do not have that much time let me know and I can make sure that I focus on the more important questions. Also, it’s absolutely fine if you do not know the answer to a question or prefer not to answer or the question is not relevant to your role – just let me know and I’ll adapt the interview accordingly. § Opportunity for questions – Do you have any questions before we start? Of course, you are also welcome to ask questions through the interview if something is not clear. Or if there are any elements that I have not come up with during the interview feel free to share them near the end. Evaluation Questions CP N G O’ s U N G oV A A - Please introduce yourself briefly including your background, job title, organization, role and how it relates to the READY program? - Probe: how long have you been in this role? What are your key areas of responsibility? The main area of focus? What were interactions or specific activities have you had with READY? (which sectors, local or national partner, advisory role, beneficiary)? X X X X X X X X X X I DESIGN EQ 1: To what extent is the READY program relevant in strengthening disease outbreak coordination, operational capacity and technical preparedness? Relevance Coherence Connectedness 1.1 How well-tailored was the design to global disease outbreak response needs and gaps? - Give examples? 1.2 To what extent did the design of the READY program involve program beneficiaries? Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 34 - Engagement & Participatory approaches? - Did baseline/landscaping inform the program and how? Do you have examples of this? - What were the priority needs that READY identified? 1.3 To what extent was the READY engagement (technical & operational) aligned to global outbreak response strategies, policy and initiatives and how? - Is READY unique in its approach and why? (yes/no). Any examples for complementarity or duplication? - How does READY fit in alongside global disease response platforms & initiatives (IASC, WHO/WHE, global clusters, CDC Afro, FAO, UNICEF, NGO’s). - Are there other platforms/initiatives that could have added value? What was the dual benefit of some of these engagements? 1.4 To what extent are program objectives and activities aligned to donor policy and strategy? - What are priority donor objectives? Can you give examples of good alignment? 1.5 How did the design embed principles of long-term outcomes, sustainability, diversity & inclusions? II IMPLEMENTATION OUTPUTS EQ 2: To what extent and how have READY activities built organizational and individual capacity to respond effectively and efficiently to major disease outbreaks? Effectiveness Efficiency Coordination Engagement 2.1 What were outputs/activities that were perceived as adding value? - How did these add value and at what level? - How did outputs/activities promote ‘‘integrated’’ outbreak response? - How did outputs/activities promote localization (link to EQ3)? 2.2 What were enablers and barriers to effective and efficient implementation of these outputs/activities? 2.3 How adaptive was READY to the changing events – leadership, funding, Covid-19 - ? - Provide examples? And how did READY mitigate some of these risks? - How did Covid-19 impact the program? Describe both positive and negative impact? Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 35 - Were there other unintended results (positive or negative)? PROBE: leadership, covid-19, funding flexibility, redesign of programming 2.4 To what extent were the different partnerships and READY’s engagement in outbreak and humanitarian structures appropriate and effective (consortium, RAGs, SAGs, TA’s, IASC clusters, GOARN, …)? - Which partnerships/engagement added value and why? In which activities? - What were barriers or enablers to effective coordinating the partnerships/engaging? - Were there missed opportunities around engagement and partnerships? How? With whom? 2.4 To what extent has READY made good use of available resources to achieve the outputs (economy, flexibility) - Were there opportunities to improve cost-efficiency? How? - Were the right resources purchased at the best possible price? Give examples? - How has budget flexibility promoted adaptation to the changing events? Give examples? III LOCALIZATION EQ 3: To what extent has the READY initiative effectively delivered on localization? Cross-cutting 3.1 What is READY’s understanding of localization and how does this compare to global and donor localization commitments and strategies? - How has the READY/BHA positioning on localization helped the project? - Was localization an agenda topic from the design phase? If not, what promoted it to become one? - Which activities/outputs promoted localization? Explain how? 3.2 In what ways has READY operationalized localization? - Describe outputs/engagement that strengthened local capacities? How? - What were areas that involved internal change or advocacy? - What were barriers and enablers in localization implementation? Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 36 PROBE: Localization principles integrated in outputs, inclusion of marginalized people in events, 25% of participants from local/gov institutions, reaching out to field-based audience, availability of languages, bandwidth technology access, need & demand coming from local partners, vendor solicitation and consultants from local partners in READY countries, IV RESULTS-IMPACT EQ 4: How did READY outputs contribute to sustainable long-term outcomes improving readiness and response capacity? Outcomes Coverage 4.1 What were the results of the READY program? - To what extent is there evidence that READY was able to promote ‘‘readiness’’ in response? - Was the result overall or also vis-a-vis specific needs/organizations, vulnerable populations, disease outbreaks (Covid-19), specific geography? - How will program beneficiaries (NGO’s, INGO’s, response platforms) carry forward learnings and resources? - What were the results of the READY program, from output to outcome? - How have partnerships & engagement resulted in integrated readiness and response? 4.2 To what extent were appropriate monitoring and evaluation mechanisms in place to measure results? 4.3 What are READY I strategies, activities, partnerships and engagement events (positive or negative) that can inform the READY II phase? PROBE: ToT strategies, improved coordination, technical preparedness, operational capacity, M&E, action reviews, Sustainability 4.4 To what extent does READY provide prospects for sustainability? - Financial sustainability? Operational sustainability? Environmental sustainability? - Were sustainability principles integrated as part of the design? - Give examples on how this was operationalized? Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 37 4.5 How has READY operationalized principles of diversity, inclusion and equity? - Were such principles embedded in the design? Give examples? - How have these been operationalized? Give examples? § The end: We have come to the end of the interview. Is there anything else you would like to add that you see relevant and we have not touched upon? - If you have relevant documents that can inform this evaluation you are welcome to send these via email. - Do you have other people in mind that could be relevant for this evaluation? § Thank you very much for your participation Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 38 Annex 4: Document Review 1. Available Technical Guidelines Assessment Report 2. READY Landscape and Consultations, Key Findings 3. READY Technical Narrative – Final 4. Progress Report Sep 2018 – March 2019 5. Review of Outbreak Preparedness Training and Gaps Analysis 6. READY Semi-Annual Program report April-Sept 2020 7. Progress Report April 2019 – Sept 2019 8. READY Semi-Annual Program Report Oct 2020 – March 2021 9. Progress report Oct 2019 – March 2020 10. READY Semi-Annual Program Report April-Sept 2021 11. Epi Modeling in Hum Settings Post-Training Report 12. READY COVID Workplan April – June 2020 13. READY COVID Workplan July – Oct 2020 14. READY Year 3+ Workplan Revised 4Dec20 15. COVID-19 Webinar Series Impact Report 16. READY COVID Workplan 17. READY Advisory Group TOR-Final 18. READY’s Commitments to Localization 19. READY Indicator reporting Maser 2021 20. READY Organigram_Oct2021 21. READY Products Updates Jan 2022 22. Global OPP Feedback Summary 23. E-mail Dear Travis and Lindsay 24. READY Advisory Group Membership – Asia 25. READY Advisory Group Membership as of July 2021 – Africa 26. SAG TOR_FINAL1 27. Copy of global OPP evaluations 28. Evaluation DRC OPP 29. Evaluation ESA OPP 30. Evaluation MEEE OPP 31. Evaluation Uganda Workshop 32. Evaluation WCA OPP 33. Global Feedback summary (1) 34. Indonesia OPP Evaluation 35. Lessons learned from Readiness Workshops Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 39 Annex 5: Level of Engagement 1. World Health Organization 2. Global Health Cluster 3. COVID-19 Task Team 4. Global Outbreak Alert and Response Network 5. GOARN Capacity-building and Training Program Working Group 6. Global Task Force for the Control of Cholera 7. United Nations Office for the Coordination of Humanitarian Affairs 8. Information Management Working Group 9. Early Warning Alert and Response (EWAR) working group 10. Global Health Cluster Information Management Task Team 11. RCCE sub-group on community engagement in low-resource settings 12. United Nations Children’s Fund Malawi 13. UNICEF-Nepal 14. Lebanese Red Crescent and the Hellenic Red Cross 15. CARE Malawi 16. United Nations High Commissioner for Refugees 17. UNICEF 18. International Organization of Migration 19. International Federation of the Red Cross 20. RCCE Collective Service’s task team 21. World Organization for Animal Health 22. Food and Agriculture Organization of the United Nations 23. International Union for Conservation of Nature 24. United Nations Environment Programme 25. WHO Expert Working Group on Biodiversity, Climate Change 26. One Health and Nature-based Solutions 27. OIE Ad Hoc Group on Disease Spill over and Wildlife Trade 28. International Union for Conservation of Nature 29. Kaya training platform 30. Save The Children’s COMPASS 31. Ecosystem-Based Adaptation in Humanitarian & Post-Disaster Contexts Working Group 32. Inter-Agency Working Group for Reproductive Health 33. Every Newborn Action Plan in Emergencies Working Group 34. IAWG Maternal Newborn Health Sub-Working Group 35. International Planned Parenthood Federation 36. CORE Group 37. London School of Hygiene and Tropical Medicine 38. IFE Core Group 39. Global WASH cluster 40. Alliance for Child Protection in Humanitarian Action Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 40 41. Child Protection Area of Responsibility 42. MHPSS Reference Group 43. Protection in Outbreaks Resource Package Advisory Committee 44. International Rescue Committee 45. One Health High-Level Panel 46. World Health Summit October 2021 47. Global Plan of Action for One Health 48. GHC COVID-19 Task Team’s Gender-based Violence Peer Group 49. United Nations Population Fund Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 41 Annex 6: Survey Questions Hello, we would like to invite you to participate in a short survey to gather your insights and feedback on the READY Initiative. This information will be used to evaluate READY’s work and help guide future work. The survey should take approximately 6-8 minutes to complete. Your participation in this survey is voluntary. You may choose not to participate. Your responses will be confidential and we do not collect identifying information such as your name, email address or IP address. We will do our best to keep your information confidential. All data is stored in a password protected electronic format. To help protect your confidentiality, the surveys will not contain information that will personally identify you. If you have any questions please contact____________________ Simply click ‘yes’ below to launch the survey. Sex Female Male Non-binary In which region(s) are you currently working in? (multiple answers permitted) Africa Asia/Pacific Latin America Europe Middle East North America Which of the following best describes your organization? Consortium Partner National, non-governmental organization International, non￾governmental organization Civil society organization UN agency Donor Government Research/academic institution/student Private sector Consulting firm Other (please specify) Which of the following best describes the level in which you work? Sub-national National Regional Global Other (please specify) Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 42 Which best reflects your current job function? Global or Regional Leadership / Director Country Director / Program manager Technical advisor UN cluster coordinators / Sector leads Operations/Logistics Other (please specify) What best reflects the technical area you currently work in? WASH/IPC Nutrition Food security/livelihoods Health Shelter Protection Other (please specify) Not applicable If other, please specify Which READY capacity￾building products or activities did you engage in? Select all that apply. (multiple options possible) Trainings (Operational Readiness training, Covid-19 micro trainings, Epidemiology and modelling of infectious disease, …) Webinars (COVID-19 series, …) READY Website (RCCE Toolkit, …) Other (please specify) None If Trainings, go to A If Webinars, go to B If Website, go to C If multiple choices, go to the multiple options of A or/and B or/and C If other, please specify. Thank you for your participation If none, what was the reason, thank you for your participation I was not aware of READY I had no time It was not relevant Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 43 to my work It was not accessible Other (Please specify) Training Which of the following trainings did you attend? Select all that apply. COVID-19 Readiness Workshop (in-person) Global or country-specific Outbreak Preparedness Planning Workshops (in￾person) Operational Readiness Training Program for Major Disease Outbreak Response (virtual and in-person) Epidemiology and Modeling of Infectious Diseases in the Humanitarian Settings (virtual) Communities in Humanitarian: Settings: COVID-19 Micro￾Trainings(virtual) Other (please specify) If others, please specify Do you agree or disagree with this statement, ‘the READY trainings are relevant in building capacity for major disease outbreak readiness and response in humanitarian settings’? Strongly agree Agree Neutral Disagree Strongly disagree If neutral and below, why not open question Do you agree or disagree with this statement ‘‘the trainings have increased my knowledge of major disease outbreak readiness and response’’? Strongly agree Agree Neutral Disagree Strongly disagree Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 44 Do you agree or disagree with this statement ‘I have or I will apply learnings from the trainings in my work’’? á effectiveness ñ Strongly agree Agree Neutral Disagree Strongly disagree What area(s) of the trainings were most useful? Open question Webinars Which of the following webinars/webinar series did you attend? Select all that apply. COVID-19 & Humanitarian Settings: Exploring the Controversial Issues Webinar Series (October 2020-January 2021), single or multiple session. COVID-19 & Humanitarian Settings: Knowledge & Experience Sharing Webinar Series (April – July 2020) Barriers to Gender Based Violence Health Services in Humanitarian Settings during the COVID-19 Response (February 2022) COVID-19 Vaccines for Marginalized Populations: Risk Communication and Community Engagement (October 2021) Strengthening frontline public health services during COVID-19: Introducing innovative IYCF tools for health and nutrition workers (May 2021) Integrating Technical Sectors in a COVID-19 Response: A framework and expert panel discussion (May 2021) Operationalizing One Health to Support Humanitarian Sectors Outbreak Response (April 2021) Introducing Guidance for Alternative Care Provision All options or combinations of options move to the next question Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 45 during COVID-19 (Jan-Feb 2021) Maternal, Newborn, and Reproductive Health in Emergencies (MNRHiE) and COVID-19: Successes, Challenges, and Next Steps (December 2020) RCCE: Perceptions, Misinformation, and Concerns in African Countries in the Time of COVID-19 (April 2020) Other (please specify) I do not remember Do you agree or disagree with this statement, ‘the READY webinars are relevant in building capacity in major disease outbreak readiness and response in humanitarian settings’? Strongly agree Agree Neutral Disagree Strongly disagree If neutral and below, why not (Open question) Do you agree or disagree with this statement “the webinars have increased my knowledge of major disease outbreak response in humanitarian settings’’? áeffectiveness ñ Strongly agree Agree Neutral Disagree Strongly disagree Do you agree or disagree with this statement ‘I have or I will apply learnings from the webinars in my work’’? áeffectiveness ñ Strongly agree Agree Neutral Disagree Strongly disagree What area(s) of the webinars were most useful? Open question Website Do you agree or disagree with this statement, ‘the READY website is a relevant source of information for Strongly agree Agree Neutral Disagree If neutral and below, why not Open question Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 46 building capacity in major disease outbreak readiness and response in humanitarian settings’ ? Strongly disagree Do you agree or disagree with this statement “the READY website has increased my knowledge of major disease outbreak readiness and response in humanitarian settings’’? Strongly agree Agree Neutral Disagree Strongly disagree Do you agree or disagree with this statement ‘I have or I will apply information from the READY website in my work’’? Strongly agree Agree Neutral Disagree Strongly disagree What area(s) of the READY website were most useful? Open question FOR ALL In the future, what additional resources would you like to see from the READY-Initiative? Open question Thank you very much for your participation. Your answers are very valuable for us. For more information on READY you can go to www.ready-initiative.org Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 47 Annex 7: Dedoose Analytical Codebook CODE 1. Strategic relevance of design a. Robustness at the start b. Changing events c. Uniqueness CODE 2.Relevance to needs a. Participation & engagement b. Baseline/landscaping c. Priority needs identified CODE 3. Technically & operationally aligned to global outbreak response a. Alignment to platforms/initiatives b. Alignment to donor policy and strategy CODE 4. Embed principles of long-term outcomes, sustainability, DEI CODE 5. Outputs/activities adding value? a. ‘‘Integrated’’ response b. Promote localization CODE 6. Enablers & barriers CODE 7.Adaptive management a. Others b. Covid-19 impact CODE 8. Unintended results (positive or negative)? CODE 9. Partnerships and engagement a. Partnerships/engagement added value b. Coherence & coordination in activities c. Duplication / gaps CODE 10. Localization a. Understanding of localization b. Operationalized localization CODE 11.Integration embedded in the design CODE 12.Embed principles diversity, equity & inclusion CODE 13.What were the results a. Appropriate monitoring mechanisms b. Activities added valued c. Prospects for sustainability CODE 14.Lessons learned Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 48 Annex 8: READY Produced and Supported products Over the life of the award, READY produced and contributed to several reports, guidance notes, and toolkits. Products noted with an asterisk (*) were pieces of work where READY provided significant technical support and input but were led or published by another actor. • 6-Step Process for Community Engagement (COVID-19) with Operational Tools • Analysis of global and regional networks and initiatives of key sectors • Barriers to Gender Based Violence (GBV) health services in humanitarian settings during COVID-19: A desk review from Cox’s Bazar (Bangladesh), Iraq & Northern Nigeria* • Child Protection Cox's Bazar: Confidentiality Guidance Note • Child Protection Cox's Bazar: Visual Referral Pathway • Child Protection DRC: Checklist for child-friendly isolation, quarantine, and treatment facilities • Child Protection DRC: Tip sheet on child-friendly vaccination (and testing/treatment) • Child Protection in Infectious Disease Outbreak: Annotated Bibliography • Co-existing with COVID-19: Calls to Action for a More Community-Centric Response* • Communicating with Communities in Epidemics and Pandemics: A Risk Communication and Community Engagement Readiness Kit (RCCE Readiness Kit) • Contact tracing in the context of COVID-19 * • COVID-19 Communication and Community Engagement Toolkit for Humanitarian Actors ("RCCE Toolkit") • COVID-19 COMPASS Modules • COVID-19 Coordination Report: Consultation Findings and Case Studies • COVID-19 Global Risk Communication and Community Engagement Strategy - interim guidance * • Essential Health Services: A Guidance Note How to prioritize and plan essential health services during COVID-19 response in humanitarian settings * • Ethics: Key Questions to ask when facing dilemmas during COVID-19 responses in humanitarian settings * • Guidance for Alternative Care Provision During COVID-19 • Guidance for Alternative Care Provision During COVID-19 (French and Arabic Translations)* • Health Cluster Survey Findings: Technical gaps and operational challenges in delivering COVID-19 response activities and maintaining essential health services in humanitarian settings • Infographics: Infant Feeding During Infectious Disease Outbreaks: A guide for policy makers* • Infographics: Infant Feeding During Infectious Disease Outbreaks: A guide for programmers* Evaluation of READY Initiative—Global Readiness for Major Outbreak Response 49 • Initial Food Security Response to the COVID-19 Outbreak: Survey Findings • Integrated Response Framework for Isolation and Quarantine as Nonpharmaceutical Interventions Against COVID-19 • IYCF Remote Counseling: How to support caregivers during infectious disease outbreaks and other settings eLearning course   • Landscaping of SBC and community engagement platforms • Making Connections: Stories of Integration in Outbreak Response • Making your Health Centre Child Friendly for Cox’s Bazar • Mapping and assessment of 41 training modules and 89 technical resources for RCCE, prior to the COVID-19 pandemic • Maternal and Newborn health services disruption during COVID-19: Literature Review* • Maternal, Newborn, and Reproductive Health in Emergencies (MNRHiE) and COVID-19: Adaptations, Successes, Challenge and Next Steps. An Expert Consultation • Micro-Training Package • Operational capacity assessment for RCCE • Operational Checklists for Global OPP Workshops and COVID-19 Readiness Workshops • Outbreak READY! A Digital Readiness and Response Simulation in English, French and Spanish • Planning Guide for RCCE during Shifting Public Health and Social Measures: With Safety Tips for Conducting Community Meetings • Practical Guidance for RCCE for Refugees, IDPs, Migrants and Host Communities Particularly Vulnerable to COVID-19 Pandemic • Promoting Collaboration Between Child Protection and Health Sectors in the Context of Infectious Disease Outbreaks: Stakeholder Consultations • RCCE Principles and RCCE SOP for Contract Tracers • Review of Outbreak Preparedness Training & Gaps Analysis • Risk Communication and Community Engagement Guidance on COVID-19 Vaccines for Marginalized Populations • Sexual and Reproductive Health and Rights during Infectious Disease Outbreaks: Operational Guidance for Humanitarian and Fragile Settings • Tips for Engaging People in Low Resource/Low Connectivity Settings Remotely and In-Person • WASH in Epidemics E-Learning Course • World Health Organization’s Operational guide for Engaging Communities in Contact Tracing*